Technology for Collision Avoidance Systems Advances

AI and proximity detection assist the operator in making decisions that prevent injury and damage to other equipment

By Steve Fiscor, Editor-in-Chief

During ideal conditions, operating equipment at a coal mine could be considered routine. Conditions can change rapidly though and sometimes vehicles and miners turn up in dangerous locations, like the path of oncoming equipment. The large haul trucks and wheel loaders that operate at surface mines have obstructed views. The shuttle cars tramming between the continuous miner and the feeder-breaker underground operate in low-light conditions on uneven ground. Hitting the brakes in a split second could be the difference between life and death.

In addition to safety-related concerns for miners, there is the possibility of damaging other equipment and structures. Just like modern automobiles, mine operators now can upgrade their equipment with similar systems to prevent accidents.

Once the systems are installed, operations and safety managers can review the data to determine the performance of the operators. Some systems will email a report to the safety manager automatically. They can tell which operators are the best and who needs improvement, and they can show trends of when accidents are more likely to happen.

IWT Develops the Guardian System

Innovative Wireless Technologies (IWT) plans to debut its new Guardian Proximity System soon. The new system uses proximity detection to help avoid collisions between miners and vehicles by automatically slowing and stopping vehicles when miners are nearby.

The Mine Safety and Health Administration (MSHA) is currently evaluating the system for intrinsic safety. Once IWT has that approval from the agency, they will roll out the system, which could happen later this year.

Guardian uses advanced technology that differs completely from legacy systems and the issues associated with them. Existing systems have detection zones that vary, which leads to inconsistent coverage. Some legacy systems were susceptible to electromagnetic interference from sources like high-voltage transformers, motors or AC power lines. Testing on the surface and underground has shown that the Guardian appears to be immune to those issues.

Coal operators approached IWT, saying they were not satisfied with what was available today and they wanted a better solution for proximity detection. Proximity systems for continuous miners are regulated, but they are not regulated for mobile haulage and other equipment. That prompted IWT to reengage proximity detection. Guardian has been in development for a few years now and IWT is ready to take it to market.

“IWT was working with the underlying technology 20 years ago, but it was not yet ready for commercialization,” said Jeremiah Colling, marketing director, IWT. “We experimented with various sensing modalities over a decade ago, and while early prototypes showed success, there were corner and edge cases that were still problematic. Today, the technologies have advanced to reach a stage where they are market-ready.”

While the initial launch will be for use on mobile haulage, the ultimate goal will be to use it on other types of vehicles.

Installing Guardian would be a fairly straightforward process. “We install the control system into the existing space and connect it to the variable frequency drives (VFDs), which control the speed of the vehicle,” said Paul May, product manager, IWT. “It controls the throttle at some percentage up to 100%. It can drop it all the way back to emergency brake range and even dump hydraulics if necessary. There’s no change to the customer equipment other than attaching our system.

The system has a robust operator interface. “It has a color, dot-matrix display that shows graphically where the sensors are seeing miners around the vehicle, and it also provides them with information about the system and any error condition,” May said.

For the shuttle car application, sensors are placed on the vehicle with multicolored LEDs to give a visual indication to the user when it slows or stops. The tags that the miners wear are small and lightweight.

“What that means for the user is that they can have confidence in knowing where people are,” Colling said. “They don’t have the false positives and nuisance alarms that slow production.”

If the proximity detection is not reliable, it can get in the way of production. Users can configure Guardian so that it is not only safe, but also does not impact production.

Guardian has performed as expected during testing as far as speed, accuracy and reliability, May explained. “During development, we were able to install and test the system in various configurations to determine the sensitivity of the sensors and reaction from the vehicles,” May said. “One of the things we do a little differently, and in this case we think is better, is the way that we are engaged with the controls of the vehicle. It can gradually reduce the speed of the vehicle as it senses a miner is getting closer.

“As the vehicle approaches a miner, it automatically slows to the point where it can come to a complete stop with the vehicle without engaging the emergency brake. That’s not always the case with some of the older equipment out there,” May said. “Some tend to be abrupt in terms of just slamming on the brakes when they get to a scenario where they are too close to the vehicle. And in some cases, that has caused physical damage to some of the drive mechanisms.

Maintaining the system is fairly simple. “The system self-calibrates as it’s in use,” May said. “We track and log the usage of the system and that allows us to set the right parameters initially. Unless they encounter physical breakage, like a vehicle backing into a sensor, there isn’t much from an ongoing maintenance perspective that they will have to do.”

Mine operators will have the ability to adjust certain parameters. For example, if they have an inexperienced driver, they can limit the throttle to reduce the top-end speed.

“At its core, it needs to be reliable in detecting the presence of people, and it also needs to be reliable in slowing and stopping the vehicle without driver intervention. Since the driver doesn’t need to make any decisions, it allows them to operate their vehicle and focus on their job with confidence.”

IWT believes Guardian will do more. Mines that are using IWT’s communication and tracking system could leverage the existing Sentinel network for further features. IWT’s tracking capability is integrated into its wireless handset and they may be able to do the same with a proximity tag.

During one of the demonstrations, Guardian detected a miner in a crosscut before the operator made the turn. The miner was not supposed to be standing there. The operator display showed the miner’s tag in the vicinity even though he couldn’t see the miner around the corner. Guardian slows the vehicle well in advance of rounding that corner. If the operator had made the turn blind at full speed, it could have been tragic.

OmniPro Improves With AI Feature

During December, Matrix Design Group, introduced OmniPro Vision AI, a highly accurate collision avoidance system that alerts operators when pedestrians, vehicles, and other potential hazards are detected in the path of mobile equipment operating on the surface. In addition to fully customizable zones, the system can work in low-light and poor weather conditions.

A high-speed processing rate and artificial intelligence (AI) minimize false positives and nuisance alerts. “The big difference with this system is that it relies on AI technology to identify objects in the vehicle’s path,” said Chris Adkins, mining sales manager, Matrix Design Group. “Miners no longer need to wear tags to be protected. We train it to detect a person in any position. They could be laying down, standing up, crouching, it could pick up a hand or a foot or whatever, and it will recognize that as a person.”

The system uses digital inputs and outputs to alert operators and, in some cases, force the equipment to slow or stop. All of the data from the OmniPro Vision AI camera feeds back to a main hub. “The main hub’s processing speed is extremely fast, like 30 frames per second,” Adkins said. “Using AI, it looks at each frame, trying to define a frame that has a person or object in it. When a zone breach is triggered, the operator can be alerted by visual and audible alarms or the system could be configured to control the throttle.”

OmniPro Vision AI works in conjunction with OmniPro InFocus, a cloud application delivering 24/7 access to an advanced dashboard with real-time metrics and analytics. “The system can provide mine operators with detailed reports, such as the safest or the least safe drivers,” Adkins said. “The company can also see the times where operations are most susceptible to accident or injury.”

All data, including zone breach incidents and timestamped photos, are recorded and stored. Essential reporting, including average daily breaches per machine and trending breaches over time, is available, along with custom reporting and analytics.

In a haul truck application, as an example, the system could be set to capture data for five seconds before and after an incident occurs. Mine operators can review that episode to see what happened and then break it down, determining whether it was a critical near miss or not. The reports can also be sent to safety managers automatically.

“Even though we are using terms like AI and talking about processing data in the cloud, the system is very simple,” Adkins said. “It has a camera, a hub and some cables. It communicates through WiFi or cellular connectivity. It does not need WiFi or cellular service to work. When it gets back in range, it will upload the information that it has to the cloud. It stores 30 days’ worth of information.”

The system has been built for high vibration. It has been designed to operate in dust, mud, and fog. Gorilla glass with a special dust-resistant coating protects the camera. “For low light situations, it uses infrared technology to detect objects in darkness,” Adkins said. “Matrix is currently working with Spartan Radar to potentially add radar to the system.”

The system could be used underground, but it has not been approved by MSHA as intrinsically safe.

Adkins said the AI feature views objects much like the human eye. “If you can see it, AI can see it and make adjustments much more quickly than you can,” Adkins said.

One of the requirements for MSHA’s surface safety program is to review available technologies. “The agency is not forcing the mines to buy collision avoidance systems, just asking them to consider it,” Adkins said. He believes that, once a mine operator considers it, they would quickly recognize the value in reducing equipment-related damage alone.

“We placed OmniPro Vision AI on a wheel loader and all it did was detect coal trucks behind it,” Adkins said. “It prevented the operator from backing into a parked coal truck. Busting the radiator on the back of the wheel loader is about $20,000 to $25,000.” Matrix is currently allowing mine operators to take a test run and pilot the system without purchasing it.