How can railroads bring together Artificial Intelligence (AI) and human expertise to reduce risk and improve safety without interrupting service? One way is to employ automated train inspection portals. Driven by AI, their high-speed camera arrays produce thousands of images and detect defects in rolling stock (running gear, wheels, axles, bearings, air brake rigging, truck side frame components, couplers, etc.) that the human eye cannot see—all at train operating speeds.
Working in tandem with boots-on-the-ground railroaders and back-office analysts, the portals help reduce derailments by identifying defects before they become a serious problem, enabling proactive, predictive maintenance. Other benefits include improved on-time performance, higher equipment utilization rates, and increased network velocity.
Norfolk Southern (NS) and CN are among the Class I railroads deploying the technology. Both shared their experiences and results with Railway Age.
Case study: Norfolk Southern
NS’s roughly 19,500 route-mile system crosses 22 states and the District of Columbia. In 2023, its first Digital Train Inspection Portal was installed in Leetonia, Ohio, and today a total of seven operate at six heavily trafficked locations (one double-track location has two portals). The portals’ Machine Vision inspection technology was developed by Georgia Tech Research Institute, which engineered the hardware, and NS’s Data Science/AI and Mechanical teams, which produced the software and advanced the AI algorithms.
Each portal features a setup of 24-megapixel trackside cameras and stadium lighting to capture “ultra-high-resolution” 360-degree images of passing railcars. The synchronized cameras net approximately 1,000 images per car at speeds up to 70 mph, detecting defects at angles that are difficult to see during stationary inspections.
How do the portals work? “The data captured—petabytes each month—is processed in real time by our AI models,” NS AVP Enterprise Data and Analytics Mabby Amouie tells Railway Age. “Critical findings are transmitted to our Network Operations Center, where a team of experts review and act on them. This enables near-instantaneous responses to emerging safety issues and supports predictive maintenance strategies.”
The experts at the wayside health desk are guided by what the railroad calls “a robust response protocol.” 99% of the conditions found by AI are “very low level” and can be repaired the next time the car is on a RIP (repair in place) track, according to NS Vice President and Chief Safety Officer John Fleps. Any critical defects found are flagged for immediate handling.
“We have tiers of severity,” Fleps tells Railway Age. “If it’s a severity Level 1 or Tier 1 condition, the wayside health desk team will tell the crew to bring their train to a safe stop and perform an on-the-ground inspection. The crew has mobile devices with an app designed to send images straight from the ballast line to the wayside desk without having to deal with the headaches of email or text messaging. The condition will be validated, and then we’ll follow our standard operating procedures on what to do. It may necessitate walking the car to a safe set-out location. It may be a condition that warrants a reduced speed, and the car can be set out at the next forward mechanical location. [Or the car] may be able to go to the next location where it’s going to be switched, and we’ll set it out there for scheduled repair.”
NS’s in-house Data Science/AI team has developed 75 “advanced deep-learning” algorithms that look for specific conditions and analyze images in real time. They are said to demonstrate “very high accuracy levels, while having very low false indicators.”
As broken wheel derailments are among the leading causes of main line accidents, the railroad continues to improve upon its algorithm for the detection of cracked and defective wheels.
Safety Benefits
Portal use has led to measurable safety improvements at NS. “In 2024 alone, we identified and handled more than 25,000 mechanical maintenance-related defects, including 85 Tier 1 critical defects in near real time,” Mabby Amouie reports. The railroad earlier this year reached a milestone with the “first-ever autonomous detection of a hairline crack in a wheel—a defect that’s extremely difficult to identify in real-world conditions,” he says. “This early detection allowed us to remove the car from service before it could cause an incident, demonstrating the system’s potential to prevent derailments and accidents.”
NS earlier this year reported the “first-ever autonomous detection of a hairline crack in a wheel—a defect that’s extremely difficult to identify in real-world conditions,” AVP Enterprise Data and Analytics Mabby Amouie says. (Norfolk Southern Image)
Cracked wheel detection, he adds, “is a prime example of how AI and human expertise are working together to reduce risk and improve safety outcomes.”
Portal use has played a role in improving NS’s main line accident rate. “We’ve reduced our accident rate by upwards of 60%, 63% in two years—from 2022 to 2024,” Fleps tells Railway Age. “The impact to line of road—train stops, train delays, emergency-type situations that occur—has also been reduced significantly.”
Turning ‘Finders into Fixers’
Using portals to capture images from moving trains not only keeps traffic moving but can also improve safety for inspectors and allow them to focus on repairs.
Portals are helping to find equipment problems earlier in the “failure cycle,” allowing railcars to be repaired proactively in a controlled environment, according to Fleps. “It puts our people at a much lower level of exposure when they’re able to make a repair in a shop instead of in the middle of the night [during] who knows what kind of weather out on the line of road,” he points out.
Removing such risk has helped NS to reduce its injury rate more than 10% year over year, leading to the “best injury rate that we’ve had in more than a decade,” Fleps reports. “This is not the only reason [for the injury rate improvement], but it’s an important piece. It’s all about taking risk exposure out of the equation.”
“Our goal is to set up our people for success,” Fleps sums up. “This is all about taking the finders and making them fixers. If we can provide intelligence [from the Digital Train Inspection Portal] to somebody who is skilled at solving problems, that’s a huge win for everybody.” More “fixing” and less “finding,” he says, “ultimately provides a safer operation for our customers, for our employees, for the communities where we operate, and for the industry at large.”
Case Study: CN
CN receives alerts when automated train inspection portals identify defects. Qualified railcar mechanics decide whether the defect should be addressed immediately or if the car can continue to the next inspection point, for instance. (CN Photograph)
CN’s tri-coastal network of approximately 20,000 route-miles of track spans Canada from east to west and down through the Midwest to the Gulf of Mexico. Seven automated train inspection portals from Duos Technologies have been placed strategically on the railroad’s core routes, with two in the U.S. and the remainder in Canada.
The rip® or Railcar Inspection Portal from Duos is described as “a modular intelligent visualization system that provides real-time detailed 360-degree imaging at high speeds” for use on main lines or in yards. According to the supplier, “the included Linear Panorama Generator assembles images gathered from cameras and stitches [together] all frames to create a continuous view of the entire consist.” Duos notes that “[o]perators can quickly select the side of interest and scroll through the continuous panoramic view.”
Automated Equipment Inventory (AEI) consist data “is the primary methodology used to synchronize captured images of each railcar,” Duos says. “The system is searchable using the AEI tag number or sequence in the train.” The company’s “high-definition imaging utilizes megapixel line scan cameras to provide an average image resolution of 224 megapixels per railcar,” and Machine Learning algorithms (AI) can also be incorporated.
“This technology helps us to identify defects that may be starting to present themselves on equipment traversing the railroad,” CN Vice President Safety & Environment Mark Grubbs tells Railway Age. Defects are classified as standard car/equipment defects or as critical defects, which if not repaired have “a high probability” of causing a service interruption. The railroad finds “thousands” of standard car/equipment defects per year, which Grubbs defines as “preventative,” such as worn-out brake shoes. Another example of a preventative defect: “In our draft systems, our couplers are held in by a bolted carrier iron, so if any of those bolts are loose or missing, [the portal] will identify that and alert us so we can make the repair,” Grubbs says. However, in the same example, if three of the six bolts in the plate are missing, that would be classified as a critical defect, he says.
CN has a Visual Analytics Team (VAT) in the U.S. and in Canada that receive alerts when portals identify defects. The team members—qualified railcar mechanics—decide whether the defect should be addressed immediately or if the car can continue to the next inspection point, for instance.
Because the portals identify railcar defects before they arrive at a terminal for regular manual inspection, “we can turn our inspection folks into repair people,” Grubbs says. “It allows us to efficiently process the equipment and removes some of the risk that our folks in the terminals are exposed to. I don’t have two or three folks possibly walking an entire train. I’m pointing them to the defect. All the hazards that could present themselves during the inspection have been removed because we’re pointing them to where they need to go.”
“It’s our next level of safety and efficiency,” Grubbs sums up.
CN aggregates the more than 24 million data points annually it receives from its automated train inspection portals, other network detectors, and its ATIP (Autonomous Track Inspection Program) railcar fleet to assess risk and to develop maintenance and capital spending plans. “Through the inspection portals, I can see the condition of my cars; with ATIP, I can see the condition of my track structure,” Grubbs says. With that data and more, the railroad can decide where investments are needed. This “ultimately helps us to keep our communities safe,” he notes.
Automated Plus Manual Inspection
At NS and CN, the automated inspection portals complement manual inspections. “There’s always a benefit to layers of protection and layers of risk control, and that’s where people are important part of the equation,” NS’s John Fleps says. “What we can do with Machine Vision is help the [human] inspector be as successful as possible.” For example, a car inspector who is going to perform an inbound inspection on a train that just went through a portal can perform a regular visual inspection and “home in on” conditions identified by the portal that they may have otherwise overlooked and validate whether or not a condition is legitimate and needs attention, according to Fleps.
“Our approach is hybrid,” NS’s Mabby Amouie tells Railway Age. “The AI augments—not replaces—human inspectors. The portals detect defects that are difficult or impossible to see during stationary inspections, while our railroaders provide the critical judgment and hands-on expertise to access, validate and address issues.”
According to Mark Grubbs, the portals guide CN to defects that can be difficult to see with the human eye and in winter weather conditions, for instance, and help it to be as efficient as possible. They also serve as “a check and balance,” he says. The images can be used to “spot check” manual inspection quality. “I’m happy to report that we do quite well,” Grubbs tells Railway Age.
In Canada, CN has received an exemption from Transport Canada as part of a pilot project (like a Federal Railroad Administration-granted waiver in the U.S.) involving its automated inspection portals. The exemption is for a subset of traffic travelling through Winnipeg and going to Eastern Canada. As part of the pilot project, once one of the exempted trains travels through the automated inspection portal CN allows its VAT team to use the images to remotely inspect the trains. Rather than using a portal’s algorithms to identify defects, the VAT team is “actively looking at the train” and leveraging the portal cameras and other data points to perform the inspections remotely, according to Grubbs.
Once one of the trains enters the terminal, an air brake test is conducted, and any required repairs are identified based on the VAT team’s remote inspection. “We’re sharing the data with Transport Canada to really pressure test the technology,” Grubbs says. “And we’re seeing some really good results.” The next step, he tells Railway Age, could be increasing the number of trains the VAT team inspects remotely.
Will FRA Regs Change in the Future?
Railway Age asked NS and CN if, down the road, portal use could change the way the FRA’s existing manual or visible inspection rules are set up. “This technology has advanced light years since the regulations were written,” NS’s Fleps says. “There wasn’t even a vision, no pun intended, for what inspection would look like, certainly not to the level of detail that we’re able to execute today. So just like the conversation that’s going on right now around automated track inspection, the same conversation needs to be had [about automated train inspection]. … I don’t think the industry is quite at the capability level that NS is [with automated train inspection], but that’s not a reason to hinder the innovation and the progress we’re seeing at work.”
Adds Amouie: “We believe that these [automated train inspection] technologies continue to prove their reliability and safety benefits, and we’re committed to working constructively with regulators to ensure that innovation and safety go hand in hand.”
NS’s goal is to have each railcar on its network pass through a portal site every 1,000 miles, Fleps says. The railroad expects to achieve about 90% coverage over the next several years. “We’ve got 17 total sites identified, and part of our plan right now, which will constitute 20 individual portals,” he says.
“We want to continue to build out our [portal] footprint,” Fleps goes on to say. “We want to continue to build out our algorithm set. Ideally, we [will be] able to do a complete inbound train inspection with the portal.”
NS also wants to share its portal technology. “It benefits everybody,” Fleps points out. Some equipment that NS portals flag could have been scheduled for repair earlier if another railroad’s portal flagged it, he notes. “So, we’re working with other Class I’s and short lines to share what we’ve done and ensure the rest of the industry is able to benefit from it.”
“Automated railcar inspection portals, particularly those using Machine Vision systems enhanced with AI and machine learning, are helping railroads make meaningful safety gains by identifying defects more effectively and consistently than manual inspections—especially while trains are in motion,” the Association of American Railroads says. “In many cases, these systems outperform human inspectors by detecting critical, visible defects that are often missed during static inspections. However, these systems are not yet trained to assess all types of defects, and their high installation and operational costs limit widespread deployment. As such, traditional manual inspections still play an important role.”
