In railroading, “close calls” are defined as “potentially unsafe events or conditions.” Tasked by the Federal Railroad Administration (FRA), the Volpe National Transportation Systems Center (Volpe) evaluated the C3RS (Confidential Close Call Reporting System) between September 2022 and June 2024, examining implementation processes, sustainability, and safety outcomes. FRA’s report, DOT/FRA/ORD-24/45, Evaluating Processes, Outcomes, and Sustainability for FRA’s C3RS (downloadable below), identified program strengths and weaknesses and made recommendations for improvement.
The Volpe research team assessed program effectiveness and suggested modifications to sustain and institutionalize the program. using a “mixed-methods” approach that included focus group discussions, workshop activities and web surveys to obtain “qualitative data from a cross-section of stakeholders.” To assess outcomes, researchers reviewed corrective action (CA) documentation provided by railroads, FRA’s publicly available accident/incident data, and quantitative data related to report intake and safety-related communication from the National Aeronautics and Space Administration (NASA), “a trusted third party responsible for processing event reports.”
Volpe identified C3RS implementation process strengths as:
- “FRA’s training support to peer review teams (PRT).
- “Prior improvements to the Multiple Cause Incident Analysis (MCIA) tool.
- “NASA’s work to maintain reporter anonymity and confidentiality, reduce barriers to reporting, and improve report processing.”
“FRA and NASA support for education and outreach activities also was seen as a strength of the pre-implementation phase,” Volpe noted. “Many of these processes benefited from improvements made by FRA and NASA after soliciting and responding to user feedback … A positive outcome of C3RS has been the consistent sharing of lessons learned among and across participating railroads.”
Volpe identified several C3RS weaknesses across implementation phases:
- “Ambiguity in the implementing memorandum of understanding (IMOU) agreements.
- “PRT makeup and succession planning.
- “Gaps in rollout and training.
- “Reporter confidentiality at smaller railroads.
- “Lack of detail in event reports.
- “Prolonged report processing times and callback response rate.
- “PRT challenges with event analysis.
- “Challenges implementing, documenting, and tracking CAs.”
“Stakeholders have acted, or are in the process of acting, to address many of these weaknesses,” Volpe said. “Researchers also documented the adverse impact of the COVID-19 pandemic on C3RS activities.”
“Institutionalizing the C3RS program will require more involvement from railroad management during the post-implementation phase, paired with less involvement from FRA’s C3RS implementation team,” Volpe noted. “The research team suggests railroads move toward imbedding C3RS within their safety management practices. Hiring a dedicated C3RS employee to take ownership of the program could ease the burden on the PRT for activities such as outreach, training and sharing C3RS successes. An internet portal for employees could also support resource and data sharing.
“FRA should continue to streamline its implementation processes where possible, provide standalone resources to railroads to support training and outreach, and leverage the PRT to act as a conduit between railroad management and the C3RS implementation team. PRT members can support a railroad’s team as needed, while providing feedback to the C3RS implementation team on the organization’s successes and challenges. FRA should continue to model continuous process improvements and experiment with ways to adapt the C3RS model to fit stakeholder needs. [We] suggest exploring the Federal Aviation Administration (FAA) Aviation Safety Analysis Program (ASAP) as a model for future improvements.”
Based on data collected during the evaluation, the Volpe team found that C3RS “is a complex system with many moving parts that change dynamically over time. The process by which employees submit reports and PRTs conduct event analysis has experienced the greatest number of process improvements compared to other parts of the C3RS lifecycle. Researchers documented a variety of process improvements that make these post-implementation activities easier and more efficient.”
However, “the processes for implementing and monitoring CAs have received comparably less attention since C3RS began,” Volpe noted. “In part, this is because railroads spend less time analyzing the impacts of their CAs than they do performing event analysis, which makes it difficult to identify positive safety outcomes at almost all participating railroads. Demonstrating the value of implementing effective CAs can contribute to a virtuous cycle where employees and managers experience the value of reporting close calls. Additionally, documenting implementation and the systematic tracking of CAs present significant opportunities for safety improvements at most railroads.”
As well, Volpe said, “The loss of institutional knowledge about C3RS when staff turnover occurs has adversely impacted event reporting and analysis across all participating railroads. COVID-19 exacerbated these losses by increasing staff turnover and reducing the time available to devote to C3RS-related activities. While SPD (FRA’s Safety Partnerships Division) and NASA addressed these challenges by agreeing to re-educate and retrain employees when requested, railroads need to demonstrate their commitment to the program through integrating C3RS into their safety management processes and relying less on assistance from SPD and NASA.”
C3RS, Volpe concluded, “is a system for learning from mistakes. All participating railroads could benefit from experimenting in small ways to improve upon their current use of C3RS … Currently, SPD is experimenting with different models of C3RS. One ongoing demonstration uses SLSI (ASLRRA’s Short Line Safety Institute) personnel as a PRT, while another tests a model of C3RS at a Class I freight railroad using a different IMOU (Implementing Memorandum of Understanding) than that of currently participating railroads. SPD has modeled continuous process improvement through its efforts to address feedback from participating railroads and in initiating this independent evaluation.”
