In mid-August PrePass published the white paper “CSA Overhaul Under Way,” describing why a congressionally-mandated review of the Federal Motor Carrier Safety Administration’s (FMCSA) Compliance, Safety Accountability (CSA) program came about, what the National Academy of Sciences (NAS) and the U.S. Department of Transportation’s inspector general had to say about it, and how FMCSA intends to move forward.
The resulting new safety program will impact how motor carriers are evaluated for years to come, so staying abreast of developments is critical for both fleets and truck drivers. And there are new developments, plus delays and a deeper dive into the new methodology – called Item Response Theory (IRT) – which will drive future safety evaluations.
- FMCSA sent Congress its response to the National Academy of Sciences study of CSA. That response, the “Correlation Study Corrective Action Plan,” laid out steps and a timetable FMCSA would take. The FMCSA response is now under audit by the U.S. DOT Inspector General, with two goals. One, has FMCSA met the NAS recommendations, and two, what challenges and costs lie ahead for the agency? The audit findings will guide agency actions and timing.
- FMCSA held a public hearing on its correlation study on Sept. 8. The agency has pledged to open dockets for public comment as each key decision point in the Correlation Study is reached.
- In the meantime, the FMCSA has removed alerts and relative percentiles for motor carriers from public display on the Safety Measurement System (SMS) website. That information will not be published until the correlation study is complete and all elements of the congressional directive are satisfied.
- FMCSA optimistically intended to begin “small scale” testing of an IRT model in September, followed by full scale testing in April 2019. While technical work continues, the actual testing of an IRT model missed the September date. The Inspector General’s audit may further delay this and possibly the April 2019 date.
- Item Response Theory (IRT) is well-known in the healthcare and education fields. It is highly technical and requires significant data , presenting challenges to FMCSA and to anyone trying to navigate from the old SMS process under CSA.
- Properly used, though, IRT brings mathematical validity to what had been arbitrary assessments of fleets under SMS, such as severity weights for violations. IRT will likely end BASICs (Behavior Analysis and Safety Improvement Categories) as we know them and will move away from the “peer group” comparisons among fleets.
- This IRT shift is intended to abandon CSA’s “crash risk” approach for a general (though statistically sound) sense of how well fleets value and implement safety across all aspects of their business. The end result may be a singular safety score for motor carriers – similar to the singular FICO score people receive from credit bureaus – instead of today’s percentile rankings.
- Experts have described the IRT approach as an emphasis on the “safety culture” of a motor carrier. Under IRT, weaknesses across a carrier’s operations may result in a lower safety evaluation than would an alert in just one BASIC.
Fleets must prepare for the advent of IRT. PrePass plans to publish a practical, common-sense guide to the new era of motor carrier “safety culture.” Stay tuned for further developments.