Goodbye BASICs, you are now “safety categories.” Goodbye IRT, Item Response Theory was too complex and too slow to be useful.

The Federal Motor Carrier Safety Administration (FMCSA) is proposing changes to the Safety Measurement System (SMS) as published in a lengthy and detailed notice. The SMS comprises the mathematical system in the Compliance, Safety, Accountability (CSA) program used to calculate motor carrier safety ratings and assign intervention levels.

“Safety is FMCSA’s core mission. The proposed changes are part of the agency’s continued commitment to enhancing the fairness, accuracy, and clarity of our prioritization system,” said FMCSA Administrator Robin Hutcheson.

FMCSA is envisioning numerous and substantive changes to the SMS. They deserve close carrier scrutiny, which is why FMCSA is taking significant steps to help explain the proposal and solicit feedback:

Until FMCSA formally adopts any SMS changes, the current system will continue in place. The agency specifically notes that SMS methodology will continue to exclude crashes found not preventable after review through FMCSA’s Crash Preventability Determination Program.

Here, in brief, are the changes proposed by FMCSA:

  • The current Behavior Analysis and Safety Improvement Categories (BASICs) would be reorganized by moving the existing Controlled Substances and Alcohol violations into the Unsafe Driving category and dividing the Vehicle Maintenance BASIC into two separate categories – Vehicle Maintenance and Vehicle Maintenance: Driver Observed. The latter change is intended to bring more focus on pre-and post-trip inspections. As for the drug/alcohol violations, FMCSA says there are too few discovered at roadside to merit their own category. The “BASICs” nomenclature would be discontinued to underscore these changes.
  • Today there are 973 different violation provisions or code citations within the SMS. FMCSA proposes to combine similar violations into 116 groups, simplifying compliance and enforcement.
  • Assigning severity weights to these violations was always controversial. The current system, using a 1 through 10 ranking of severity, allowed room for subjectivity. FMCSA proposes to assign either a 1 or a 2 to each violation, with the higher number indicating a truly severe infraction.
  • SMS compares motor carriers to similar fleets in a “safety event group,” largely based on the number of inspections and violations. But when a motor carrier moves to another group, the carrier’s rating can change significantly. FMCSA proposes a “proportionate percentile” system to smooth out any rating changes.
  • For each safety category, SMS establishes an Intervention Threshold, informing law enforcement that a carrier needs attention. FMCSA proposes adjusting those thresholds to reflect the potential for crash involvement better. At the same time, FMCSA would emphasize more recent violations and adjust upward the assumed vehicle miles traveled by each carrier power unit.

Finally, FMCSA declined to propose two changes called for by some observers:

  • FMCSA does not propose any adjustment for the geographic variations in enforcement emphasis, saying that jurisdictions need the ability to address their local road, congestion, topography, and weather conditions.
  • FMCSA also declined to increase the number of crashes required to assign a percentile in the Crash Indicator category, noting that research supports the current SMS standard.

Why abandon IRT? The trucking industry had earlier objected to disclosing information required by Item Response Theory which it regarded as proprietary. FMCSA decided that using IRT necessitated “an advanced understanding of statistical modeling and analysis,” and would not be useful to motor carriers seeking to understand and improve their safety rating. For its part, FMCSA noted that IRT analysis took four weeks to run, compared to the two days for SMS.

Much more detail is available in the “Foundational Document” included in the agency notice. PrePass will keep readers informed as FMCSA considers these changes.

The PrePass blog and podcasts are published as a public service of PrePass®, the most reliable and technologically advanced weigh station bypass and integrated electronic trucking toll payment platform in North America. PrePass also includes INFORM™ Safety and INFORM™ Tolling software for improving truck safety scores and lowering toll costs.