The back-and-forth over the Compliance Safety and Accountability enforcement program continues, this time with the Federal Motor Carrier Safety Administration countering recent criticisms by the Government Accountability Office.

GAO noted in its analysis that FMCSA had “significant and substantive disagreements,” and on Wednesday the agency spelled out its concerns.

GAO’s contention was that the system does not track violations closely enough to measure crash risk for individual carriers, and that there’s not enough data to reliably compare most carriers’ performance with their peers.

But GAO’s analysis is off-target, said Joe DeLorenzo, director of FMCSA’s Office of Enforcement and Compliance, in a presentation to a CSA advisory group meeting in Alexandria, Va.

CSA is the system the agency uses to measure carrier safety performance and determine which carriers pose a higher risk, so it can apply its enforcement resources where they will do the most good.

DeLorenzo said GAO focused only on large carriers in its analysis and assumed that carriers that have not had crashes are not at risk.

Further, the alternative approaches GAO suggested are not data-driven, he said. And GAO did not consider the compliance status of the carrier or the purpose of the CSA Safety Measurement System, which is to prioritize the agency’s intervention resources.

The safety agency agrees with GAO that having fewer inspections creates more variability in the data. But the Safety Measurement System is reliable for its purpose, DeLorenzo said. It requires a pattern of inspections with violations before it targets a response and it compares compliance between carriers to find the least safe carriers.

GAO proposed that the agency score only the carriers that have more information – 20 or more inspections – noting that this would cover fewer carriers but the CSA scores would be more reliable.

But this approach would limit CSA’s effective coverage to 10% of the industry, DeLorenzo said.

GAO also suggested that the agency stop grouping carriers by their number of safety events such as inspections with violations and crashes. 

DeLorenzo said this would create variability by treating carriers with 20 inspections the same as those with 10,000 inspections. This creates a bias toward carriers at the lower end of the standard and a virtual free pass for larger carriers with more than 100 inspections, he said.

Another issue raised by GAO is the agency’s pending rule on safety fitness determination. This is the long-awaited rule that will take the CSA program to the next level, from a risk assessment tool to a method for determining fitness to operate.

GAO said the agency should consider the limitations in CSA data when it writes the rule.

But GAO did not characterize FMCSA’s position correctly, DeLorenzo said.

“We’ve never said we will use the Safety Measurement System scores in the Safety Fitness Determination,” he said.

The agency agrees that the safety fitness determination requires different considerations than those used in the current CSA program. The Safety Fitness Determination may use the same data that’s in the Safety Measurement System, but it will not use relative scoring based on someone else’s behavior, DeLorenzo said.

“Data is one thing, scores are another.”

The Safety Fitness Determination proposal is scheduled to be published in August.

DeLorenzo also presented the results of an effectiveness test of the Safety Measurement System. In this analysis the agency used historical data to measure how well SMS identifies high-risk carriers.

He said it found that the carrier population identified as high risk in SMS has more than twice the national average crash rate.

SMS is not designed to predict an individual company’s crashes in the future. Rather, the patterns of non-compliance that show up in the Behavior Analysis and Safety Improvement Categories (BASICs) relate to a higher future crash rate for groups of carriers, he said.