In the two years it has been up and running, the Federal Motor Carrier Safety Administrations Compliance, Safety, Accountability program has demonstrated many flaws and shortcomings. But the program is credited with the signal achievement of shining a brighter light on truck safety.

"Whether you like CSA, consider it a work in progress or hate it, what it has done is change the culture of safety and made it far more important to all carriers. And that's a good thing," says Steve Bryan, CEO of Vigillo, the CSA data services provider.

Many trucking companies have the same take. "CSA literally has changed the way carriers do business," says Dave Heller, director of safety and policy for the Truckload Carriers Association. "It has put safety to the forefront, much more than it ever has been before."

These observations reflect what the American Transportation Research Institute has found in its research on CSA: The vast majority of carriers surveyed stay on top of their CSA data and use it to improve their safety.

FMCSA Administrator Anne Ferro cites a 167% increase in traffic on the agency's CSA website over the past year as an indication of the program's impact.

The result is an 8% decline in violations at roadside inspections and a 10% drop in driver violations per inspection.

"These represent the most dramatic decrease in violation rates in a decade," Ferro says.

All of that said, the list of CSA issues is long and daunting.

Ask practically anyone in trucking what's the biggest problem with the program, and the answer comes back quickly: CSA does not account for fault in crashes. The safety advocacy community has a different perspective, saying carriers are putting too much emphasis on accountability.

The accountability issue is just the start of the list of complaints, which includes:

- The system does not touch enough carriers.
- The data does not accurately reflect crash risk.
- There are disparities in how the states collect and report the data that goes into the system.
- FMCSA has not been candid about the system's limitations.
- The process for correcting the data, called DataQs, is cumbersome and unreliable.
- Shippers are misusing CSA data in their carrier selection process.

The anxiety and frustration with these problems is compounded by suspense. FMCSA is working on an overarching rule that will formally incorporate the CSA data and rating system into a standard for determining if a carrier is fit to operate.

John Hill, the former chief of FMCSA who now is a safety consultant with The Hill Group, says that the sooner the agency can get that rule out the better. The agency is essentially regulating motor carriers without having done a rule-making under the Administrative Procedures Act, he says.

But the level of difficulty for this proposed rule is quite high.

"It will be very interesting to see how they roll out a rule based on a system that everyone believes still needs a lot of improvement," says Vigillo's Bryan.

Ferro says the safety fitness determination proposal will be published by next summer.

A work in progress

Meanwhile, CSA is a work in progress.

FMCSA just implemented close to a dozen changes, including a new Hazardous Materials Compliance category in its Safety Measurement System.

This is part of a process of continuous improvement in which the agency posts changes for public review and comment before putting them into effect.

The agency also is soliciting advice from its Motor Carrier Safety Advisory Committee, a group representing law enforcement, carriers, the Teamsters union and safety advocates.

The committee is reaching out to other CSA stakeholders - brokers, shippers and researchers - to hear suggestions for improvement.

The challenges were on full display in early December as a CSA subcommittee of MCSAC took in testimony from the analysts who designed the program and researchers who have enumerated its flaws.

Crash accountability was a case in point.

Trucking interests believe that FMCSA needs to find a way to measure fault in the crashes in the CSA database.

Right now the data does not say whether or not the carrier was at fault. The Safety Measurement System rates a carrier's crash history in comparison to other carriers' history, presuming a certain degree of fault throughout the data. This has led to the conclusion that past crashes are a predictor of future crashes no matter who is at fault.

Robert Petrancosta, vice president of safety for Con-Way Freight, says the data is not useful to him if it does not indicate fault.

"How do we train to that?" he asks. "Wouldn't you rather me dedicate my resources to a problem we can fix, rather than just correct a score?"

Bryan put it this way: "CSA was designed to identify the behavior of drivers that law enforcement could focus on to reduce future crashes. If you're measuring all crashes instead of those that can be prevented, then your output's wrong."

TCA's Heller cites an instance of a driver having lunch in a restaurant when a drunk driver slammed into his parked truck. There are no-brainer accidents like these that should be identified so the carrier is not tainted, he says.

American Trucking Associations holds that the agency can and should weight crashes that clearly are not the carrier's fault.

Hill contends that the agency could reduce the tension over CSA by resolving this issue.

"I believe that if the agency would move ahead with addressing these non-preventable crashes, it would take a lot of the anger out of the regulatory community," he says.

But safety advocates are concerned that the process of adjusting for fault might sully the data with subjective judgments about the quality of police accident reports.

John Lannen, executive director of the Truck Safety Coalition, says he is concerned that in the process of assigning fault the police report will be misinterpreted.

And Stephen Owings, CEO of Road Safe America and the incoming chairman of MCSAC, contends that there is no cost-effective way to determine fault.

It is not clear how this gap will be bridged. The agency has a research program under way, focusing on the cost-benefit question, on whether or not police reports are reliable enough to determine accountability and on how it might manage an accountability process. The report is due next summer.

Auditing CSA

Another element of uncertainty arises from an audit of CSA being conducted by the Transportation Departments inspector general.

The audit was sparked by a hearing on CSA held last September by the House Highways and Transit Subcommittee at the request of carriers, shippers and brokers. It is due to be completed by next September.

Committee members asked the IG to look into a range of issues, including the accuracy, reliability and significance of CSA scores, in light of testimony that the system does not accurately rate carrier performance.

This question reflects research by ATRI indicating that while some CSA scores are accurate predictors of crash risk, others work in the opposite direction.

Looking at how scores in the five public Behavior Analysis Safety Improvement Categories (BASICs) relate to actual crash involvement, the institute found a strong correlation in three and a negative relationship in two.

Scores in the Unsafe Driving, Fatigued Driving and Vehicle Maintenance BASICs are positively related to crashes, while in the Driver Fitness and Controlled Substances and Alcohol BASICs, the higher the score, the lower the risk.

FMCSA says it understands that exceeding the score threshold in Driver Fitness does not in and of itself indicate high crash risk. But it has data showing that 75% of carriers above the threshold in that category also are above the threshold in at least one other category, so it is an indicator of risk.

ATRI s proposed solution to this problem is to change the way CSA data is revealed to the public.

Rather than using the percentile scoring system, the agency should show the carrier s relative safety status by placing it in one of three categories of increasing risk.

The first category would be for carriers that have enough data in at least one BASIC, but no score. The second would be for carriers with scores in at least one BASIC, but no "alerts" reflecting severe violations.

The third category would be broken down into five sub-groups, based on the number of "alerts" a carrier has.

The agency could continue to use percentile scores for internal purposes, but this approach to public data would address concerns about shippers and others taking the wrong message about a carriers safety risk from the BASIC data now on the agency's website, the Institute states.

The agency says it would consider this approach as part of its ongoing review of CSA, and the idea also is under consideration by MCSAC, the advisory committee.

Another confounding issue is the contention by third-party intermediaries, such as brokers, that shippers are misusing CSA data in their selection of carriers.

This adds yet another moving part to the CSA puzzle. A group of small carriers and brokers is suing FMCSA to get the Safety Measurement System data removed from public view.

Vigillo s Bryan, a member of the Transportation Intermediaries Association, says the brokers have a point.

"CSA is intended to be a tool for law enforcement to place the highest priority on carriers most at risk for future crashes, but the plaintiffs bar doesn't care what FMCSA said the purpose of CSA was," Bryan says.

"They now have data that they can spin and present to juries to make and tell any story they want to tell."

But it's too late to change that, Bryan says. "The responsibility is on brokers to be aware of what's being presented about the carriers they place freight with. [The data] is out there and they're going to have to deal with it."

Getting the job done

Bryan gives the agency credit for installing an open process for rolling out CSA changes - and dings the industry for not commenting more robustly on the coming changes.

On the other hand, he says, he didn't see a lot of evidence that the agency paid sufficient attention to the comments that did come in.

This observation is echoed by Rob Abbott, vice president for safety policy at ATA, who has expressed concern about a lack of candor from the agency.

"It's our sense that the agency is eager to point to the strengths of CSA but appears less willing to acknowledge its weaknesses," he says. "While it is understandable that they would take this approach to their own product, it does not facilitate a fully informed process for improvement."

ATA says CSA is an improvement over the old safety measurement system. However, the association has adopted a policy of confrontation with the agency when the agency's views differ from its own.

John Hill's view perhaps reflects his experience as the former chief of the agency.

"I do believe the agency has made great strides in terms of the amount of data and the quality of the data," he says. "I know a lot of people dispute that, but I think if you look at the big picture, there has been tremendous improvement."

He and practically everyone else who is connected to CSA agree that more improvement is needed - and that the key to getting that done is probably the most difficult challenge of all: open and trusting communications between the agency and its stakeholders.

Agency posts CSA changes

The Federal Motor Carrier Safety Administration implemented a number of changes to CSA in December. At the top of the list is creation of a new HazMat Compliance BASIC (Behavior Analysis and Safety Improvement Category).

The agency changed the old Cargo-Related BASIC to HazMat in response to concerns about tracking hazmat problems.

This BASIC is visible only to carriers and law enforcement, an acknowledgement by the agency of industry concern that the scores in the new hazmat category are inconsistent.

Other changes:

- The Fatigued Driving BASIC is changed to the more specific Hours of Service Compliance BASIC, to more accurately reflect violations in this area and weight HOS paper and electronic logbook violations equally.
- The Vehicle Maintenance BASIC now includes cargo load se-curement violations from the previous Cargo-Related BASIC.
- Intermodal equipment violations that should be found during drivers' pre-trip inspections are now included.
- Speeding violations in the 1 to 5 mph range are removed to en sure citations are consistent with current speedometer regulations.
- Changes ensure that all recorded violations accurately reflect the inspection type. For example, only driver violations will be recorded under driver inspections.

CSA impact on drivers

The American Transportation Research Institute found that CSA has not caused the wholesale loss of drivers that had been predicted. ATRI, the research arm of American Trucking Associations, found in a new study that only a small fraction of current drivers have been put out of work by CSA.

On the other hand, CSA and the new Pre-employment Screening Program have had a significant impact on the hiring of new drivers, it says.

The programs are helpful in screening out undesirable drivers, but they also have made it more difficult for carriers to find drivers, ATRI reports.

Two other points about drivers:

- CSA has prompted carriers to offer financial incentives for good performance, which promotes safety compliance.
- Most drivers are not taking the initiative to learn about CSA. "A more targeted learning approach is necessary for drivers," ATRI says. It is up to "informed stakeholders" to reach out to driv ers, rather than wait for the drivers to act. Drivers are motivated to learn, but it's hard for them to find the time to access the material.