Fleet Management

Proposed CSA Scoring-Improvement Method Gets Thumbs Up

October 22, 2017

By David Cullen

SHARING TOOLS        | Print Subscribe
FMCSA's Joe DeLorenzo speaking in Orlando. Photo: David Cullen
FMCSA's Joe DeLorenzo speaking in Orlando. Photo: David Cullen

ORLANDO – Boiled down to its essence, the congressionally mandated review of the Compliance, Safety, Accountability program by the National Academies of Sciences found that the Federal Motor Carrier Safety Administration should move to a more scientific method to drive the Safety Measurement System that signals which motor carriers are at highest risk for future crashes, according to an FMSCA official. 

“The National Academies said to FMCSA, ‘The model you have is pretty good, but it’s not based on a principled scientific approach. We would like to see a better statistical model behind SMS,’” FMCSA Director of Compliance and Enforcement Joe DeLorenzo, said on Oct. 22 during an  educational session here at the American Trucking Associations’ annual Management Conference & Exhibition.

He said NAS recommended that FMSCA take about two years to study a more scientific modeling approach known as “Item Response Theory” and then consider implementing it to make the SMS function more accurately. “Our focus,” he told the audience, “remains focused on the prevention of crashes, not the prediction of crashes.”

DeLorenzo said the hope is that an IRT-based method would help develop “an estimate of the measure of ‘safety culture’ for each carrier and be used to monitor and identify carriers in need of intervention,” rather than just zero in on violations.

To accomplish all that may take digging into a variety of data points, such as driver pay, driver turnover, and miles travelled. He likened such an approach to the IRT-type scoring used in such standardized tests as the SAT (Scholastic Aptitude Test), familiar to generations of students applying for college admission, that weights answers not just on whether they are “right” or “wrong” but also on how they reflect a student’s depth of understanding.

“This is a long-term development,” DeLorenzo conceded. “The next step will be developing an action plan [in response to the NAS survey] per the FAST Act [which mandated the survey]. FMCSA will continue working with the National Academies on this over the next several years. All this [work] is very data-intensive; it will take a long time to run [all the numbers].”

Steve Bryan, president of president of data analysis firm Vigillo, Inc., which provides safety analytics, told the audience that after attending all of the public meetings on the SMS study held by NAS and digesting the 134-pg. report produced, he is in favor of seeing an IRT-based model adopted.

“IRT is a very sophisticated statistical model,” said Bryan. “But the good news is it has been battle-tested in other industries. IRT won’t predict the next crash, but it will shine a light on the cultural weaknesses of carriers that lead to higher crash rates.

He allowed that incorporating the NAS recommendations into the CSA system will be “a heavy lift,” but said the result will be worth it as trucking will end up with “a much better safety scoring system.”

Comments

  1. 1. Brent [ October 23, 2017 @ 08:28AM ]

    The title says "Proposed CSA Scoring-Improvement Method Gets Thumbs Up". Who is giving the thumbs up? Per this article, http://www.ttnews.com/articles/item-response-theory-model-has-potential-reduce-crashes-fmcsa-says

    when asked to raise hands if they understood it, no one raised their hands. Then after explaining it, they asked again for a raise of hands of who understood it. Only one person raised their hand. This seems like a biased article title to push an agenda rather than reporting what actually happened at the conference.

 

Comment On This Story

Name:  
Email:  
Comment: (Maximum 2000 characters)  
Leave this field empty:
* Please note that every comment is moderated.

Newsletter

We offer e-newsletters that deliver targeted news and information for the entire fleet industry.

GotQuestions?

ELDs and Telematics

sponsored by
sponsor logo

Scott Sutarik from Geotab will answer your questions and challenges

View All

Sleeper Cab Power

Steve Carlson from Xantrex will answer your questions and challenges

View All