We set tongues a-wagging recently with the publication of some fuel economy test results. While the comments section of the original story doesn't indicate as much, I received a lot of email and more than a few calls, mostly from folks wondering how the results of separate SAE J1321 Type II fuel efficiency tests could be so strikingly different.
I've done a lot of research since we ran the story, and I confess that I have not yet come up with a definitive answer to that central question. Nor have I any interest in getting caught in the middle of a showdown between the tester, Quebec-based Performance Innovation Transport (PIT) group, and the testee, SmartTruck Systems Inc.
SmartTruck says testing has shown the fuel economy of its undertray aero system is higher than recent results we reported from PIT.
I should point out that Heavy Duty Trucking, and I as the reporter who wrote the story, are in no way connected to the testing. We did not commission the testing. The results were distributed by PIT to this magazine and others.
Briefly, PIT published the results of tests conducted on four brands of trailer side skirts, and three brands of undercarriage air deflectors. While PIT named the brands tested, it did not divulge the exact results achieved by each brand. Instead, PIT offered a range of results from lowest to highest.
From the PIT press release: "Fuel savings with [with the side skirt] devices ranged from 5.2% to 7.45% compared to similar vehicles without skirts. Fuel savings measured on vehicles equipped with [undercarriage air deflectors] ranged from 0% to 2.2%."
SmartTruck claims a 5.5% fuel saving for its basic UT1 undertray device, and up to a 10.5% improvement for the more elaborate UT6 configuration. We don't know where SmartTruck sat in PIT's 0% to 2.2% range, but even if it was the top scorer, that's still a big discrepancy.
SmartTruck asked for an opportunity to share its side of the story, which is what prompted this piece. The company issued a statement, which I've included here. We also arranged a follow-up conversation SmartTruck Systems' chief commercial officer, Mitch Greenberg.
"SmartTruck has conducted multiple rounds of testing over several years using numerous testing methods to determine the factual fuel savings of its UnderTray systems. These tests always strictly follow all U.S. EPA-recognized testing guidelines and addendums with no exceptions or shortcuts. EPA requires all protocols be followed without exception or variation, or the test results are rejected as invalid and erroneous. This stringency enables the testing methods to minimize variability and any possible compromises of the testing data. Before releasing any product for sale, SmartTruck utilizes SAE Type II (J1321) testing required by EPA’s SmartWay Program, coast down testing as required by EPA’s Heavy Duty Vehicle Fuel Efficiency Regulation and computational fluid dynamic (CFD) testing using world class computing systems — with all rounds of all types of testing providing the same statistically significant fuel savings results. In addition to testing, SmartTruck also has a wide array of customers who have verified these fuel savings results with their own real-world experience with their UnderTray systems."
During our conversation, Greenberg explained how the company used computer modeling (computational fluid dynamics, or CFD) in the design and testing of its products, along with wind tunnel test and eventually track testing to SAE J1321 protocols.
"The J1321 test is a tough test to run," he said. "There are many variables that come with testing a full tractor-trailer unit in a 40-mile test segment. Because of those difficulties, EPA is moving away from J1321. In fact, in the newest greenhouse gas regulations for tractor and engine systems, for aerodynamic assessment, they do not use J1321. They use coast-down tests, wind tunnel and CFD as the three main listed protocols for how to test an aerodynamic device."
Greenberg also supplied copies of test results conducted by the BMI Corporation of Greenville, S.C., among others, which says:
"Testing on the UT-1 UnderTray System was conducted the week of November 7, 2010, at the Continental Tires Proving Grounds in Uvalde, Texas. Test results using the Test Truck to Control Truck (T/C) ratios detailed in the SAE J1321 protocol conclude the UT-1 Trailer UnderTray System produces a 5.5% improvement in fuel efficiency."
While the reader now has SmartTruck's side of the story, we're still left with the conundrum of the discrepancy between the reported results.
I can't possibly say with any certainty which of the test results are more accurate, but I'm leaning toward the conclusion that both sets of results are accurate for the circumstances under which they were performed. Frankly, there were some minor inconsistencies between PIT's test and the results from BMI that SmartTruck supplied.
Perhaps they are not so minor. But with my limited understanding of aerodynamics and testing protocols, I'm not going to wade any further into this particular debate.
I spoke with several sophisticated fleets to get some background and a sense of where they are with aerodynamics, testing, reported results, etc. All said they routinely take the manufacturers' claimed test results and chop the numbers in half. That, they all said, gives a reliable estimate of what they will see over a year of in-service monitoring.
That suggests to me that the results of any controlled, snapshot type of test, like SAE's J1321, may not be that useful to the consumers of such technology. Unfortunately, it also casts doubt over EPA's SmartWay verification and certification process. More on this to come.
Trailer Talk Blog: Are 2.23% Fuel Savings Too Modest to Sell this Aero Improver?