Fuel economy testing is a game of inches. Unless you're comparing a classic-style tractor and a bare-bones dry van to the latest aero tractor and slick, aero-festooned trailer, you're not going to see huge differences. In some cases, add-ons will provide gains only in the tenths-of-a-percent range.

If the ROI is there, then even tenths of a percent is an improvement, but statistically, a percentage that small is noise. The actual improvements could be difficult to separate from anomalies in the testing.

And your tires can skew your test results significantly.

I'm not talking about evaluating and comparing tires, but how tires impact the testing of other products, such as trailer skirts, fairings, tire pressure monitoring and inflation systems, etc.

Regardless of the anticipated gains, accurate test results are necessary. Savings accrued from the improvements are part of the return-on-investment calculation, as well as the long-term contribution of the add-on to your bottom line. If your test results are off by a few percentage points, an anticipated 18-month ROI could become a 24-month ROI in the real world -- or vice versa.

Testing methods vary with fleet resources. A big fleet may have the people and the equipment to run SAE or TMC Type II or Type III tests than involve weighing fuel tanks, switching out equipment from the truck or from one truck to another. Smaller fleets may not have that luxury, and might tend to test products on two or more comparison trucks over a period of time. One test gets quick results, the other takes time; both require discipline and diligence.

The problem is results can vary with the accuracy of the test. The short-term Type II and Type III tests are likely to produce better results if the tests are done properly. The longer term comparison tests are subject to myriad variables -- drivers, loads, equipment condition and tires.

According to Bob Wessels, a former customer support manager for Caterpillar, now a consultant, tires can be the biggest variable in testing. There can be a difference of up to 3% between a new tire and one that's ready to come off the truck. If you have the luxury of a large sample size, such as 25 trucks or more, Wessels suggests using a mix of new and in-service tires, rather than switching to all new tires.

"The differences will disappear in the aggregated data over time," he says.

The biggest contributor to inaccuracy as far as tires are concerned are tread depth and inflation pressure -- and an unaccounted-for mix of fuel efficient and non-fuel efficient tires.

For obvious reasons, you won't get reliable data if you're comparing one truck with trailer skirts and one without, if one has fuel-efficient, low rolling resistance tires and the other does not. While that may seem like a no-brainer, it's easy to foul up your results.

"In operational testing, trailers get switched, tires get changed, any number of things can go wrong," Wessels cautions. "You have to control those variables and remove the bad data from the results. That involves almost daily monitoring of the test program."

The control group has to be diligently maintained, including tire pressure checks to ensure consistency across the group, and ideally they would be the same type of vehicle with identical specs. If not, create control groups to match the truck specs.

Tread Depth, Mileage, Fuel Consumption

Because fuel economy testing is all about two variables, fuel burned over a given mileage, both measurements need to be accurate. And since tire size and circumference relate directly to how the truck records mileage, you must provide accurate mileage to get accurate fuel consumption comparisons.

A tire worn to the take-off point will have a smaller circumference than a new tire, and a low-rolling-resistance tire will be somewhere in between.

"On a comparison using just two trucks in revenue service, ideally you'd equip both trucks and trailers with the same brand and model of tire, in the same condition," advises Curtis Decker, manager of product development at Continental Tire North America. "If there's any change to the test, like a failed tire or a trailer gets switched out, drop that data from the comparison."

One alternative to tracking mileage from the hub or the vehicle odometer is GPS. That's accurate, no matter the tire size. Electronically generated mileages from fuel tax programs can be useful too, provided they have been vetted beforehand and compared to GPS or actual truck miles. You just need a number, but you have to have a trustworthy number.

"Using mileage based on fuel tax programs, combined with truck miles and expressed as a ratio will give you an accurate measurement," Wessels says. "But then you have to get into some tricky math to come up with a usable number. Remember, if we're testing devices with small economy gains, a percentage point or two inaccuracy in your data can wipe out any gains the device provides, or inflate them by a similar amount."

If you think short-term testing might be better for your operation, you can find out how to conduct such a test read Recommended Practice 1102 and 1103 from the American Trucking Associations' Technology and Maintenance Council, or you can visit the SAE website and search for Type II and Type III Fuel Test Procedures.