BLAINVILLE, Que. –So you want to do a test to see if a product, say a trailer-mounted aerodynamic fairing or a vibrating gizmo patched into your fuel line that reduces fuel burn?

Could it be any harder than running an unmodified rig (control trip) to Houston and back while tracking miles driven and diesel burned, then taking the same trip again with the product installed (experimental trip), then comparing the difference in the fuel burned on both trips?

Truth is, conducting experiments is a very specialized type of work that researchers spend years learning how to do and then invest much thought and preparation in every experiment they carry out. Even experts make many mistakes and a key job of other scientists in peer reviews is to look for flaws in their work.

Take those runs to Houston. In order to be confident that the experiment will reveal any changes in fuel consumption that can only be attributed to that new product, everything else about the trip must be kept exactly the same: driving style; engine condition; route length and conditions; load weight; wind; idling time; tractor and trailer type and configuration; fuel measurement equipment; fuel temperature; tire pressure; windows up or down; air-conditioning running or broken; season -just to name a few.

The items in that list are called variables, because they can vary. As they vary, so can fuel consumption. The researcher’s job is to think of every variable imaginable and then attempt to keep them from varying: the only thing that should distinguish, or make those two Houston trips different, is the product itself.

Otherwise, any difference in fuel consumption could arguably be due to uncontrolled variables. The rules of experimental methodology state that no amount of huffing and puffing can make uncontrolled variables go away. Doubt is doubt. It is not acceptable to say,”Oohh, that! That variable doesn’t matter.”Who knows what matters without doing a test to see what matters?

Let’s say you do measure a 5% difference in fuel consumption between your control and experimental trips. That’s exciting, right?

Look again: what if over the previous 300 trips fuel consumption varied by as much as 9%. Now your 5% means nothing for sure. Or what if you measured no difference in fuel consumption in the control and experimental trips? Is the product ineffective, or is it because your driver got sick in Buffalo and Mr. Lead Foot got behind the wheel to finish the trip? Making sense of results, or lack of results, is not straightforward.

“Operational observation demands a lot of care regarding the recording and analyzing of the fuel consumption, distances, routes structure, loads, drivers, etc.,” says Marius Surcel, research engineer with FPInnovations -Feric division. He suggests “There should be at least three months baseline, without technology and six months test stage with technology installed on the test vehicles. There should always be at least five control and at least five test trucks, sorting the trips, using similar routes, similar loads, the same drivers for the test-control couples.”

Even having run a sound experiment, the data has to be analyzed according to generally-accepted rules. There are specialized statistical tools that are appropriate for different experimental designs, sense that has to be made of the numbers…this is a speciality area of its own.

So what is a person to do? Go to product manufacturers, who are armed with “statistics” that “prove” that their gear saves such and such an amount of fuel? Nothing says they are running good experiments either.

The ideal alternative would be to go to independent, trained researchers who will test products under controlled conditions.

With good clean results from tests done in a simplified environment, fleets are properly positioned to ask the follow-up question: “Will this product, which I know works on the track, work for my fleet?”

FPInnovations -Feric division and some Quebec trucking companies, with the support of federal and provincial government agencies decided in 2007 to carry out independent tests of fuel-saving technologies (Truck News has carried several articles on the 2007 trials) called Energotest 2007, and this year’s trials, called Energotest 2008.

Everyone assembled at the 6.4- kilometre Transport Canada test track in Blainville, Que. with detailed test procedures and experimental designs, tractors, trailers, products that vendors were willing to have tested (they paid for this, aware that Feric would report test results, good or bad), scales, CBs, computers, top drivers and a mission to figure out which products worked and which ones did not.

Feric designed a test procedure based on the SAE J1321 Joint TMC/SAE Fuel Consumption Test Procedure -Type II.

This recites in detail how the trials should be run, and according to Feric, if it is followed to the letter, test results can be taken as gospel (a bold claim, but that is what Feric says). Interested persons can buy the test procedure from SAE International for US$60.

Here is a brief summary of how Feric ran their trials: Each test of a technology involved two trucks -an unmodified control truck and the test truck.

“The vehicles had the same general configuration and were in good working condition, with all settings adjusted to the manufacturer’s specifications. The load weights were representative of fleet operations and remained the same throughout the entire test period,” Surcel explains.

In the baseline phase the unmodified control and unmodified test trucks did at least three runs of 15 laps together around the track.

Temporary fuel tanks installed on the trucks were weighed (measuring fuel consumption by volume is inaccurate, as it changes with temperature) before and after each run.

The baseline part of the trial was considered complete once the ratio of the two trucks’ fuel consumption across three runs were within 2% of each other, that is, each truck’s fuel consumption was stable.

In the final phase, the unmodified control truck did laps at the same time as the test truck with the technology installed and the ratio of fuel burned was again calculated across three 15-lap runs.

A comparison of the baseline and test ratios yielded the percentage difference (if there was any) in fuel burn between the unmodified and modified trucks.

There is not enough space here to explain what this ratio business means, but the point is, many rules had to be carefully followed in setting up the trucks, measuring the fuel burn, how and exactly where on the track to drive the trucks, monitoring wind and track conditions, after-trial truck inspections, etc.

If a rule was violated, such as missing a gear during acceleration, a trial had to be repeated.

Feric et al believe that the ’07 and ’08 trials demonstrate conclusively how well the technologies and techniques they tested work. With half the job done, what remains is for fleets to correctly match proven technologies to their operations and figure out their return on investment. •