Why then are their tests results so varied? And they themselves say you can’t compare results between tests? .....he way tire rack says they do the tests is the easiest to replicate, as it’s just relative comparison of the 3-4 different tires in the test on that day…. I’m glad they do their testing but I think there is room for improvement, the wet braking example between the Prius and Toyota 86…. Something in their methods is a bit flawed to have results that varied?
In addition to the fact that Tire Rack doesn't use the SRTT (Standard Reference Test Tire) and doesn't adjust their values based on the SRTT's results, they also do on-vehicle tests - which means - particularly for braking tests - that the results might be influenced by the vehicle.
But I suspect the biggest difference is temperature. Allow me to give you some scenarios:
It's late March in northern Indiana. The Tire Rack guys have been getting ready for a test for the last month. They've done everything they could do that didn't involve the test track - fuel economy, ride, noise, etc. The only thing left is traction. They have to do the wet tests first, because the dry tests destroys the tires - and they have to wait until it is 45° because they can't operate the water sprayers below that temperature. - that lines may freeze up.
And today's forecast is for 45° Sure, the water will be a bit more viscous and that means the water is deeper and hydroplaning will be more of an issue - and sure, the tires will be colder and not as grippy - but it's the same for them all and the test at least is internally valid because they are testing at about the same time and temp.
Some months later - let's say late August - there's another test for the Tire Rack guys to run, and all they need is a clear day. Sure, it's going to be 90°F and the water will be thin, so the water depth is less so hyrdoplaning is less of an issue - and they are having trouble with dry spots because the water sprayers don't pump out water evenly and it evaporates, but they can just drive around them - and the tires will be more grippy - And that means the test is different than it was in March, - but it's the same of all of the tires being tested so the test will be internally valid, but it's certainly going to be different than it was in March.
On that same August day, at a test track in US southwest, a tire manufacturer is ready to conduct a wet traction test. They mounted the tires the day before, checked the water pipe flow to make sure they could control the water depth per the test spec, checked out the traction trailer for proper data pickup, so off they go. They start off with the SRTT - and get results that are expected, so they can conduct the next runs with confidence. In a couple of hours, they've done over a dozen different tires, so they end the test with another run of the SRTT - and again get expected results, so no reruns! They hate it when they get bad results from the SRTT, because that means they have to run the test again another time - and that means more mounting of tires, more expense.
The test crew sends the results to a statistician who adjusts the values per the procedure and forwards both the raw data and the adjusted data to the tire engineer who ordered the test. The tire engineer compares this test to one they conducted last year, and there's some differences in the raw data, but the adjustments made because of the SRTT makes those differences small. Even then there are some reversals, so the tire engineer reports to his management that fact when he publishes the results. Good thing they have other tests to help make some sense out of things.
