Originally Posted By: SteveSRT8
But I also like to sift the numbers, and then decide for myself. CR's scoring system was designed for 2nd graders...
I don't know if I'd say that it was designed for 2nd graders, but it's certainly designed to appeal to a base of readers who are probably NOT technically-minded. In fact, that sort of describes their readership by definition; most technically-astute people will find other, and more data-rich, resources. I think like an engineer and an artist would have a terrible time trying to make heads or tails out of a table of numbers I might give him or her. Just as I don't understand the world of art one bit.
I recognize that it's got to be a challenge to devise a scoring system that is applicable to nearly every type of product out there, demonstrates the relative ranking among the different products tested, and can be easily comprehended by those not familiar with insider jargon. Interestingly, the type of scoring visuals that CR uses, called "Harvey Balls", isn't used only by them. In fact, it was invented by a man named Harvey Poppel who worked for Booz Allen in the '70s. Today, they're used by a variety of organizations and agencies to give a quick visual representation of data. You can also summon these symbols up in popular productivity software such as Excel and PowerPoint.
Another criticism I have with CR's data is the total score of a product is a weighted average of the individual metric scores. CR, being who they are, weight safety-related factors higher than others, so Wet Braking, for example, gets a higher weight in the score than does Ride Quality. Fine...but they don't tell me what the weighting system is. And I may not agree with it. Many years ago, I took a complete tire testing results table and assigned a value to all of the Harvey Balls. An Excellent got a 5 and a Poor got a 1. So I made a numerical matrix of the tires tested and I could then apply my own weighting to the table to see the tire that would best suit my preferences. Or I could apply no weighting and see what the straight average was. The tire that my spreadsheet would recommend was sometimes very different than their "top pick" based on what weights I gave each performance metric.
So...I think in numbers and certainly want more data from them. I also understand why they don't present it numerically, even if I don't agree with it. Yet another suggestion commonly made to CR is to make the empirical data available to someone if they want to see it. Their response went back to them not wanting a reader to assume a certain level of performance based on a number rather than a relative ranking.