Credibility, trustworthiness of Consumer Reports

[alarmguy]

My problem is not that I think they are biased because someone bribed them. My problem is I don't think they are the expert in what they are testing against and they are not reviewing things that are the most concerning to me.

Obviously as to who should I trust instead, for me today would be the internet forums of things that hobbyist and experts hang out in, like BITOG for oil. There are plenty out there for everything. I would also trust the opinion of people using things they buy for a living more than those who only buy things because that's the right things to do (like construction workers instead of men who think men should drive pick up trucks).

I don't know, depends on what you value you should ask people with similar value and opinion as you I guess? and find where they hang out and ask them?

I respect what you say above, we all chose what works for us.
We all use what source we feel comfortable with. I have read Consumer Reports for decades on end and know how to use the information it provides.

Example would be vehicles for major failures. Some of that information is spot on. I use the information and form my own opinion onto what is important to me, not blanket recommendations. No way can forums give that cross sample to me.
Another example would be many hundreds of people (or more) reporting back nation wide about a transmission issue in a particular vehicle or engine issue. I wont get the cross section of people anyplace else.

Bottom line I look for faults in a product not recommendations provided by them or anyone else for that matter. If I did, I must likely would not own GM cars. *LOL*
However I have purchased in the past Hondas, Subaru, Nissan and all let me down in one major way or the other. Many times those faults showed up in CR's reliability ratings.
With our GMs I look for major stuff, not the minor stuff and never had an engine, transmission or exhaust system issue. If I saw a particular model or engine I would avoid it.

 

[mark pruett]

Awaiting Krapstorm, I agree with you, may God have mercy on your post.

Oh, I'm not beating the drum for the Ridgeline!

My point was that the CR reports mirror the industry press opinions for the same reasons, yet at the same time the truck-buying public prioritizes different things. And often those things aren't actually realistic or even logical.

Which makes the CR reviews on trucks very useful for people choosing their mid-sized trucks for rational reasons, but utterly useless for people who buy them because they look macho, or have macho towing and off-road capability that they rarely, if ever use.

 

[Pablo]

They give the impression their sample audience = truly large population ie random and large. Which of course is fairly impossible statistically speaking.

Does this mean the information is useful or useless? No, it can match real world but we don’t use it as much as we used to.

 

[ArrestMeRedZ]

...Which makes the CR reviews on trucks very useful for people choosing their mid-sized trucks for rational reasons, but utterly useless for people who buy them because they look macho, or have macho towing and off-road capability that they rarely, if ever use.

Mark, are you saying that if you want a girly looking truck that can't tow or leave the pavement, get a Ridgeline? That's a pretty harsh evaluation of Honda's truck! What next, are you going to call it a minivan with a bed?
Just funning with you. You are correct in that many people buy a truck to use as a heavy duty sedan that sits up high. A Ridgeline would fulfill those needs just fine.

As to Consumer Reports, I used to subscribe and use their reviews a lot. Over time it provided less and less value to me. I believe they couldn't be bought, but there seemed to be, IMO, a non-scientific bias creeping in. I was never a big fan of their car reviews.

 

[IndyIan]

They give the impression their sample audience = truly large population ie random and large. Which of course is fairly impossible statistically speaking.

Does this mean the information is useful or useless? No, it can match real world but we don’t use it as much as we used to.

I'm not a statistics expert, but it is surprising how small a sample can be and still be a good representation of a population. They do get more surveys back from car owners than how many people prescription drugs are tested on... 20-100 people are all that's needed to establish drug safety and dosage, and move on to further trials, for effectiveness, then its 100-500 people and if nothing bad happens and it seems to work maybe a couple thousand people are in the trial before it gets sold to the public... So 500 reports from 2015 Camry owners every year is statistically pretty good

Also the CR surveys are from their subscriber population and their usage profile, which is better than random, if you are a CR subscriber with a similar usage profile as the folks the survey sample is collected from. Yes they don't review 1 ton pickups used in the oil patch, because they don't get many of the surveys back from those owners.

I do wonder about their ability to predict reliability for new models, but I guess if they do some research on parts suppliers and have looked at parts on 1000's of cars over the years, they probably have a good idea which company went for max profits and lower quality parts on a model?
That would be interesting if they called out manufacturers specifically on bad design or parts quality, predicting specific failures? I guess it probably get them sued. But I'm sure and the automakers keep track of parts quality from suppliers at specific price points. Ford managing to screw up the toyota designed hybrid CVT transmission in their vehicles shows that some companies have different priorities.... And maybe why Ford's are rarely predicted to have good reliability?

CR aren't perfect but probably better than anything else.

 

[Pablo]

Samples can are good but the inherent bias of the subscribers can’t be ignored

 

[mark pruett]

Mark, are you saying that if you want a girly looking truck that can't tow or leave the pavement, get a Ridgeline? That's a pretty harsh evaluation of Honda's truck! What next, are you going to call it a minivan with a bed?
Just funning with you. You are correct in that many people buy a truck to use as a heavy duty sedan that sits up high. A Ridgeline would fulfill those needs just fine.

As to Consumer Reports, I used to subscribe and use their reviews a lot. Over time it provided less and less value to me. I believe they couldn't be bought, but there seemed to be, IMO, a non-scientific bias creeping in. I was never a big fan of their car reviews.

Well, I'm a good example of someone who doesn't tow anything, rarely off-roads (and when I do, it's because of cruddy dirt roads on boy scout campouts), but frequently hauls ~750-1200 lbs of dirty garden stuff like compost, manure, and mulch, as well as boy scout gear and passengers.

So a Ridgeline is perfect for me, but a lot of people I know wouldn't consider one because it doesn't seem competitive stats-wise, and don't realize how much nicer the ride is. That, and they're concerned about the macho police giving them a hard time about it being "not a real truck". Which is BS, but it's a concern of theirs.

 

[Cujet]

CR review of the Corvette:

Adequate acceleration, cramped quarters, uncomfortable seat fails to recline, twitchy steering, harsh ride on all but the best roads, smooth engine that is unusually noisy when loaded, lower than average predicted reliability. EDIT: Limited visibility

(yes, it's a joke, those are my words)

 

[alarmguy]

CR review of the Corvette:

Adequate acceleration, cramped quarters, uncomfortable seat fails to recline, twitchy steering, harsh ride on all but the best roads, smooth engine that is unusually noisy when loaded, lower than average predicted reliability.

^^^^ I suspect you posted a joke above, but I’m not sure

I always like to post a source so here is mine in the interest of accuracy.

“

Latest Model​

2020 Model Redesign Year

The eighth-generation Corvette uses a mid-engine design, with its 495-hp, 6.2-liter V8 visible under the rear glass hatch. It's coupled to a rapid-shifting eight-speed, dual-clutch automatic that contributed to its 3.4-second sprint from 0 to 60 mph. Handling agility is superlative, with immediate turn-in response and almost zero body roll. Yet the ride is tolerable, thanks in part to the optional adaptive suspension. Rear and side visibility, as well as cabin access, are severely hampered. The front trunk is small, but the cargo area aft of the engine can hold the removable targa top and a soft-sided bag or two.
The E-Ray is a hybrid version that has AWD capability. The high-performance Z06 has a 670-hp, 5.5-liter V8. BSW and RCTW are optional. 2025 saw the introduction of the ultra-high-performance ZR1 with a twin-turbo V8 making in excess of 1,000 hp.”

Source
https://www.consumerreports.org/cars/chevrolet/corvette#latest-model

 

[mark pruett]

My problem is not that I think they are biased because someone bribed them. My problem is I don't think they are the expert in what they are testing against and they are not reviewing things that are the most concerning to me.

Obviously as to who should I trust instead, for me today would be the internet forums of things that hobbyist and experts hang out in, like BITOG for oil. There are plenty out there for everything. I would also trust the opinion of people using things they buy for a living more than those who only buy things because that's the right things to do (like construction workers instead of men who think men should drive pick up trucks).

I don't know, depends on what you value you should ask people with similar value and opinion as you I guess? and find where they hang out and ask them?

I think the catch is that one person's "expert" isn't someone else's, and we don't have a good consensus on what is and isn't an expert.

I mean, we'd all agree that tribologists and automotive engineers would be good experts for motor oil. But past that, what about mechanics? Fleet managers? I mean, what would a mechanic actually know about motor oil? Just anecdotal word-of-mouth type stuff. Fleet managers might (and it's highly speculative) have some better handle on it due to reorder rates, higher repair rates elsewhere.

Hobbyists are notorious idiots, because we all get into patterns of groupthink, "old way is best way" thinking, and counting angels on the head of a pin type arguments. I mean, none of this oil stuff actually matters in the typical lifespan of a vehicle. The rest of it is going to fall apart around the engine in most cases, and if there really is a discernible difference between Castrol and SuperTech, you'll only see it at some absurdly high number of miles. Or we'll argue about using SAE 30 or 40 in cars specified for 20, despite decades of evidence that 20 works just fine for the overwhelming majority of engines in nearly every usage case. It's what hobbyists do; on PC forums, they'll argue about 5-10 FPS in games, on cyclist boards, they'll weight-weenie about 2-3 oz here or there, instead of just making sure to take a dump before they ride, etc...

 

[ecotourist]

Samples can are good but the inherent bias of the subscribers can’t be ignored

My guesses as to the CU reader demographic (which is the demographics of the "sampled" population):
Over age 30.
Probably not very poor or very wealthy, but typically average to better off.
More educated than average.
Concerned about quality, durability, reliability and safety.
Not especially flamboyant.
Likes to check things out before making major decisions.

If that sounds like your demographic, CU reports should be meaningful for you.

But here is an important principle - volunteers are not average. They typically have better health and better outcomes than the average population for example. And CU's main information source is the "volunteers" from a particular demographic (something like my guesses above).