Does anyone question Consumer Reports study?

Status
Not open for further replies.
I think that like with all kinds of reports, reviews, road tests, etc, you have to take it with a grain of salt and read between the lines a little. Especially with subjective opinions and even measured, quantified tests because experiments are only as good as they are designed to be.

Have I seen inconsistencies? Sure I have, but some of them have reasons. For example, a new Honda model comes out. Without waiting for any data to come in they instantly predict reliability will be good. Woah! Wait, hold the phone here!! Man, isn't this the most blatant anti-American pro-import garbage you've ever seen? Well, actually no. They can say that because for the last 20 years or so every Honda manufactured vehicle has had pretty good to outstanding reliability.

I think it's unfair to bash CR as being nothing but biased propaganda, and I think their tests and comments do have their value and place - just a grain of salt remember. People would figure it out eventually and they'd lose credibility once they take their advice and find it burns them. Besides, pretty much every article you read is "biased" at least a little bit anyway. Take any review written by Car and Driver for example (one of my favourite mags BTW).

Just my $0.02....

[ June 30, 2003, 03:46 PM: Message edited by: 4DSC ]
 
CR makes for interesting reading but one can carry statistics too far. Driving a car as boring as a Camry just because it's CR's top choice misses the point for me. I feel they are biased against GM products but you don't have to accept their opinions, just weigh their data and draw your own conclusions.

As far as the 6K oil changes I would do that if I didn't have short trips--the study clearly doesn't apply to us low mileage, short trip drivers. CR may not be perfect but there's no similar service that's any better IMO.
 
Case in point: CR recommends the Toyota V6 3.0L engine as a solid performer in Avalon or Camry.

They also by implication promote running a good off the shelf oil and Fram filter. So lets say we run Toyota 5w-30 ( Mobil drive clean under contract) with that Fram for 6000 miles and do nothing else and I'll show you a bag of sludge at the end of 60,000 miles, in a taxi or on the soccer field parking lot !

I agree CR is a good tool but be careful trusting them explicitly when it comes to tribological automotive issues. This realm is so underestimated by everyone that a website like this attracts thousands to talk about oil for goodness sake ! Many who are professional automotive related engineering and chemical types. ( Our colleges stink at balanced and thorough automotive tribological training).

By the way SWRI, SouthWest Research Institute and Institute of Materials, Savant would be my choices for good testing houses on automotive issues. I wish I could compete with them on that scale.

[ June 30, 2003, 06:27 PM: Message edited by: Terry ]
 
I read that test with interest when it came out. I do not doubt the conclusions they came to, and that is because of what they tested (americam made cabs), how long (60K), and the oil change intervals used (3K, 6K). Those engines are low temp, low stress, slow revving, heavy duty ones. They run all day, and so are constantly warmed up. You will remember from engineering school that bearing and especially cylinder wear is lowest at higher temps.

In my life, the engine with the most stress is my turbo Miata. It is required to make at least 2.5 times the rated output daily and for drag strip use, 3 times the original HP. I know I have engine wear issues, I would like to see the poor quality brands hold up for 6K in my car. Heck, M1 is shot a 3K.

How about 180K miles for a good test? In my experience, parts that are just broken in are going to measure out correctly.

I feel the test was very valid, and I feel the data from the test has great value. We now know that taxi cabs are probably NOT the best choice. We also know that 60K is far too short. I suggest 3 times that distance as the benchmark for wear measurement. I also suggest varied locations and driving patterns.

Chris
 
Well said, folks. Spector and Terry hit the nail...that test was at least 5-10 years ago.

No one mentioned it (or maybe I missed it) but didn't they weld one on the cab's drain plugs and drive XXXX (?) miles?

My biggest problem with CR is their political bent...yes they have a big one (as most of us do) but when I subscribe - 12 months every 3 or 4 years - I get fed up and don't re-up.
 
I do not often read CR these days. I did in the 1980's because my Mom and Dad used it as a tool. I think in terms of their automitve rateings they are 75% right on the money!!! What really kills me is that the people that get bent out of shape about the good scores for imports do not own Japaness cars and never have( not accuseing anyone here of this)! So how do you know what is true if you do not drive the competition. I can not think of a single Domestic brand that handles better, brakes better or scores better on JDP IQ test then it's Japanesse counter part. That is really sad because IQ report is only based on first 90 days of customer ownership! Out of all the engines Toyota offers they have had a sludge problem with two engines produced dureing a 4-7 year time frame. That is really insignificant compared to all the problems we see all the time from domestics. They called Windstar right. Go to Alldata and look at the sheer volume of TSB's and recalls. Ford has had so many problems with the Windstar brand that they are completly droping it and changeing the name to Firestar to try to escape the bad reputation of the of the Windstar name. The cars are tested in Penn on the roads and at their test facility. Their facility is set up so that cars that have good brakes and tighter susepeons do well. They are not testing to see if the car will go straight on a smooth road and hold a super gulp! To date their has not been a single Domestic front whell drive car that can compete in the same EPA class as the Camary, Accord and Maxima for fit&finish, build quality, durability and owner satisfaction.

[ June 30, 2003, 09:43 PM: Message edited by: JohnBrowning ]
 
Posted by Terry:

quote:

By the way SWRI, SouthWest Research Institute and Institute of Materials, Savant would be my choices for good testing houses on automotive issues. I wish I could compete with them on that scale

What do you mean Terry? What testing do they do? Do they do oil analysis? If so, commercially or for specific research only? Is this the one in SA,TX?
Thanks, Rick
 
Both do all sorts of testing Rick. Yes SWRI is in SAT and has facilities around the country. They do testing for all the big guys in the Auto and oil industry. Test tracks with cars running all day and night etc.
Really amazing what is tested there.

CR or Cu can't hold a candle to this place.


If you are planning on hiring them get the checkbook out with lots of $$$.
 
quote:

I'll show you a bag of sludge

lol.gif
Terry that's priceless!
 
One way to test to see whether CR really is objective is to see if they mention anything about the alleged sludge build-up problems in some Toyota vehicles. If they continue to rate these vehicles highly and do not say anything about any such problems, It shows a bias (and a bias that cannot be excused, because plenty has been written about the alleged Toyota problems).

Taxi cabs may be driven under fairly harse conditions, but the worst conditions of all is a vehicle driven only a few miles to and from work, often in cold weather, with the engine not warming up enough. You need to test a fleet of different vehicles (cars, trucks, vans, SUVs) under differing conditions.

And when it comes to synthetic oils, CR did notice that synthetic oils like Mobil 1 poured at lower temperatures. Extended oil changes would have eliminated some of the conventional motor oils. I do not think that Mobil 1 would have failed an extended mileage test.
 
After reading this thread, I gotta weigh in with my opinion about CR. I'm pretty anal about consumerism and getting the best perceived "value" for a lot of things I purchase. I have often assisted friends and aquaintances identify what they really need (objective), what they really like (subjective) and what they can afford to spend when purchasing a car, computer or stereo/home theater equipment - passions of mine for 30+ years. I've also been reading CR for almost that long.

Many of those that bash CR, in my opinion, tend to have an ax to grind because CR has published something that they personally disagree with in either their expertise, values or preferences. In my experience, CR has an unmatched history of delivering reasonably objective and reliable evaluations of mass consumer goods, without being directly supported or obligated to any for-profit corporation or business. They provide product review and purchasing advice targeted to mass consumers, not boutique interests such as car and electronic enthusiasts and experts. Can Car and Driver, Road and Track, Home Theater Review or PC Magazine make support those claims, and how often have they ever flat out said an advertised product is "unacceptable?"

Nope - I sure don't always agree with CR, especially with regard to audio and video - but then again as an enthusiast I have more discerning values (that have become my needs) than an "average" user. As pointed out by others in this thread, over the long haul CR is far more often right than wrong in providing insightful things to look for when purchasing appliances, lawn mowers, cars, etc. Their reliability and frequency of repair indicies are more accurate and unbiased for the "average" consumer's use and care than most other sources - in large part because they collect information from consumers, conduct some research, and maintain accurate historical records.

Even with the availability of solid objective data, I think most of us are still heavily influenced by opinions -- often incorrectly presented or interpreted as fact -- and our own subjective preferences.

In a nutshell my long experience with CR as one of my trusted references has demonstated to me that it is a valuable and generally unbiased information baseline for consumer goods and services. For instance, CR was waaaaay ahead of the curve in providing pretty accurate and detailed suggestions on new car buying; this same general info. has become commonplace elswhere only in recent years. Those interested in digging for more info. about a particular service or good can, and should, also look into other resources like special interest publications, web sites, epinions, etc.

One last comment. I'm lost on complaints about "foreign" brands receiving higher evaluation rankings than "American" brands. Don't forget that many foreign cars and products are now made in the USA and employing our neighbors and relatives. The reverse is true with American nameplates. It's silly to suggest a bias from CR and other legitimate sources against American-branded products on the basis of national origin.

There now, I'm off my soap box...
 
quote:

Originally posted by Terry:
Both do all sorts of testing Rick. Yes SWRI is in SAT and has facilities around the country. They do testing for all the big guys in the Auto and oil industry. Test tracks with cars running all day and night etc.
Really amazing what is tested there.

CR or Cu can't hold a candle to this place.


If you are planning on hiring them get the checkbook out with lots of $$$.


Wow! I live 3 minutes from this place. All I always see on my way to work is the stinky monkeys (as I said on another thread), never imagined they had so much more going on. The place is huge though, maybe that's why I though it was only a Bio-Tech lab.
Rick

PS: The way you say it, it sounds expensive as heck.....I guess I won't be using them.....although you guys have to admit that it would be nice to do an oil change and have a UOA and VOA in just an hour.
smile.gif
 
Posted by KCDoug:

quote:

One last comment. I'm lost on complaints about "foreign" brands receiving higher evaluation rankings than "American" brands. Don't forget that many foreign cars and products are now made in the USA and employing our neighbors and relatives. The reverse is true with American nameplates. It's silly to suggest a bias from CR and other legitimate sources against American-branded products on the basis of national origin.

Well, here is my input: I cannot comment on reliability issues because I just don't know, BUT, being here in Germany for more than 3 months now I can tell you that these European cars are MUCH better designed than Americans; at least as far as interior goes and also handling. These vehicles look much better than American in the fit and finish deparment (althoug some car models are downright ugly) and the handling is superb. I drive a VW Sharan TDI minivan ...it's a 4 cylinder turbo diesel...fit and finish inside and out is awesome and it handles like a sports car (call me crazy, but it feels almost as good as my Z28). Even the American brands; GM's Opel/SAAB look nice all around. Maybe we are just used to krappy stuff in the States.
dunno.gif

Rick
 
You know how they get their data, the send a questionnaire out to their subscribers. So if someone has an axe to grind with a particular product they have the chance to slam it.

I was once a subscribed for several years. Every product they listed as poor, I had and never had any of the problems they listed. It seemed like a magazine written by importers to me.
 
Consumer Reports lost their credibility with me in the mid 60's when
they panned radial tires. I haven't seen anything in it since to
convince me otherwise.

As far as quality goes, CR and much of the rest of the media is living
in the 70's. How about rust? The Japs are still behind us on that.
 
Wow, so much to comment on ... but very little time. I'll just say that I mostly agree with Terry, Matt89, CJH, 4DSC, Cujet and KCDoug.
smile.gif


Consumer Reports is useful for automotive info (in general) but it does have its shortcomings and idiosyncrasies. They should stick to amassing data on recalls, reported mechanical problems, etc ... for the most common cars purchased. I also think, like JohnBrowning, that people who would not consider buying a foreign-brand car unfairly dismiss them out of hand for their consistent recommendation of Hondas, Toyotas, etc ... over GMs, Fords & Chryslers. Survey respondents with an axe to grind should apply equally to all brands. If hundreds/thousands of people do not feel the urge to slam the Honda Accord they purchased a few years ago in a survey ... there's probably a good reason for that.
wink.gif
I have found that their overall view of a brand or model matches my anecdotal experience despite their tendency to favor the more mundane (read: boring) cars. I've seen them complain about cars having too much power, extra firm ride, small seats in the back of a Honda Prelude, etc ... People should know what type of vehicle they are getting into ... but then again, many people are blithering idiots.
rolleyes.gif


Having said that, CR will go out on a limb where they don't belong and that taxi-cab-motor-oil test was a perfect example. They simply don't know their limitations. The type of vehicles and nature of use are both atypical, the short duration of the test (60K miles) makes it meaningless. As Ponchos pointed out, you could probably run a car on vegetable oil or ATF for 60K miles.
rolleyes.gif
And the whole test was done unscientifically with no oil analysis, etc ... It's half-a$$ed stuff like this which gives ammunition to those who want to criticize CR and they simply should not have done it. If most of us here can see that, they should have been able to see it themselves.
rolleyes.gif


Instead, here's a test I'd really like to see and it would be affordable to a group such as CR. Gather 20 or so new (and identical) Briggs & Stratton (or Tecumseh, Kohler, Honda, etc ...) 3hp or 5hp engines for a shoot-out test. You could use larger, more complicated motors with multiple cylinders, full-flow filters, etc ... but that would increase the cost of the test. Anyway, break them all in using a control oil, then refill them with a handful types/brands of different oils. Now, torture all of them under various climactic conditions, various throttle settings, various loads, etc ... Some should get more frequent oil changes than others. Just keep the variables to a manageable number given your limited sample size of machines. And be sure at least 2 of each machines get the exact same treatment so any failures or problems can't be blamed solely on an odd manufacturing variance. Analyze all the oils at each drain. When some start to blow up and/or seize, you know you are getting near the end of the test. Maybe continue the test until you are down to only one or two machines left running? Finally, tear them all apart, measure the parts with a micrometer and take pictures of everything for comparison purposes. Sound good?
cheers.gif


Having said all that, there are problems with this sort of testing. The biggest (obviously) is the engines are not automotive ones and do not use the same type of pressurized lubrication. Still, it would be vast improvement over that CR nonsense and I personally would pour over the results as if it were the 6th Harry Potter book.
grin.gif


--- Bror Jace

[ July 01, 2003, 11:00 AM: Message edited by: Bror Jace ]
 
I can remember several years ago when CR rated FRAM oil filters very highly. Around about that same time, I started to have problems with FRAM oil filters on my personal vehicles. For example, a FRAM oil filter I tried on my Toyota Corolla had no oil in it when I did an oil change-it was dry. I had to use the (expensive) Toyota oil filter.

That FRAM article on oil filters caused a lot of confusion to me. The people I considered to be experts were saying one thing, and my personal experience was saying something else.
 
Regarding advice from anyone, including consumer reports...you have to give it your own sanity test and follow the advice if it makes sense to you. Following advice blindly is asking for trouble, even if it is from an expert.

The best thing to do is educate yourself, review the available data from all reputable sources, and make your own decisions.

As far as I know, CR is the only magazine that gives reliability any consideration. You can read CAR & DRIVER, ROAD & TRACK, MOTOR TREND and the others and you get almost nothing on reliability. When I spend $15,000 to $50,000 on a vehicle, I wnat to know that it is more than just fun to drive.

I have had it with GM (I own 3 GM cars) and their constantly warping disk brake rotors, short-lived alternators and leaking intake manifold gaskets. They know about these chronic problems and just let them go as they have with so many problems in the past. Hondas and Subarus don't have these problems. Bye bye GM!

[ July 01, 2003, 02:01 PM: Message edited by: CJH ]
 
Personally, I don't believe much that Consumer Reports says. They badmouth American made autos so bad that it almost seems like they are on the take from the jap automakers
mad.gif
. Kinda wonder how many Americans they put out of a job because of this. Oh well, I have used Amsoil for the last five years now and will continue to use it along with Wix or Hastings filters. BTW, I am not an Amsoil dealer so there's nothing in it for me.
cheers.gif
 
Status
Not open for further replies.
Back
Top