Does anyone question Consumer Reports study?

[Oilmeup]

What's the most expensive product you've ever covered?
Motor oil. To evaluate motor oil and we went down to New York City, commandeered 75 taxis, changed their engines, and put oil in religiously every 3,000-6,000 miles. Then we took the engines out of the taxis, looked at the engine parts to see how much wear there was on the pistons and how much carbon buildup there was and concluded that they all did fine for 60,000 miles. There was no sign of wear on any of them. So our advice was you can change your oil every 6,000 miles rather than 3,000. And we said you can buy the cheapest oil you can find. You don't have to worry about the $2-a-quart whiz-bang oil, because that's what we tested and it wasn't any different.

Now it's very simple to say in a paragraph, but when we did the test it was very scary because we entered into something that cost us well over $300,000. Suppose the experiment failed? I was very pleased about that motor oil project because it answered a burning question. All the ads about the best oil—you don't need to spend that money and you don't need to change it as often because our experience with these taxis demonstrated fairly conclusively that it's not necessary.

----------------------------------------------

Now, I don't mean to start any trouble. But it would really suck if we are doing all this stuff for nothing.

 

[mormit]

What do they mean by "it was very scary because we entered into something that cost us well over $300,000. Suppose the experiment failed?" What would they consider a failure?....That it not support their preconcieved notions regarding motor oil? CR lives on stating that low tier Brand X performs as well as expensive Brand Y.

How can the experiment fail unless you anticipate a result. An inconclusive result is just that, inconclusive. A true scientific collection of data should be neutral and support what ever result the test concludes.

Experiments can be designed to give support to any theory you wish. Their experiment supported the result they wanted so they were " very pleased about that motor oil project".

I think CR has their expectations as to how certain products should perform and will bend the norms of statistical analysis or the scientific method to get what they expect. I've referenced CR at the local library on occasion before a major purchase. Have not found it to be of much use.

[ June 29, 2003, 10:27 PM: Message edited by: mormit ]

 

[RussellA]

Now let's see them do this with 75 soccer moms driving Toyota Sienna Minivans with the sludging V6 loaded with kids and all their junk.

My point is there is probably a very limited number of types of cabs and the driving habits of cab do not represent all types of driving habits. For one I doubt they ever get any moisture in a cab engine, they get hot every day and stay hot all day long.

 

[Terry]

CR's lube testing was not a major player IMHO. It is dated and flawed. I agree the testing was not real scientific. They backed up none of it with lab analysis to correlate the engine raters conclusions. Many folks went out and used Fram filters( single pass test only) with off the shelf oil thinking their engines would go forever with minimal wear and I've been looking at UOA's since that report came out in the late 80's and CR was wrong.

Maybe if you trade your car at 60,000 miles of service you wouldn't care much. Problem is the car would sure run better and use less fuel in that 60K interval if using a better quality oil and filter.


The SL GF3 oils of today will perform more adequately but the oils in 1987 or 88 sure did NOT. The updated report in the mid 90's wasn't much better.

Just my opinion.

 

[JeepZJ4.0]

These are the same people who said the Jeep Grand Cherokee is the most unreliable vehicle sold in America..... I have no respect for them.

 

[nicrfe1370]

Here's to Terry speaking his mind and destroying any credibility that CR may have hoped to have for this test.

Thanks Terry!

 

[Dan4510]

CR has done the same for good stereo equipment. To them anything over $300 for a set of speakers is wasted money. That simple conclusion spoke volumes to me about the flawed testing mentality of the magazine.

To me, they continue to have perpetrate this same flawed testing, especially about motor oil. Police cars and taxi-cabs are usually purpose built to heavier specs than regular cars, larger radiators, engine oil coolers, and trans coolers that give them inherent advantages. A better test would have been to use cars regularly available to the public withouth these enhancements.

If I remember right, I think cr recommended the vega as a good buy....

Dan

 

[UncleS2]

Another thing: it's generally agreed, I think, that cold starts are one of the major causes of engine wear. Yes?
Well, the CU test almost eliminated cold starts. I remember reading it when it was first published- they kept those taxis in service constantly. So, you eliminate the major cause of wear, then state the oils all produce the same results. Yup. Sure.

 

[JeepZJ4.0]

Same people who said that the Ford Windstar isn't reliable.... hmmmmmm once again, they were wrong.... its been just as reliable as the Toyota Avalon (well, the Ford has about double the mileage) and the costs are equal. Both dealer maintained and a 50K mile Windstar and a 24K mile Avalon have cost the same to maintain.... something to think about.... doesn't matter, my Grand Cherokee rules

I'm going to Mobil 1 Synthetic next oil change. I'm running Clean Drive Plus right now.

 

[harper]

quote:

---

Originally posted by JeepZJ4.0:
Same people who said that the Ford Windstar isn't reliable.... hmmmmmm once again, they were wrong.... its been just as reliable as the Toyota Avalon

---

If you read the statisical reliability data it does not mean that every car with low reliability ratings will be unreliable or for that matter a well rated one reliable. It just means that based on their statistical sampling (no one else does this) your probability of getting a reliable car can correlate with other owners problems also. No one else gets a data bank like theirs. Its not the end all be all of data but no one else does this comprehensive of surveys at least for this data set.

[ June 30, 2003, 07:06 AM: Message edited by: harper ]

 

[Matt89]

I think the biggest outcome of that CR study was that "Stop & Go" driving is not nearly the menace that Jiffy Lube/Pennzoil etc would like us to think it is, as long as your cooling system is on the job.

With no cold starts, the study was not very realistic for everyday people. I hadn't thought of it until Mormit brought it up, but they definitely went into this study with the "desired result" in mind. Too bad really.

 

[Chris 2421]

Consumer Reports is junk info.It's biased and unreliable information.They slam Chrysler products every chance they get.They also say that RCA products are not any good.Well I have several of each brand,no problems to "report" here.

 

[CJH]

quote:

---

Originally posted by Chris 2421:
Consumer Reports is junk info.It's biased and unreliable information.They slam Chrysler products every chance they get.They also say that RCA products are not any good.Well I have several of each brand,no problems to "report" here.

---

I have to support CR in general. I have 3 GM products and they bash them too. However, all the problems (brakes, alternators, etc.) I have with these cars are identified as big problem areas in CR. I also bought a new Honda Civic Si in 2000 based on their reliability projections and so far it has been problem free. There are some areas where I strongly disagree with CR comments, but their overall reliability projections seem to be right on.

 

[timzak]

I haven't seen the article. Do they document anything? Photos of engines being put in, then taken apart? Any lab results, or just "naked eye" examination of the broken down engines? Any receipts printed for purchased engines and labor to install?

Based on the little paragraph that started this post (I can't tell if it's a direct quote from the magazine), it sounds like they just came to a conclusion, then *said* they tested all this stuff to support their conclusion. Was there more to the article than what's in the first post of this thread?

 

[3 Mad Ponchos]

60k test is nothing, you could probably get an engine to run on vegetable oil for 60k miles.

Cheers, 3MP

 

[G-MAN]

Here's my problem with CR: They have a limited staff who think they are experts in EVERYTHING and are capable of objectively testing EVERYTHING. Translation: They are full of crap and 90% of what they print is useless. JMHO, FWIW.

 

[Schmoe]

You got to respect that CR is ACTUALLY doing something and using some type of controlled tests when they make their comparisons. I mean, who the #ell else does these things? When they start dogging a particular car, it is based on complaints from ordinary citizens such as ourselves. Yeah, I too own stuff that they said was trashed and managed to get a lot of good respectable years out of it. Like my Dad said, check and see if it was made on a Monday or Friday. Makes a difference. Think about it. But, like I said, at least they are trying to be unbiased and you got to appreciate and repect that. I mean look at the New York City test. They took the same cab, same engine, same driving conditions, same oil change frequency and just changed the engine oils. Seems pretty fair and uniform to me. It's all good.

 

[Chris B.]

CR is CRAP! I have found all kinds of inconsistencys(spelling?)! You know how they give cars an orange circle filled in when it is good or an empty one if it is bad. Well I remember when I was reading about a Suburban(and several other car) they gave it all orange circles in all catagories and then in the write up they bad mouth it saying it is not know for it's relyability??? I don't think they even read it befor it goes to print. Total crap...all Jap cars thumbs up all American cars thumbs down!

I wondering who is paying them off?

 

[bretfraz]

Per the May 03 issue, CR has 17 staff members in their Technical Division who conduct auto-related tests. So the comment about a "limited staff" is inaccurate.

They have 131 total staffers in the Technical Division. By comparison, Road and Track magazine has a total of 29 people on their staff who create the content of the magazine.

CR may not be perfect by they have BY FAR the largest staff of engineers, analysts, planners, project managers, product buyers, and statisticians of any consumer-focused publication on the planet. Not bad for a publication that is not funded (read influenced) by advertisers.

 

[Spector]

The taxi report at issue, I believe, is verr very old as Terry noted and I believe that it did not test any synthetics.

I think the intent of the test was to show that oil did not have to be changed at 3000 mile intervals, I think that most people would agree with that now, well most people on this board, certainly not true of the zillion of owners lined up at the quick lube joints.