Oil Analysis Not Scientific?

[troy_heagy]

[Note: Please don't shoot the messenger. I'm just curious what you guys think about this poster's ideas.]


==================================================
From: "bretthans"
My 2 cents:

It is interesting that they [posters at this forum] are going to such trouble, but their used oil analysis is very unscientific. One poster changes from one oil to the next in his/her engine and compares it to measurements from someone else's engine or to their old results with different mileage without regard to oil filter, leftover contaminants, oil removal method, or driving style. For example, one poster removes the oil drain plug for one measurement and uses a syphon device for another. There is no way to draw any conclusions from this.

The only meaningful test would be one with identical engines and filters running under the same temperature and load conditions for the same amount of time, each with a different oil (performed by an independent lab, not performed or funded by a manufacturer).

I have no idea how to interpret the new oil analysis and, not being a chemical engineer, I'm not going to try.

-Brett
Silver '01 5-spd Insight

 

[VeeDubb]

I guess by the strict definition of scientific, he has a valid point. But to me, the issue is not whether what people do here is textbook scientific or not but whether what we do reveals some useful information. Like anything in life, the question is not whether we are attaining perfection or not, but whether we are collecting information that improves our knowledge (even if the info is not perfect). I for one am not going to go out and spend thousands of dollars so that I can collect thousands of oil samples under the exact sample conditions and/or control for every possible factor imaginable. There is always going to be sampling error caused by a host of factors. To me that is not worth it. However, $20 bucks for some "unscientific" yet useful information is worth it to me. Even if I can't identify the perfect oil, I can still uncover major mechanical issues, if any.

If I became as dogmatic as this guy, I would have to say we should just get rid of all social sciences and any other discipline that cannot use strictly controlled laboratory experiments.

 

[3 Mad Ponchos]

Mmmm, there's more than one way to collect scientifically valid data.

One is to have perfectly controlled and comparable lab conditions like he wants.

The other is to collect hundreds, or thousands, of observations from the real world and try to make sense of it all.

You don't think evolution was explained in a lab do you? Just to cite one example.

Cheers, 3MP

 

[CJH]

In my opinion, science is using data to make decisions, rather than opinion. Of course, there is the old saying "Figures don't lie, but liers figure."

Minimizing variations among measurements is good science, and the guy has a point in that respect. However, to say that there is no science involved in comparing UOAs because the samples are drawn differently or the engines are of different design is not a valid reason to deem UOA as unscientific. What's the alternative? Depending on advertising or opinion?

 

[Gary Allan]

quote:

---

and A shows Iron at 40 two times back to back and then after a flush you run B back to back and it shows Iron at 20 two times, I'd be fairly certain that B is doing a better job.

---

Zactly! The sharing (or collecting) of the data is not for the absolute truth of it ...just the comparitive value of one oil over another under the same conditions. It doesn't have to be a subject of two identical engines under the identical conditions tested in the exact identical way. Therefore UOAs from a taxi fleet are just as "valid" as a set from a highway patrol fleet as long as the thing being compared is relative performance between one oil (filter, drain interval, etc.) over another.

The rest is just indexing and integrating the data into your application.

 

[JohnBrowning]

One thing alot of people lose site of is that we are looking for trends. It all boils down to trend analysis not specific numbers so much. The only time one individual report means alot is if the trend is disurbed or if any number are just insanely high and no trend has been established yet. Most engines of the same design tend to wear in a simalr fashion example: Nissan Maxima V6 tends to produce high lead number the first 30,000 miles. I did not know this until I started to frequent this board. If I had seen those lead reading on a UOA I would have thought the mains were going.

 

[buster]

quote:

---

It all boils down to trend analysis not specific numbers so much.

---

Very true.

 

[widman]

In the best of all possible worlds, analisis tells you a lot about how your oil holds up and how good your maintenance is. There is a lot of good analisis here, with a lot of good comments from people with different experiences. I have sent more than 1,000 samples to labs for customers and have helped them change brands, filters, head gaskets, etc. These samples have seved thousands of dollars for customers by identifying problems. One sample yesterday with sodium in it can save the engine if they react before it leaks more water into the oil. I find it valuable to see trends with different engines and different oils, but as mentioned, for maximum information you need to know the driving conditions and all the other variables.
I have a saying that I use in class: "The analysis of oil is a SCIENCE. The interpretation of those analysis is an ART."

 

[MNgopher]

Yep, the poster is correct that what goes on here is not all great science. Nobody here has the pocketbooks to do what the poster would call "science". Instead, we here use a tool (analysis), to detect trends, primarlily. After a while, it becomes pretty obvious that brand A does better than brand B in most situations. We also can see that not all engines post the same numbers. Similar to the Maxima motors, the Jeep 4.0l comes to mind, as the king of iron. Normal on that engine would send owners of other motors into coniptions!

The comment that the ONLY meaningful tests are the two cars under identical conditions has all the signatures of a laboratory weenie who never gets out in the real world to see how things are done. Its very hard to make things perfect, but that doesn't mean there isn't information that is meaningful! As a civil engineer, I see that everyday when I deal with people who don't understand how things get done in the real world. On the logic there, I should only ever look at lab experiments of Item A and Item B. Of course I would never know how it really works when Joe schmoe gets a hold of it!

 

[Mystic]

I don't know how it would be humanly possible for people at this web site to test identical vehicles under identical conditions, even if the people at this web site had the money to attempt it.

What happens to the scientific results when the identical 500 Corvettes being driven under precisely the same conditions by 500 members of this web site has one Corvette rained on? When one of the 500 Corvettes experiences more headwinds?

I doubt that Exxon-Mobil has the funding to precisely test 500 identical vehicles under precisely the same conditions, unless they do the testing in some super lab, and even then you could argue that the vehicles are not the same, different batches of oil, etc.

But the VOA and UOA done for people at this site does enable the people at this web site to have some clue as to motor oil quality. If a certain brand of motor oil consistently produces poor results, what conclusion can you possibly come to? If another brand of oil time after time produces great results-what are we supposed to think?

You do the best you can with the resources that you have. Even Exxon-Mobil has to do the same.

 

[Terry]

Great comments folks. Widman was right on. I can provide as much SCIENCE as you can afford. The ART is something I do provide for $35 for Joe Schmoe, and 9 out 10 times give him the same result in a more meaningful presentation with the trended analysis.


In interpreting reams of standardized data on a particular oil, honestly I see more meaningful REALITY with a trended less scientifically controlled test.

The ability to properly read the indicators and relationships of each element takes years to do in a automotive engine sample because of variability.

Few industry( Automotive or lube) Chemists or Engineers have the background to know enough about the lube and the engine to properly interpret the results of any testing protocal in practical application, and thats scary.

This board would not exist if they did a better job.

They do know how to formulate to barely meet the standard or barely pass the bench tests to look like the final product passes muster in the field.

 

[Jay]

Oil analysis has strong points and weak points. The strong point is that it's a real world test in a real engine. Even better, it's your engine, under your specific driving conditions. You can't get better correlation than this.

The disadvantages are that it's a very time consuming test, and because of this it's difficult to make it a good controlled test.

 

[YZF150]

The only meaningful test would be one with identical engines and filters running under the same temperature and load conditions for the same amount of time, each with a different oil

Nonsense.

If my oil analysis results demonstrate that there is glycol in my oil, and, as a result, I get a leaky head gasket fixed that I didn't otherwise know about and that could have resulted in a catastrophic (--expensive--) engine failure, then that test was most definitely meaningful to me.

There is more to this than simply crunching wear rate numbers.

 

[buster]

quote:

---

There is no way to draw any conclusions from this.

The only meaningful test would be one with identical engines and filters running under the same temperature and load conditions for the same amount of time, each with a different oil (performed by an independent lab, not performed or funded by a manufacturer).

---

There is some truth to this, but lets look at it from a different angle. There are many variables at play in any scientific study. He is right that in order to have a completely sound study across the board, the conditions above would have to be met. However, this does not mean you can't draw any conclusions about the oil. You can argue anything if you want to, but for the most part, you can hypothysize what is going on. If you run two consecutive brands, A and B and A shows Iron at 40 two times back to back and then after a flush you run B back to back and it shows Iron at 20 two times, I'd be fairly certain that B is doing a better job.

[ May 24, 2003, 12:05 PM: Message edited by: buster ]

 

[VeeDubb]

Another problem I have with this guys's logic is that it seems to be based only on a superficial understanding of what "scientific" means. He seems to think that controlling for everything is a necessary condition for good science and that simply isn't true (or is it possible). The key to it all is random sampling. For example, if we want to know whether iron content is higher under Amsoil or Mobil, we don't need to know whether all samples were were drawn under the exact same conditions; what we need to know is whether there is a systematic bias in the mobil samples vs. the amsoil samples. For example, if all Amsoil samples were collected via the drain plug, and all mobil samples were collected using a siphon, then that is a problem and it needs to be controlled. But if the method of sample collection was random, then it doesn't matter that some Amsoil samples were collected from the drain and some were siphoned. It all cancels out when the sample size gets large.

While it is nice to control for as many factors as we can (to reduce sampling variance), it's not necessary so long as the driving/sampling conditions are *randomly* distributed across oil types and you have a sufficient sample size. While we have not conducted statistical tests on this board, most people here seem to base their conclusions on numerous samples and not just one or two samples like this guys seems to think.

 

[metroplex]

He definitely has a valid point. In the scientific community, controlled testing is extremely important.

Using a syphoning device for 1 test and just letting it drain out can throw off the results!!!

In Chemistry lab, using tap water instead of distilled to wash out the test tubes and beakers can destroy an entire experiment.

For the folks who compare silicon levels - you have to remember that a slight vacuum leak can add more silicon than normal.

However, UOAs are not supposed to be scientific in that we're not proving/disproving something - we're using it to sample our oil and see whether or not our engines have major problems (excessive wear, anti-freeze found in oil, etc...)

That being said, you'd probably have to use a specific engine oil type for 4-5 changes straight using the SAME sample extraction method for each oil change in order to compare the results for one engine.

 

[troy_heagy]

quote:

---

Originally posted by VeeDubb:
(snip)

---

Hey VeeDubb, do you still have your Honda Insight?

Troy

 

[VeeDubb]

quote:

---

Originally posted by troy_heagy:

quote:

---

Originally posted by VeeDubb:
(snip)

---

Hey VeeDubb, do you still have your Honda Insight?

Troy

---

 

[MolaKule]

quote:

---

You don't think evolution was explained in a lab do you? Just to cite one example.

---

Since no one has observed Evolution, and since it is merely a theory based on interpretations, I see no validity in that theory.

Back to the Topic

Science is knowledge which can be gained through the five senses, and via a number of established methods.

Trending and statistical inference, which we are doing here, are just as valid scientifically as other methods in science.

[ May 25, 2003, 11:17 AM: Message edited by: MolaKule ]

 

[Drstressor]

Science is a method for uncovering knowledge (i.e. the scientific method). It requires postulation of a testable outcome (a hypothesis) that is put to trial either by experimentation or observation. A scientific experiment or analysis can only support or reject a given hypothesis. It can’t by itself tell you what is really going on. After many experiments or analyses fail to reject a hypothesis, it is elevated to the level of a theory, which must continued to be tested. Hypotheses that can explain all existing observations and withstand the test of time become accepted as scientific principles or laws (i.e thermodynamics or evolution). This doesn’t mean they are correct, just that they suffice to explain what we see or think we know. The point is that science is a specific way to answer questions. You can’t, for example, postulate that oil brand A is better than oil brand B in protecting an engine and prove that hypothesis using the scientific method. But you can use science to test whether there is a statistically significant difference in protection between brand A and B by postulating that there is no difference between the oils. This is what is called the null hypothesis. If the data show that the null hypothesis cannot explain the results, then the null hypothesis is rejected and alternative hypotheses must be considered (brand A is better than B or visa versa).

As 3MP correctly pointed out, there are 2 ways to answer oil performance questions scientifically using UOA data. The first is to conduct controlled experiments were all variables are the same except for the brand of oil. As several have mentioned, this is almost impossible to do in the real world. But here is how it could be done: Several identical cars would need to be driven under very similar conditions using 2 different oils. Analysis would be performed at identical intervals. The actual number of identical cars needed for a valid set of data would have to be determined in a pilot experiment that would measure the variance in wear metal accumulation between vehicles using the same oil brand. The greater the variation between engines (due to manufacturing processes), the more cars that would be needed for a valid scientific study. Statisticians do this determination all the time in what are called training studies. All of the cars would then need to start the final study simultaneously to control for variables such as temperature and road conditions and be driven under a carefully proscribed mixture of city and highway conditions. There is going to be variation wear rates as determined by oil analysis within the brand A and brand B data sets due to different engines and different drivers. But by postulating that there is no difference between the mean values for wear metals between the brand A and brand B data sets, statistical analysis will either support or reject the hypothesis.

The second way to use science to extract useful information on brand performance is to collect a very large amount of uncontrolled UOA data (different engines, different sample intervals, etc) and run what is know as a meta analysis. There are statistical techniques that will look for trends in these type of data. In general, they work by sorting out all possible variables and comparing each against all other variables. To be valid, a very large number of samples are required. I have actually used one of these techniques on a set of 100 UOA results and was unable to find any correlation between oil brand and wear metal accumulation rates. The only correlation that was statistically significant was between miles on the oil and wear metal accumulation. But this was not a scientific study because the sample size was too small. Again, the sample size necessary to see a meaningful correlation increases with the amount of variation seen with a given brand of oil.

So is UOA scientific? Not in the formal sense. But that doesn’t in any way detract from the usefulness of UOA for predictive maintenance. It is a science based methodology just like medicine. If a physician always waited until all possible test results are in before making a decision, he would loose a lot of patients. Most of the impact of science on human activities is through science based methodologies (engineering, drug approval, safety standards, etc) rather than pure science.