I have a distinct disdain for "tests" like those LSJr runs because there is essentially no ability to claim something is better/worse than another lube based on one or two UOAs. And I mean none, zilch, nada, zippo ... To be fair to him, he's not the only person who puts way too much trust in a few UOAs; it's a problem most everyone practices and that includes most BITOGers.
UOAs are great tools, but they are so very much misunderstood, misused, and misinterpreted. The technology itself (in this case, ICP) is very robust and known to be fairly accurate when properly set up and calibrated. But that's only part of the story.
The problem comes with having a sample size of one (or a small set). This is very, very poor methodology; it completely ignores the topics of product variation, process variation, and gauge R&R. It is of the utmost importance to not only understand averages, but the variation of anything you test/track.
Looking at one or two UOAs and thinking one can claim something is better (or not) is a fool's errand and really illuminates the limitations of LSJr's understanding of the UOA as a tool. Whereas you can look at a singular UOA and compare/contrast it to other large groups (macro analysis), it is completely useless to take a few samples from one source and think you understand the standard deviation of the application in micro analysis. As the sample size drops off, the reliability of the stdev calculation (accuracy of result) plummets; going essentially parabolic in its degradation in both positive and negative directions with fewer and fewer samples. It takes a minimum of 30 samples to even remotely get a decently reliable stdev value; and 50 would be better.
For him to claim "
statistically something significant" happened in a couple UOAs is, without any doubt, an indication of his total lack of understanding of statistics and proper testing methodology. I abhor this kind of summary; it's just junk science.
I would remind folks to read this to understand where UOAs can help us, and where they can be misleading.
Reviewing UOA Data Used oil analyses (UOAs) are tools. And like most tools, they can either be properly used or misused, depending upon the application, the user, the surrounding conditions, etc.= There are already many good articles and publications in existence that tell us how to interpret...
bobistheoilguy.com
.