Originally Posted by kschachn
Originally Posted by dnewton3
I will give $100 to the first person whom can prove that there is a true, discernible real-world difference between the EG, TG and XG in normal use for 5k miles in a normal, well-running car.
I am not talking about lab studies (HALTs) which manipulate the conditions to bias an outcome. I'm talking about real world data, analyzed with accurate statistical processing, where we would be able to discern the filter selection via the wear rates, outside of "normal" variation and gauge R&R.
For the most part there is never anything such as a "real world" study. When you do try such a test, the statistical analysis shows you have invalid data due to uncontrolled variables.
And a proper laboratory test is never something you can "manipulate the conditions to bias an outcome", if you're doing that then you should be fired as a laboratory technician. Proper laboratory tests such as those produced by the ASTM or ISO are specifically designed to isolate the variable under test and eliminate bias. Are you staying there is a fatal flaw in ISO 4548-12 procedures that produces systemic testing bias or are you saying that the testing facilitates all inject bias?
I would disagree; there are plenty of "real world" studies.
Most often, they come from warranty claim data and/or use data such as UOAs, PCs, etc.
I work in the HVAC industry. Where I work now, we do lots of HALT testing. There are times when the HALTs do a decent job of predicting relative performance deltas; there are times they do a horrible job because real world warranty returns show something completely different. HALTs are tools and have both pros and cons; they are not perfect (and I don't think anyone is claiming them to be). Further, statistical analysis can be done in both a micro and macro sense. Macro data analysis takes into account all factors of operation, environment, severity of use, etc. The "real world" variation is accounted for, and in fact is preferable, because it represents what actually happens in life. Having "uncontrolled variables" as you call them does NOT invalidate statistical analysis; it only must be acknowledged as part of the study analysis protocol. For example, I cannot control environmental elements such as the weather which might affect an outdoor football game, but I can include the effects of weather as an uncontrolled variable and account for it in the overall effect of results. The variable (weather) is not controlled, but can be tracked as a variable non-the-less and therefore used in the analysis as yet another input.
When I say "bias" in testing, I am talking about the conditional parameters chosen for the HALT environment. For example, the infamous GM filter study (SAE 881825) ... They tested filtration efficiency and theorized that tighter pores would make for less engine wear. So they set about testing their hypothesis. But, to achieve the HALT time schedule, they grossly accelerated the induction of wear-inducing particulate AND (this is important here) eliminated the effects of OCI (they never changed oil for the entirety of the test). The dirty little secret that they don't discuss openly is that the amount of contaminant (fine AC dust) they induce is the equivalent of more than a lifetime of ingestion for the common vehicle). The "bias" comes from the fact that they purposely negated a very integral and important part of "real world" vehicle use; no one I know of runs a normal full-flow filter with no bypass element for an OIL CHANGE INTERVAL OF FIVE-HUNDRED-SEVENTY THOUSAND MILES. The "bias" is due to the choice of eliminating a very common component of typical "real world" ownership; oil changes are paramount to engine maintenance. There are several variables which factor into engine wear; TCB/oxidation, viscosity, particulate load, oil change interval, filtration efficiency,etc. To focus on the filter efficiency as the desired variable and to get it done in a sensible time frame, they manipulated the OCI to an unnatural state; they NEVER changed oil in the equivalent of 570k miles! That, my dear sir, is intentional bias. I am not claiming that the bias is "wrong", but it must be acknowledged to understand the true nature of the results. IF and only if you intend to run a 570k mile OCI, and IF and only if you intend to change your filters when the dP gets to 10 psid, THEN the GM filter study has direct application to your world. But for me (and essentially the rest of the real world), the OCI duration has a MAJOR impact on wear rates. GM even acknowledged that the test result conclusions would not be seen in the real world. I quote the study ...
"It is important to note that this analysis is used only to compare relative wear rates. Used oil analysis from engines in the field will not typically show such a clear correlation since wear metals generated between oil changes will be at much lower concentrations."
Allow me to break their statement down into short segments ...
"It is important to note that this analysis is used only to compare relative wear rates ..." - what you see here in the results are only applicable under these relative conditions
"Used oil analysis from engines in the field ..." - real world data collected from actual use in normal application of everyday life
"will not typically show such a clear correlation ..." - are not going to be aligned and have little to no alignment
"since wear metals generated between oil changes will be at much lower concentrations ..." - we never changed oil and you always will, so the wear metal disparities we tracked to conclude something "better" won't exist in your garage.
In other words, the relative performance improvement in using better filters is only proven in the relative relationship in the test; real world use will not reveal such disparity because filtration cannot induce the same effect when the OCIs are included. GM basically took an element of real world use and then purposely negated it (causing bias) by casting it into a non-normal state; they created a grossly exaggerated "constant" condition which never exists in the real world. And so, when I say "bias", it is completely accurate to make the claim as I do. They "bias" the results by making choices of what variables to include and exclude, and what elements will be held "constant" and in what manner. By excluding a paramount component of typical real-world maintenance (the OCI), they biased the result of the test. Had they included OCIs in the test, it is highly likely they would have not been able to discern the difference in wear control. So, my long-winded point is this: HALTs by their very nature have purposeful bias in that the DOE criteria causes one to choose to include and exclude certain conditions which would otherwise alter the outcome of the test relative to the expected outcome in "normal" use. The bias is not within the intended manipulated variable; rather, it exists as the both the inclusion and exclusion of all other contributors of "normal" conditions which have been purposely altered to enhance the effect of the intended variable in the HALT.
In summary:
- there most certainly are "real world" tests; every person whom uses a product is doing so, and warranty data and/or macro-use data gives a good indication of what really happens in our lives.
- there most certainly is bias in HALTs; it is part of the DOE choice protocol and must be taken into account to understand the results and how they will and will not affect our real world experiences.
We shall agree to disagree, then?
