So does that make the NOACK number (which MANY on here consider not only the most important spec, but sometimes it seems, the ONLY spec) just as "less than useless" as the hated on here VI number, due to the test's fallibility/non-reproduce-ability?
Whether or not it is really as
important/relevant as the VI number spec, IF it were a reproduce-able test is a subject for a whole other thread.
There have been threads on NOACK.
NOACK alone doesn't tell you anything. You can have a Group II 20W-50 and its NOACK will be very low. All it shows is that it has a very thick base oil, which means it doesn't evaporate much as a result.
NOACK is only useful if you are comparing base oils with the same viscosity. Then you can get a sense of the quality of the base oil.
Otherwise, NOACK doesn't tell you about the quality of the base oil. You can have an 8 cSt Group II base oil with a lot lower NOACK than a 4 cSt Group IV (PAO) base oil. Despite that, the Group IV oil will have much less traction (surface) friction, much higher oxidation resistance, much better cleaning, etc.
NOACK is primarily important in lowering the oil consumption but if it's already less than 10 - 13% or so, it doesn't help much with oil consumption to lower it further.
NOACK doesn't determine an oil's cleaning ability or durability (oxidation resistance).
If you understand to compare NOACK for only similar base-oil viscosities -- such as 10w30 vs. 10w30 but not 10w30 vs. 5w30 -- than it could be a good indicator of base-oil quality, even though it still won't be an absolute indicator.