There is "some" truth to this mentality... but it almost certainly won't affect an internal combustion engine in a disastrous way. Now, if we were talking about some of Shannow's turbines, and you dumped in 500 gallons of oil straight out of a jug without using a pre-filter cart, then you may actually be doing significant damage.
If you hop over to the VOA section, I had 4 VOAs done with particle counts. Ravenol was the cleanest, PUP was 2nd, Motul 4100 Power I believe third, and Delo XLE was the the worst. Even though the ISO numbers were only 4-5 different, this is a count of THOUSANDS of additional particles of a given size, and the Delo actually had some particles >50 microns! Now... prefilling most filters will take somewhere between a pint and a quart to fill, and put these particles on the "clean" side of your filter. If your oil tested very clean on the PC, you have almost nothing to worry about because 99.99% of everything smaller than ~15-20 microns is going through the filter anyways.
Me, I am a stubborn bird. All my engines are PFI, all my engines are non-Dexos nor require "special" oils, and I prefill my filters because I choose to try to prevent 1-2 seconds of metal-on-metal contact as the pump builds pressure throughout the engine after an OCI. However, you can do whatever you choose. Filling or not prefilling the filter on a car is not going to be the thing that keeps it from not making it 300k miles.