Interesting question...
Compare the damage to your paint if you don't often use car washes, and don't often wash your own at home - as the OP describes "It’s usually dirty" when someone just casually walks by your car and brushes up against it. The dry silicates, etc. that are ground into the finish...............
Now further...what if that car is in the rust belt, and you can actually get a good portion of that salt off your ride once a week or so?
I'll present a counter point. I abide by the principle that the less I touch my paint, the less micro marring it gets. I maybe wash my car about 6 times a year. I live in semi-country area and have a gravel drive. My car often has a very light dirt/dust film on it. Except for mid summer, we get lots of rain here to wash off and/or accumulate a film. I have no problems with people brushing against my car, so that's a non issue for me.
I also seldom wash my car in the winter (it's Krowned). Salt doesn't corrode anything painted or oil coated. A wash will never last more than a few days because we have such frequent precipitation - washing becomes more of a "feel good" thing IMO. I hypothesize that high pressure water maybe forces salty brine deeper into tight seams. Again, I don't have issues with people brushing against my cars to cause abrasion. A damp, warmer garage accelerates winter rust. Outdoors in the cold is actually better.
My cars still look great at 10 years and beyond. I only detail them twice annually (spring/fall). I have never needed polish to correct my paint (except accident repair). Of course they are not show car quality, but swirls are not visible in sun or bright lights unless you start "inspecting" it vs. normal viewing.
This has worked for me over 30 years. OH, the question - I only occasionally use one of those DIY spray booth washes (take my own buckets).