When I first started working on cars (mid 70's) the mechanic I learned from showed me how to use a drop or utility light and shine it through the backside of an air filter. He told me that if the light could be seen reasonably through the filter, then it did not need replacing. Please remember though that this was in the day of carburetors and round canister air filters.
There were no mass air flow sensors, electronic fuel injection, or variable valve timing engines. That being said, my new rule of thumb is to peel back the pleats and see what debris is in the folds. If it looks small and minute then I give it a light tap or vacuum and put it back. I also keep a new air filter on the shelf to also compare it to visibly. Paper and synthetic elements can discolor over time, from find dust and fallout since most intake air now comes from outside of the engine compartment. If the element paper is discolored, then I replace it.