Then I guess, the answer depends on the specific circumastances and application.
As a dissenting personal story:
two years ago I changed the fuel filter on my old '93 Camry with over 150,000 miles. I changed it just because I figured the mileage was so high, there HAS to be some junk in there restricting flow.
Well, I, of course, cut open the original filter (with 150,000 miles) and I found that there was just about nothing in the filter...I couln't believe it, and I probably wouldn't beleive had I not done it myself. And owned the car since new.
Anyway, every situation is different.
Again, I feel rather strongly that changing the diff fluid is more important than changing a fuel filter.
But, if I heard my fuel pump struggling, I would certainly change the fuel filter before I did anything else.