I'd go with the Crown Vic no question. It's a proven car, with a low service cost, and IMHO the best all round police car on the market today. The Charger has a too small trunk, windows that are too small, unproven relability and durabilty record, and I am certain it will be more costly to maintain. Although the V6 Charger has similar power to the Vic, I am sure that the 4.6L has more torque throughout the power range, which will make the Crown Vic perform better overall.
I don't know if you have driven a new CVPI, but there has been significant improvements to the car since 2000. The suspension and chassis were heavily revised in 2003, and the steering feel was vastly improved in 2005 (I always disliked the lack of feel on the older Vics). The car is much more refined than it used to be, and by 2005 they nailed it just right. For '06 there is a new instrument cluster that I don't care for (speedo is too small), and the also revised the steering (it's slightly higher effort than '05, but less road feel than '05).
Yeah, the 350hp Hemi would be nice. I admit that the 4.6L really lacks low end power, but in all reality how often do you really need that extra power? You said yourself that you patrol a quiet area. Take away the Charger's looks and the Hemi, and I don't think anyone would consider it over a Crown Vic. The Charger really appeals to the boy in us all, not our practical adult side.
The Impala isn't even in the ball park for a regular patrol car IMHO. Too small and wrong wheel drive. Plus, that 3.9L is from the awful 60 degree V6 family, infamous for intake problems.
Go with an '05 or newer Crown Vic and you won't be disappointed.