I've done some tests with fax and modems over VOIP. I'm using an Asterisk box with a TDM (analog) card as one endpoint and a PAP2 as another. All the VOIP stuff is on it's own network with only a switch between the PAP2 and the Asterisk box, which has it's own Ethernet interface for the VOIP stuff.
Latency-wise, this is as good as it gets. (I also set the PAP2 to use 10mS packets instead of the 30mS default, and of course using the G.711u codec).
I cannot get modems to connect at any speed with that configuration. (I tried forcing the speed to 9600bps or less but that still didn't allow a connect).
Fax works, but occasionally some pages get garbled (could be due to clock slip?)
Incidentally, I have Comcast Digital Voice. I connected a modem to the Arris device that Comcast installed, dialed an ISP, and got a 53k connect. I believe that is because the clock (which controls the PCM sampling) in the Arris device is synchronized with that of the rest of the telephone network, which is a requirement for V.90/V.92 to work.
There is no PCM clock synchronization with regular VOIP.