I have a '97 Camry I4 and had a variable idle speed problem, but not as severe as yours. The RPM would "hunt" +/- several hundred RPM around the normal 750 RPM when stopped in Drive. Put it in Park or Neutral and it would smooth out immediately. It would not do it in Reverse. It did not happen every time I stopped, but it did it often for several months in the Summer. I tried different brands and octanes of gasoline with no improvement. After a few times that I thought that it might stall, I decided that this was beginning to be a safety problem! I decided that the transmission was abnormally loading the engine at idle, so I had the transmission fluid changed, although the transmission oil had been changed only 20K miles previously. The transmission oil-pan gasket was leaking just a little and torqing the bolts no longer helped, so I decided to have the pan dropped by the dealer and look for problems (the extended warranty was due to expire in 2 months). Anyway, the pan was pretty clean, the oil screen was clean, and we just put it back together and filled it up with fresh ATF. This instantly fixed the problem, and it has not returned. Perhaps this will help you, too. The dealer was puzzled that this would fix it, and I certainly don't know what the transmission problem was.
Otherwise, in addition to seotaji's excellent suggestion, I would make sure that your air filter is clean, the PCV valve is OK, the throttle body is clean, that you have no air leaks, that all of the electrical connectors on the engine are intact, that the few vacuum hoses are in place and not cracked, that the EGR system is working right, that the exhaust system is not restricted, etc. You might also try a fuel system cleaner - your injectors might be trying to plug-up or you might have some water in the gas? With a fuel system cleaner in it, run it hard with O/D off on the highway and do some hard accelerations on the on-ramps.
Hope you find it - let us know your results!
