Don't worry; it's no big deal. I wouldn't worry too much about it and would simply replace the oil with a preferred viscosity. That's because:
(1) Fresh 5W-20 has HTHS viscosity around 2.6 cP, which is still significantly higher than a used 5W-30 (HTHS viscosity around 2.1 cP).
(2) As you said, you didn't warm up the engine much anyway for the oil to reach very high temperatures.
(3) Oil-film breakdown due to too little viscosity only happens during high load (high acceleration [especially from a full stop], very high speeds, or uphill driving) and/or engine lugging (driving at low speeds in high gear).
(4) While being heavily biased and making quite a few false statements, we could give Dr. Haas some credit that most gasoline-engine (= low-torque-at-low-RPM) cars could go without significant damage at most driving conditions with the smallest SAE-J300-allowed HTHS viscosity (2.6 cP).
(5) Even if you had oil-film breakdown, antiwear/extreme-pressure/friction-modifier additives would provide some degree of protection.
(6) Last but not least, if you want to have the oil-change job done right, do it yourself.