Humid air doesn’t make the AC system work twice as hard or harder at all. The efficiency may drop if the humid air causes the evaporator coil to partially freeze over, but if the system is functioning properly, that shouldn’t happen.
Moisture removal by the AC system is a by-product of its function, not its main purpose. AC system is simply removing BTUs, so it’s based on temperature difference. Both dry and humid air at the same temperature have the same amount of BTUs as far as heat energy, thus require the same energy to remove the heat.
Problem is with humans and how we perceive dry vs humid air. Humid air feels hotter and less comfortable to us then dry air at the same temperature. Therefore we feel like it takes longer to cool the cabin down, simply because we don’t feel the same comfort level. So we either lower the temperature to get that same comfort level, or wait longer for the humidity to get lower.
And since lowering moisture from the air is simply a by-product of AC function, it takes much longer then lowering the temperature.