I'm guessing it's very dependent on local.
Why you would I'm not sure. I've never been in an SxS but I'm guessing it's not quite the couch on wheels that my Corolla is? let alone anything larger? For bouncing around on a dirt road, the SxS is probably wiser. But is it better on pavement? I'm thinking, between having heat and a/c, not having to listen to wind noise, having an actual safety cage around me ('cuz it's always the other drivers fault not mine)... and then actual running cost, that if most of the drive is on pavement, then a "real" car is always better.
I tend to think of off-road tires having short lives when forced to run on asphalt--which is probably not true, low traction dirt roads probably wear tires faster, due to that slippage--but at the very least, probably not rated to get the same milage as regular auto tires. Then actual fuel burn. Then what if my family members insist that I take leftovers home?
It's a free world, go for it, have fun. I recall a coworker who once road his riding lawn tractor to work (maybe it was his parents, we were after all in high school). All backroads so precisely 100% zero chance of danger to anyone at any time (it was maybe 2 miles in a podunk small town and on roads that people actually drove slow on). But funny all the same.