Even if google doesnt know where you are, your service carrier does. I was a network analyst in the earlier days of cellular and was tracking people even back in the 80's-90's on the old analog AMPS phone systems with a pretty decent level of accuracy. I had printouts of what channels were active in all the cell sites, what the signal levels were on most major roads, which cellsite antenna sectors served which areas, etc and could track a phone within a few city blocks just by looking handoff patterns and signal strengths. Helped the police find more than a few stolen cars that way.
With todays phones, even with the most basic, no frills model you can find they could probably tell you what room of your house you are in.
And yes, bored technicians working on the towers late at night do listen to your calls. All the time...99% of them were boring, but 1% were quite entertaining.