I don't know the technical details, but I can't imagine how they would know where you were if the phone was turned off. When you turn it on, it starts to exchange signals with the carrier and tells you it is ready. Presumably they can at least tell which antenna you are closest to, and possibly they could tell more or less how far away from it you were if they installed equipment to precisely measure the time delays in the signal interchange.
I also leave mine off except when I am making a call, which is pretty seldom.