It’s good to have a phone with you as you’re out and about for emergency reasons.
Keep it in a Faraday shield (wrapped in foil, inside a metal box, or a commercially available shield pouch), and take it out only if you need to use it.
You won’t get calls or texts, but the OS won’t get GPS information, so no tracking.
The only thing is, they’ll know the phone has disappeared. If you leave it at home, then they are still tracking it but don’t know you’re doing something you don’t want them to know about.
I just got a couple Faraday pouches for our phones. Mine is in airplane mode much of the time, as well.