I would wipe the contacts list and all message/call log entries if I thought I was at risk. Heck, I would scrub the entire phone and reinstall daily, leaving nothing on the phone other than the ability to contract someone or be contacted if I needed to. No links to accounts, whether email, financial, medical, or otherwise.
Carry a burner phone. Want my phone FIB, here ya go!