The device you desire does not exist. Wanting to “download radio apps” means you’re getting some kind of WiFi-enabled device which will come with something like Alexa.
Your best bet is an Android (google) phone that you can then strip down to the basics.
ROFL!! Even a stripped to the basics Google enable phone spies on you as much as Alexa does.
I'd suspect that the original poster wouldn't know how to strip a Droid based phone down to the basics anyway after they recoiled in horror at just having Alexa.
For the record, I have three Amazon Echo's in my house all tied together, a Droid SmartPhone and several Apple iOS enabled devices. It's not terribly hard to secure all three however it does take some technical prowess and patience to do it.
BTW: Are you using a Windows PC by any chance? Yeah, not the most secure platform either.
Every so often at the flea market I run across one of the old 1” x 3” FM radio receivers with an armband and headphone jack.
Vintage junk.