Raspberry Pi and an Altoids tin.....
Yup, the Pi might work.
You could write a small secure OS for the Pi and use it as the link to the web. It would be mighty hard to compromise a Pi with a secure OS :-)
Broadcom made the SOC for the Pi, it’s not an open SOC and you have to sign a non-disclosure to get the nitty-gritty on it.
Perhaps a small and cheap secure computer built with the guts all inside an FPGA would be better still.