Yup, the Pi might work.
You could write a small secure OS for the Pi and use it as the link to the web. It would be mighty hard to compromise a Pi with a secure OS :-)
Broadcom made the SOC for the Pi, it’s not an open SOC and you have to sign a non-disclosure to get the nitty-gritty on it.
Perhaps a small and cheap secure computer built with the guts all inside an FPGA would be better still.
No, just build a dedicated encryption/decryption box, not connected to the net. Sneakernet the messages between the 2 for encryption / decryption. The net never touches the encryption box so keys can’t be discovered, short of physical access to the box.