It's an obscure bit of history, but the concept of using a "bank" structure within the postal system is really old.
The idea is that government has a monopoly on letters. Bills and payments for bills (checks) are, in essence, simply letters. By creating the "bank" the customers make deposits, agree to bill payments, and the postal bank credits the accounts. Small wage earners and workers can also open savings accounts.
The Postal Bank is an enormous institution in Japan (for example).
Given government's monopoly over money (in place since the abolition of private currency in the early 1800s) it's really never been considered "socialistic" for the government to own a bank ~ not even in the US. Abolish that money monopoly and government ownership of banks would be a moot issue.
LOL!
I believe that it was called “United States Postal Savings Bank” initiated in 1910.
Thanks for the reminder. ;-)