I was misinformed. You learn something new everyday, thanks.
I was surprised to learn about it, too.
I can't remember where I first learned about it, but this is the official FAA publication that specifies it:
https://www.faa.gov/documentlibrary/media/order/7110.65s.pdf
A summary of the relevant section is here:
https://en.wikipedia.org/wiki/Aviation_call_signs
Reportedly, the Secret Service refers to the President's limo as "Cadillac One".
A bit of trivia: "Navy One" has only been used once, for a S-3B Viking used to ferry Bush to the USS Abraham Lincoln, where he declared an end to the major fighting in Iraq.
The aircraft has since been retired, and is now in a museum in Pensacola.