I believe the “silent majority” was meant to be people who voted for the conservative candidate—but didn’t march in the streets screaming about it. They did/do indeed exist.
After Nixon was re-elected in 1972, leftist kook Pauline Karl (a “film critic”) lamented his landslide victory. She claimed nobody she knew voted for Nixon. Her attitude kind of illustrated how the elites view middle class Americans.