“IN 1963, when the CRA was introduced and being debated in Congress, America was a country still steeped in segregation. It was still legal to refuse to serve Blacks, Latinos, Native Americans, and Asians at restaurants and in hotels … to deny people transit on private transportation … to deny women access to certain jobs.”
In other words...property rights and freedom of speech were still protected rights in 1963.
And ... the CRA only applied to the Southern states.