Free Speech. Electors voting their conscience is not a Constitutionally protected freedom.
I think I could argue that the Electoral College rises above that, given its importance in getting the Constitution even ratified in the first place.