Anti-polygamy laws were religious neutral - no one could have more than one spouse, not just the Mormons.
The Riot Act was a British law, so it did not violate the Constitution.
Of course rights are not unlimited. They can be limited when they interfere with someone else's rights. But my right to keep and bear arms does not interfere with anyone else's lawful exercise of their rights. If I do interfere with their rights, there are other laws to deal with me.
Can we restrict rights based on a record as a felon or a legal finding of insanity? Yes, but that has to be decided by the representatives of the citizens. Do we forever forbid someone who committed a crime from ever owning a firearm? Does someone who had psychiatric treatment permanently lose their right? That is why we have a representative form of government - but they have to have a good reason for restricting anyone's rights. That is why we have elections and courts, to check the representatives (when working correctly).
The Riot Act has been adopted by many states. Libel and slander definitely limit speech and properly so. Further, there is criminal libel though its enforcement may be way down. As far as your statement that the first amendment states congress shall not limit freedom of speech, apparently you have not heard of “the Incorporation Doctrine.”
+1. Nice!