Sure, I agree with you there. But this isn't about your rights, this is about a law (the Constitution) and what it applies to and how it is interpreted. IE - we have a government that rules by written laws and not what one person happens to think at any given moment.
Sure, and "right of the people" and "shall not be infringed" is supposed to be part of the highest law, and is supposed to, at a minimum, bind Congress under the rule of that written law.
If Congress or the Executive are free to ignore that highest law, the one I personally swore to support and defend against all enemies, foreign and domestic, then we have no rule of law, just 2 wolves and a sheep deciding what's for dinner.