There are very few restrictions in the Constitution concerning crime and punishment. If the people cannot decide via their legislators what should be illegal and how it should be punished, within the confines of those aforementioned Constitutional restrictions, then who exactly should be empowered to do so? Laws don't write themselves.
The absurdity of the argument you are making is that we can, via due process, Constitutionally deprive a man of his livelihood, all of his belongings, his freedom to come and go, the company of his family, even his life, but somehow his right to be armed is the sole thing that he cannot be deprived of. That is as illogical as it gets.
The loss of freedom and self-autonomy is what punishment is. There are many arguments to be made for changing current laws and associated punishments, but that is an entirely different matter than stating that the people cannot be allowed to deprive felons of freedom.
Virtually the entire Bill of Rights concerns itself with various acts which cannot be criminalized by the government or with protections for those who are suspected or charged with a crime.
Here are a few excerpts from the main body of the Constitution:
The Senate shall have the sole Power to try all Impeachments. ... : but the Party convicted shall nevertheless be liable and subject to Indictment, Trial, Judgment and Punishment, according to Law.
... To provide for the Punishment of counterfeiting the Securities and current Coin of the United States;
... To define and punish Piracies and Felonies committed on the high Seas, and Offences against the Law of Nations;
... The Privilege of the Writ of Habeas Corpus shall not be suspended, unless when in Cases of Rebellion or Invasion the public Safety may require it.
No Bill of Attainder or ex post facto Law shall be passed.
... No State shall ... ; pass any Bill of Attainder, ex post facto Law,
... The President ... shall have Power to grant Reprieves and Pardons for Offences against the United States, except in Cases of Impeachment.
... The President, Vice President and all civil Officers of the United States, shall be removed from Office on Impeachment for, and Conviction of, Treason, Bribery, or other high Crimes and Misdemeanors.
... In all Cases affecting Ambassadors, other public Ministers and Consuls, and those in which a State shall be Party, the supreme Court shall have original Jurisdiction. In all the other Cases before mentioned, the supreme Court shall have appellate Jurisdiction, both as to Law and Fact, with such Exceptions, and under such Regulations as the Congress shall make.
The Trial of all Crimes, except in Cases of Impeachment, shall be by Jury; and such Trial shall be held in the State where the said Crimes shall have been committed; but when not committed within any State, the Trial shall be at such Place or Places as the Congress may by Law have directed.
... No Person shall be convicted of Treason unless on the Testimony of two Witnesses to the same overt Act, or on Confession in open Court.
The Congress shall have Power to declare the Punishment of Treason, but no Attainder of Treason shall work Corruption of Blood, or Forfeiture except during the Life of the Person attainted.
... A Person charged in any State with Treason, Felony, or other Crime, who shall flee from Justice, and be found in another State, shall on Demand of the executive Authority of the State from which he fled, be delivered up, to be removed to the State having Jurisdiction of the Crime.