Just like every other right, including even our most basic freedoms, this right can be taken away as a punishment for crime, as long as it is by due process of law and the punishment is not considered cruel and unusual by the courts. This idea is very solidly grounded in English commonlaw and our own traditions.
There is a strong tradition in law that the punishment must fit the crime. That tradition has it's roots in the laws of ancient Israel (read Leviticus?). Gun prohibition cannot be logically defended for non-violent actions.
Most importantly, if the government can take one gun from one individual, then they can take all guns from all persons. It's only logical. That is why the founders prohibited all gun laws, not to protect the rights of criminals; violent people can be kept in prison for life in a functional republic.