The Bill of Rights was written to control the federal govenment not state governments. Later amendment addressed different concerns.
Anti-Federalist feared the federal government would be tyrannical if it had a standing army. Hence, state militias were a major component of the federal military structure. They were a resource and a check so the Second amendment was to reassure that the “Congress” explicitly identified shall pass no laws etc.
It isn’t just the historical record which confirms this but the very structure of the amendment itself.
It should also be noted that, throughout Federalist 42, Madison repeatedly used the word “privileges” interchangeably with the word “rights.”
http://www.constitution.org/fed/federa42.htm
The Powers Conferred by the Constitution Further Considered