Your use of the word library has no bearing. A place is not the same as a group of people, such as citizenry or militia. Now if the framers had used the word armory, your analogy would work.
So once again, in my example who can read books?
And your example offers no formal, organized and well regulated way that the citizenry will go about obtaining and reading books.
You essentially say, "if people read, they will be educated, and that's good". Try again.