We have right to do anything in nature. We surrender some of these rights to the community so that we can have order. By order I mean that me prevent someone from exercising their right to punch you or me in the face. The rights the government borrows are those we have agreed to allow them to have through the constitution. Our constitution (minus the bill of rights) specifies the government rights, not the people's rights. Its not neccessary to believe in a creator, simply think of them as natural rights.
Simply put some people view rights as being given by government and others as rights given by the people to government. The founders of this country believed in the second option. I suggest reading a comparison of the writings of Rousseau and Locke. Rousseau believed in government devolving power while Locke was the philosopher behind the thinking of the founders. He coined the phrase "Life, liberty and the pursuit of property."