The founders were closer to my view than yours; they believed there should be no standing army, and that "national defense" meant that everyone was armed and ready to defend his life, liberty and property.George Washington, a founder with some military experience at the top of the food chain, knew this was balderdash which is why he had to conscript an army.
Washington did not use conscription to fill the ranks of the Continental Army. One of Washington's dilemmas was a chronic shortage of men available for military service. There was conscription on the state level, as all able bodied free men within certain age ranges were subject to militia duty. However, during the War for Independence, the states did not generally enforce the requirement. The high level of Tory sympathies, especially in New York, New Jersey, Georgia, and the Carolinas, would have caused more trouble and unrest than the states could afford in wartime had they attempted to enforce militia duty.
The experience of the Continental Army may have motivated both sides in the Civil War to establish conscription, as well as during both World Wars and the Cold War era (including Korea and Vietnam). The Northern Civil War and Vietnam era drafts were unpopular and led to riots and demonstrations. Support for a war has to be held by a large majority of the population for a draft to be effective.