The patriots had to contend with both the British and a large percentage of loyalists. Then as today, many Americans were not actively supportive of freedom and liberty.
It sure looks that way