That alone is enough to hold the government to that meaning, regardless of what future actions indicated or arguments were put forth.
I, personally, think that the Anti-Federalists were correct in most of their arguments and that the Constitution gives the government too much power. But I was not present then and the only thing we have is the writings of the folks how were there. We have to use those writings to put meaning to any kind of ambiguousness and hold them to what they promised the Anti-Federalists and the people.
"On every question of construction carry ourselves back to the time when the Constitution was adopted, recollect the spirit manifested in the debates and instead of trying what meaning may be squeezed out of the text or invented against it, conform to the probable one in which it was passed." --Thomas Jefferson to William Johnson, 1823. ME 15:449
Had the constitution not been adopted the nation would have fallen apart and split into two regions hostile to each other and aligned with different European powers.
Federalist administrations made the government and nation strong enough to withstand the democrats which followed though it was touch and go.