I think Masison was worried about the young Republics survival on a financial and organizational basis. I have read about the post revolutionary debt turmoil leading to Shay’s rebellion and similar incidents. The states were only looking after their own interests and letting the larger republic flounder.
Many felt we would soon fall apart and be devoured piece meal.
I often wonder what might have been different if his friend Thomas Jefferson had been in the country for the framing--if he'd been there. Jefferson surely demonstrated a greater mistrust of government than Madison. TJ trusted Madison, and seems to have somehow been a bit of a centrist on the issue of the Constitution, but I wonder if he'd have had a different view if he'd been there. I even wonder if it was an accident or not that he was out of the country at the time.