You are saying that the Constitution doesn't prevent the President from doing that. That is not the logical requirement (as much as you would like it to be). There are all kinds of things it doesn't specifically list that are still unconstitutional. The constitution is a limiting document. It describes powers. All other powers are reserved to the people. It doesn't list the power to initiate an act of war as among those of the President. Bombing an aspirin factory is not on the list. If the Constitution doesn't say that he can do it, HE CAN'T, not the converse.
In the case of the action against the Barbary Pirates, when US ships were fired upon, they had the right of defense. When Jefferson issued an order to send an expeditionary force to Tripoli, he should have had a declaration of war. That he got away with it, has been used to expand executive powers ever since, such as we saw with Beelzebubba.
Read the book for what it says instead of looking for loopholes.