So they declared themselves seceded from the union even though they knew it was a provocative act. The soldiers that were stationed in the area - who were Americans BTW - instantly became "the enemy" (even if they were southerners). The townspeople and local militia immediately became hostile and belligerent. Anderson, not wishing to ratchet up tensions decided to retreat to Sumter until he had clear orders to withdraw but Beauregard insisted that he quit the fort. Ultimately the southerners fired on the fort until the union forces surrendered.
The southerners deliberately instigated the hostilities which inevitably led to the war. THAT was an act of war.
Well, you’re leaving out the part where the folks in Washington DC decided to not give the order to withdraw troops, but instead decided to try to resupply them, knowing full well that such an act was the equivalent of throwing a match at a powderkeg. The southerners held their fire until that act, which made it apparent that the US had no intentions of ever withdrawing their troops.