It may make you mad, but it doesn't cause harm. Burn one in my presence, and it may.
If burning Old Glory is 'speech', than it definitely qualifies as 'fighting words'.
So you would infringe upon my First Amendment right because you disagree with what I say. Of course the only speech that is free is the kind that EV agrees with. That's real liberty...