Better analogy - using a high-powered microphone to listen in to what's going on in your house is an invasion. But if you're shouting so loud the neighbors can hear it, it's not an invasion for them to listen. You are broadcasting to them.
If you are arguing that you can receive my IP packets and inspect them, you may have a point.
But the issue isn't my wireless broadcasting to your computer, it's your IP packets coming into my house and then through my connection that I'm paying for, and then requesting information across that same connection I am paying for.
BTW, there's probably specific laws covering this in some places now, since it isn't so easy to directly tie it to existing laws.
If the cable company accidentally connects your wire up in their box, in some places you can actually be prosecuted if you DON'T call them and tell them.