I have always thought orcas are completely intelligent enough to know that messing around with humans can only cause trouble for them, so...they generally don't. Otherwise, we would be toast.
I could be wrong, but orca attacks outside of a tank are pretty rare indeed.
In this video I saw a few years ago Penguin Escapes Pod of Killer Whales, the penguin seems to stand on the edge of the boat taunting the whales, who just swim away, seemingly knowing it isn't worth the trouble messing with inflatable boats of humans, even though they could easily do so.