I can just imagine how angry He came across. It wasn't like He was just asking everybody to leave, please.
John 2:13-16 13 The Passover of the Jews was at hand, and Jesus went up to Jerusalem. 14 In the temple he found those who were selling oxen and sheep and pigeons, and the money-changers sitting there. 15And making a whip of cords, he drove them all out of the temple, with the sheep and oxen. And he poured out the coins of the money-changers and overturned their tables. 16And he told those who sold the pigeons, "Take these things away; do not make my Father’s house a house of trade."
Luke 19:45-48 45 And he entered the temple and began to drive out those who sold, 46saying to them,"It is written, 'My house shall be a house of prayer,' but you have made it a den of robbers."
That surely disrupts the passivism image some would have us believe.