IMO the only hysteria was in the reporting. I agree the plane should have left with just the two if the piolet was ok with that but anyone that gets off a plane or any public transportation because they consider it unsafe should get a refund or exchange ticket.
One huge problem with not charging the deplaning passenger(s). Luggage belonging to individual passengers, must accompany said passengers. A deplaning passenger's luggage would have to be located and removed, which would delay the plane and create a huge inconvenience for all others.