Once the man refused to follow the directions of the flight crew, it would have been a simple matter for the captain to announce that he considered the uncooperative passenger to be a safety risk to the crew, to the other passengers and possibly to the aircraft itself. And that although the flight was not cancelled, under his absolute authority as captain the aircraft would not leave the gate until the uncooperative passenger returned to the terminal. Simple solution to the conflict.
But that isn’t what happened, not by a long shot.