Whitebeard was a vet or something. He was a big dude and he warned the guy off, one good left hook and some rights later and someone needs to call the amba lance.
Probably Boxed in the Service as well, he handled himself like he knew what he was doing. Maybe even golden gloves when he was a young lad...