I had a problem with them once. They made a nest in the exterior door facing. I waited until it went in and plugged the hole.
It was very angrily buzzing but eventually died.
Fifty caliber is about right for the hole size at my house too
surprised they didn't chew their way out... i have rough-cut lumber siding varying from one-two inches thick on my house, i find piles of sawdust on the ground all summer long