Are they Carpenter Bees?
http://ohioline.osu.edu/hyg-fact/2000/2074.html
Look for 1/2 inch to 3/4 inch round holes in the exterior woodwork on your house. The males who ‘patrol’ the area cannot sting, but the females who stay in the nest can sting when aggravated.
Checkout the pesticide section of your hardware store. Look for termite and carpenter ant killer (contains bifenthrin). It also prevents/kills carpenter bees.