Thanks for all the information. I haven't seen one of the wasps at all, just the hole and have not seen any new holes since I put water down it. I have assured my neighbors that they are harmless. It must have been another type of ground bee that swarmed and stung me a few years ago. They were much smaller than what these sound like.
Allthingsplant is great for getting gardening help. I had only posted the photo 20 minutes before someone ID'ed it for me even though I was in the wrong forum.
