There are red and brown paper wasps here that tend to be aggressive. A house mate was outside and felt something on her ear and brushed it with her hand thinking it was her hair. It was a red wasp and it stung her face when she brushed it away. She ran around the yard and it chased her, then she ran in the house and it followed! She was running from room to room with that thing in hot pursuit! Finally we knocked it out of the air with a fly swat and squashed it. I'd put it in the garbage and ran some errands, then came home and found it crawling out of the garbage- it was like the terminator wasp!
Thankfully I've never had them build nests in the garden, only on structures. Bad enough, but at least I haven't had to be concerned with the wasp killer damaging plants.