Hi Lisa,
I've read that wasps and bees are attracted to the color yellow so if you have a feeder with yellow feeder ports or any yellow color on the feeder you might want to change to a plain red colored feeder. They do make feeders with bee/wasp guards but I heard that those types tend to drip a lot which will attract ants and bees or wasps.
I have the Hummzinger Feeder which supposedly doesn't attract wasps but last year for some reason there were wasps on occasion at one of the feeders. I moved that feeder to another location in the yard and didn't see wasps again.
Hopefully others who feed the hummers will pop in with some advice!