Hi Sheridragonfly, I think you're talking about the Yellow Garden Spider (Argiope aurantia). They can get pretty big, can't they? But, I don't think they are interested in hummingbirds. Maybe the spiders see the feeder as a good nectar source, making it a good spot to catch insects in its web.
This year I also had an argiope that made its web right next to one of my hummingbird feeders, part of the web was actually attached to the hanger for the feeder! Like you, I kept knocking it down but the spider just kept rebuilding in the very same spot! It has never bothered the hummingbirds or hindered them from going to the feeder. I think the spider will only take whatever gets trapped in its' web.
Anyway, a couple of days ago my dh put the sprinkler on to water his fruit trees, which also hit the spider and it's web for about two hours. It hasn't been back since, I just looked its not there.
Here's the spider I'm talking about:
**Now if you see a Praying Mantis hanging out near your hummingbirds - BEWARE! The will definitely take a hummingbird for food!! 
