Don't give up, Kim. It may just be that the other birds haven't found your feeder yet. I have different birds at different times of year. When the grackles are a nuisance, I do go out and scare them away, and I've been known to shoot at them with my pellet gun (which won't hurt them but will scare them) if no other birds are there.
The bluejays and woodpeckers compete with the grackles here very well, as do the cardinals, titmice and dozens of sparrows.
I do have to grease the vertical part of my big shepherd's hook to keep the squirrels from climbing it. I've also seen rats on the bird feeders at night and the only way they could get there is by climbing the pole. So I keep a tube of SuperLube and an old toothbrush handy and grease up the pole every few weeks. It helps preserve the paint and the metal as a side benefit.