I like the red perennial salvias like 'Hot Lips' for the hummers in early spring. I take cuttings in the fall, and keep them going in a sunny window indoors so that if the plants die back, I have some new starts going that I can plant out in February when the migrating hummers come through here. The migrating ones stay longer in the fall, because there are more flowers for them usually. We don't see too many in the summertime here.
Regarding migration, you may have some migrating through your area, and that might explain why you had a couple, then a whole lot, then just a couple that stayed. Still, if you are on a migration route you will always see many in spring and fall if you maintain the nectar plants for them, but don't be disappointed if many move on again.