Answer: The USDA zone numbers are based only on average annual minimum temperatures, i.e., how cold tolerant or "hardy" will any plant be. However, many more factors influence a plant's ability to thrive, including soil type, elevation, rainfall, humidity, etc. For example, zone 6 occurs coast to coast, in such varied states as Washington, Colorado, Tennesee, and New Jersey. Obviously the same plants are not all going to do well in all of these areas, so some trial and error is involved to find what works best in your particular microclimate. However, many perennials will grow in a range of zones. Here's a list of possibilities for your zone 3 taken from Burpee's American Garden Series Perennials by Suzanne Bales.
Achillea (yarrow) 3-8
Q&A Library Searching Tips