I've found that if I use larger containers and plants that are hardy at least to 1 zone colder, perennials do fine that way! I tend to overcrowd containers with "starter" size plants and then thin out, leaving some in the container and planting others out in the garden.
I also overwinter a lot of up-potted perennials (starter plants that grew larger over the summer and are now in trade gallon or larger containers). Again, that's not a good idea with anything marginally hardy in your zone, but otherwise they do fine, especially as I group my containers closely together on top of a gravel pad or back patio, so I get that "heat sink" effect to moderate temp changes.
The one exception to this seems to be grasses... at least for me, grasses die over the winter in pots or containers, even those hardy to zone 3 (in my zone 6b).