That's what I did! I cut the bottom out of several 3 gallon pots and planted them in a shady garden bed where I've had volunteer peony seedlings before. Put the seeds out mid-summer and several came up this spring. It'll be another 5 or 6 years before I know if I'll get anything pretty out of it, though.
