My favorite places to buy peonies are Solaris Farms, Adelman Peony Gardens, Hidden Springs Flower Farm, Fina Gardens and Song Sparrow (probably their last year though). I have also heard very, very positive comments here about Lyndaker Farms, but haven't had the opportunity to order from them.
I like that I get nice hearty divisions, 90% of which will have at least one flower (sometimes more!) that first spring after planting. I know that if I contact them about anything at all, I will hear back from a knowledgeable person who will answer my question in a kind a respectful manner. With the above small businesses, I also know that if I am by chance sent the wrong peony cultivar, they will make it right (send you the correct replacement OR credit you if they are unable to due to lack of stock). But honestly, in all of the years I have ordered from these places, it's happened maybe twice.
But that's just me. I love freshly dug, nice sized roots shipped at the optimal planting time for my area. And knowing that I am supporting small peony farm operations with a passion for what they do.