I think you're under watering.. and I almost never say that! The plant looks like it wilted in the sun.. how's your weather been?
There's a good chance it's not deep enough, which is why it's drying out so fast, so Luke's advice to mulch it well is very smart.
Don't dig it up now. You'll do more damage than good when the plant is in full growth mode. I would never dig up a peony unless it's dormant in fall or just starting to come out of the ground in early spring. I would follow Luke's advice, on the deep side. If you just planted it this year, it's got a very immature root system, so try to water more regularly.
Do you remember how deep you did plant it? If it's deep enough it won't freeze, I wouldn't dig it up at all. If it's only an inch or two below the surface you may want to dig it up in The fall to plant it deeper. If the tuber freezes solid, it will die over the winter, which is why you want to be ~6" deep