I looked through my caterpillar books and there are so many similarly coloured ones (and caterpillars can change colour at different stages too) that I wouldn't take a guess at what it is (it does look more like a caterpillar than a sawfly larva although I can't clearly see the prolegs - there don't look to be too many for a caterpillar).
If the holes are all over the peonies and there is only one caterpillar then it may be an accidental visitor and not the culprit. Did you see it actually eating the leaf it is on? Is there a tree overhead that it could have fallen from? You could try posting on the peony forum, or the insect and bug ID forum.
