Tree peonies are grafted - if your peony came in a pot, the graft was buried a few inches underground so you never saw it. What you are seeing are shoots from the herbaceous root stock peony. Remove them but, make sure its a herbaceous peony stem and not a new shoot from the tree peony roots. Given time, the tree part will develop its own root system but the herbaceous roots will always be there.