There are NO pests here!!!!
Personally, I wouldn't "take it back" at all…If it were mine, I would just "limb it up"…In other words, prune the lower branches coming off the main trunk. Leaving the upper branches to form a small tree. Then let it go from there and see what develops.