My three things would be:
1. Rodents (squirrels and rabbits especially)
2. Cynanchum louisea, this viny weed is everywhere in my garden and neighborhood and just keeps resprouting indefinitely from root fragments.
3. Ornithogalum umbellatum aka stars of Bethlehem. Yes, the flowers are very pretty but only one in every 10000 plants actually blooms. In the meantime, its spring foliage is so dense that it smothers and chokes out everything in sight.
I have a few other annoyances like garlic mustard, sorrel and mint that are quite easy to manage in the landscape compared to the horrors listed above.
