First question: I'm 99% sure it's a Geranium robertianum otherwise known as Herb Robert, class B weed. Really pretty but yep, it's a bit aggressive. Another common name for this plant is 'Stinky Bob'.
The plant is in our ATP database.
Herb Robert (Geranium robertianum)
Second question: Get rid of the Holly tree first or you will be pulling seedlings for the rest of you natural life.
Sorry, about that. Since you are so close to the water, hand pulling is the best. Put yourself into a Zen mode, get comfortable and just do it. You could use plastic to 'solarize' (if that's the correct word?) and kill the seedlings if you have time.
