I get the impression you want the stone to show, so would prefer something short.
A south exposure like that is great for annuals, but they can be more work. I would think something with some height, like cosmos or zinnia.
I highly recommend the benary zinnias both for big double flowers and for tall plants that will look good in that space. They come in many colors. Other varieties of zinnia can be kinda skimpy. Cosmos differ primarily on flower type - they're mostly all tall.
Check back and let me know if this isn't the sort thing you had in mind. You left things pretty wide open, so a lot was guessing what you like, rather than what will grow.
I've had problems myself with annuals purchased from garden centers not growing as tall as I hope. I don't know if they are treated with something or just smaller varieties, but starting from seed will get you much better plants.
You could grab some dahlia or gladiolus tubers to plant right now for pretty cheap that would be quicker to establish. Your choice if you want to dig them in the fall or just re-plant.
Actually, south facing next to a stone wall in an area with a lot of snow might be a micro-zone that will be warm enough for them to over winter.
For something perennial: Daylilies are pretty fool proof and will eventually fill in and smother weeds. You can probably find someone giving away the orange weedy ones for free.
Other, somewhat random ideas for large, vigorous perennials that like sun:
comes in lots of hybrid colors, but I don't have experience with them to know how sturdy they are
(lots of cultivars with mostly blue/white/yellow color options)
(blooms in fall when other things are done)
I'm not recommending any shrubs since you said cheap, and just pulled a bunch out.
I think my choice would be to combine some perennial starts with some summer bulbs to fill the space the first year. Maybe let the kids sow some annual seeds and just take your chances on whether they sprout or not rather than worrying about coddling them.