Sue, it is the soil !
When I bought my place, the whole property was covered with snow. I didn't know that the garden area where I planned to put my rose garden was dead soil. It was covered with weed barrier and decorative rock. Of course, the owners were in their 90s and weren't gardening any more. When I removed the decorative rock, weeds would not even grow in that area. The parts of the yard that I have not cultivated still won't grow weeds !
I got the roses planted and for years it looked like all I grew were roses and weeds
This year, I was finally able to start planting other plants. It's going to take a few more seasons before I have what I call a garden. It's a four hour round trip to a decent nursery, so even getting the plants is a challenge.
Your garden gives me hope. Mine will never look as lush, but it will be more than roses and weeds.
Smiles,
Lyn