Sounds like you're in the same boat as I am. I've taken out 2 five gallon buckets of coal so far and a 7 gallon bucket of glass, metal, broken dishes, canning jar lids etc.
I have to admit most everything I planted did really good except for the giant sunflowers. But the problem with those was not enough sun. I added blood meal, pot ash and manure to help the soil. I'm going to build a soil sifter that fits over a wheel barrow before I plant anything this year.
I moved in April with mid May my planting date so it didn't give me much time to find a place to plant the plants I brought with me. The few gardens around the house were in bad shape. I tried planting impatiens, nicotonia, & black eyed susan vines but it was a struggle for them. The impatiens got smaller and smaller. Every thing was stunted. Except for the wild violets!
Here's a pic of the garden. The arch is a cattle panel. Lower left is Autumn Joy sedum. Arch left (facing West) is Butternut Squash which are hanging in a nylon stocking sling so they might be hard to see and right side (East) is green beans. Under the arch is acorn squash and behind the arch is my 8' Sun Gold Tomato, Delight Tomato, peppers, Cosmos, and if you look hard on the right you can just barely see the Purple People Eater Datura in bloom. I laid down a soaker hose then covered it all with straw. Now its all covered with snow.