It sure does, Becky. It will always be a 'work in progress' because our heat and heavy rain breaks down and washes our good organic stuff out of the soil pretty quick. So raised beds and "pick your battles" is my order of the day.
I've gone almost completely to Earth Boxes for my veggies, just so I can move them around to find enough sun (especially at this time of year). The shadow of the house, and shade from the big oaks and my mango tree leave an ever-smaller window of sunny space in my back yard.
I started out 8yr. ago in this house with 4 large oak trees all around the yard. I always did have some worms, because the oaks had been dropping leaves (and happily the previous owners had very little lawn and did not rake them all). But as you say, the more organic material I've added, the more worms I've seen.