I rely on landscape fabric
under every raised bed to prevent weeds from coming up into the bed from below. The key word is "fabric"...not plastic. It is the plastic landscape fabric that will cause soil to sour. Using the fabric type I have never had any problems - period. This is the one I use.
http://www.samsclub.com/sams/p...
Have you checked to see if there are pollinators working on your plants? Sometimes we have neighbors who use poison to kill insects and that reduced the population of pollinators available to insure a successful harvest.
What size is this garden, please? and how deep is the soil? Did you start with native soil or have you amended it with bagged soil or compost? Are you watering by the overhead method? or using base or drip watering? Sometimes overenthusiastic overhead watering can wash away the pollen before the plants have been worked by the pollinators.
Lastly, did you take any photos of the garden? Pictures would be helpful.