Your drainage sounds like a good beginning, but does water flood out the bottom of the buckets? The landscape fabric might be getting clogged up, and slowing things down. Watch for that. I buy bridal veil netting at the fabric store for use in the bottom of my pots. It's cheap, lasts well and the holes are big enough not to get clogged.
After seeing your pictures, I'm going to stick with my guess from above - I think you have an inconsistent nutrient problem with those plants. It may be that you got more manure in one pot than another, it was unevenly mixed, that the plants didn't reach the manure until they put down some deeper roots, or that the manure released inconsistently. Your soil mix also may be lacking other components needed by the plant.
I'd advise you to use some good quality controlled-release balanced fertilizer (N P K numbers about the same plus micronutrients) that is rated for use on edibles. Apply it sparingly at first, as these are young-ish plants and let's see what that does. I use Osmocote, and have had great results with it. For a quick green-up you could give them some soluble fert first.
Btw, ants don't harm your plants if they are just in the soil. If they are on the plant, they might be 'farming' something like aphids - they eat the honeydew excreted by the aphids. But in general, ants don't harm plants. I don't see any sign of aphids, and the yellowish leaves aren't really mottled like they would be for spider mites so . . . again look at nutrient deficiency.
My three tried and true pest and disease controls that I use on my edibles are the soapy water spray I mentioned in my first post, a baking soda and water spray for fungal prevention (a biggie here in FL) and Bt which is a caterpillar remedy. (I haven't had any caterpillar problems on peppers though). Anything else you're tempted to try, please read all the literature on it first, because a lot of things are not rated for use on plants with edible crops - for good reason!
