It happens when I crowd them in - no air circulation. I did remove all stems that showed any signs of mildew. We'll see what happens with new growth.
If yours is a plant that is supposed to rise to 3.5' or 4' (or more) you can always remove the lower leaves. It will not harm your plant.