Tom, usually if there are not bite or bug-evidence on the leaves it tends to lead to either over-fertilization or over-watering. I think I'd check to make sure your soil is fairly well-draining, especially since your nights may still be a bit cool up your way. (Peppers love heat, hate cold feet!) :>)
If you've been feeding them compost or the like that wasn't fully composted that, too, will make leaves fall. Same if you've been feeding a stronger-than-needed doze of liquid fert (e.g., Miracle-Gro).
Does any of this sound plausible?
Shoe