I always remove the leaves in the fall....both as a 'sanitation" measure, and to prevent them from "catching" every tree leaf that blows through the yard ! If you remove them (I cut them back to about 4"-6") you're betting rid of some insect eggs, and diseases, and every little bit helps! They will have plenty of time to grow "from *nothing* to normal height by bloom time ......the iris leaves of most cultivars, while they may still look a little green....aren't really "evergreen" in our climate. The problem with the borers is the rascals can fly in from the neighbors house, or a bush in your yard, or any one of a hundred places near your beds....they lay their eggs on many things...not just iris. So , cutting back the fans *helps*, but it's not a 100% cure all !
