Some people use a specific daylily that the midge seems to particularly like as a "trap" to attract them away from other daylilies, and then they destroy all its buds once they are infested. When you say you "toss" them, I hope it is in a way that kills the larvae immediately or at least prevents them from staying in your garden?