I am not 100% sure in your case but those nasty little bugs called thrips
can cause damage to the blooms. I look at the bottom of the plant and if I pull a leaf off, sometimes there are little bugs and eggs. I start spraying with a natural spray containing pyrethrins, you have to repeat spraying about every 3 weeks. Keep all debris cleaned out of your flower beds because they can overwinter in leaves. I don't like to spray at all but it depends on your tolerance level. I don't have much tolerance for anything eating my daylilies.