Leslie Iris borers usually do their damage to the rhizome of the plant and late in the season. I would guess that if you have a worm in the flower it is something else. You could check the plant leaves to see if there is any indication of iris borer activity. They tunnel down the leaves to reach the rhizome. They can be squished inside the leaves to kill them. Yucky, but effective.
I treat the iris beds here with Bayer Grub control on the early spring to help stop the pests.
Good luck!
