I have both deer and rabbits in my garden.
The rabbits leave the daylilies alone.
The deer will eat both the flowers and the buds (and foliage, if there is neither flower nor bud), but they often leave a partially bitten bud behind, so you know that you've been raided.
They also aren't shy about eating most (and sometimes all) of the buds on a scape.
Confirm that it is very easy to knock off daylily buds....