Hi Kristina. Welcome to NGA!
I would also think "critters, not bugs". Whatever was eaten, vanished. If it were bugs or slugs, I would expect to have seen many leaves with holes, over a period of days.
If you have a roll of some fencing that would keep rabbits out, you might try putting that out first. If you see burrowing under it, it's small critters. If it's trampled flat, its deer or moose or elephants.
I'm guessing that deer would have mangled the flowers more and uprooted some. But something as small as a rabbit could go under the blooms and bite off the leaves from below.