@Kabby, I have had deer problems ever since we moved here 14 or 15 years ago (lost count). But, I do have a large portion of the back yard fenced in (low 4 foot chain link fence) and the deer have not jumped the fence for many years now. The first few years here it was common to find deer and turkeys in the fenced in area, but that has not been the case in years (I have not planted rye grass in the back yard for a while till this year, my gutters have caused a wash in a few areas and I have been trying to correct that).The largest part of my daylily collection is outside the fence however, but the deer have caused very little problem with the daylilies. The true lilies and other plants are a different story, the deer love to eat them. All my hostas are inside the fence so they are eaten by the slugs and not the deer. My ferns are both inside and outside the fence, no problem with deer with either of them. Also I have had no problem with deer eating my Caladiums so far. I planted some as a test a few years ago and so far the Caladiums have not been bothered by the deer.
The deer seem to bed down for the night near the branch in the back, in my neighbor's back yard. That is actually only a few feet from my main bog bed filled with daylilies, and so far the deer have been a very minor problem. So by planting rye grass again this year I am hoping it does not cause a huge problem with attracting deer and tempting them to cultivate a taste for daylilies.