Deer will eat fig leaves, as well as goats and pigs. None of them is exactly known to be a gourmet.
But honestly I have never seen any caterpillar nor grasshoppers nibbling at them: fig tree pests are usually either scale insects or feed on the figs themselves.
