Ahh, thanks Lindsey! Capparis spinosa is indeed the real Caper plant.
Alice, you have a well taken point there, I really don't want them to teach their next generation to dine in my backyard!! I will look into repellents and perhaps some deer netting strategically placed.