Normal olive does not survive here. I have a number of Autumn Olives, which are not related to the regular olives. I eat my Autumn Olives. The fruit is a little tart/bitter, unless dead ripe. It is very healthy. You are wise to only eat what you can accurately identify.
