No, don't feed it. Fertilizer is a reward for healthy plants.
Although Olive trees are from arid regions, they do require regular water but also fast draining soil that doesn't hold moisture against the roots. It could be that your tree is slowly dying of root rot. That is a problem that is not fixable and takes 15 or 20 years for the affected tree to die.