Small fruit sounds like the fruit should have been thinned when it was about the size of olives.
Brace up branches.
Prune tree so it has srurdy scaffold.
Farmers and city folks hear in valley put supports on branches every summer. Some put wire around whole tree. Prune no more than 25%
😎😎😎