Sorry for the delayed reply. I've been away for a while and am just now getting back and catching up!
Your yellow, dusty looking stuff under the leaves may be rust sprays that would explain the loss of leaves. I suggest you take some leaves showing the yellow stuff to your County Extension Office for them to look over and diagnose. If it is rust you may be needed to spray periodically with a product containing chlorothalonil, but I would not do so until you get a positive, accurate identification.
Your Extension Office's contact info is:
St. Lucie County Extension, 8400 Picos Road Suite 101, Fort Pierce, FL 34945-3045
Phone: (772) 462-1660
Q&A Library Searching Tips