Everything with its location and your care sounds right. The leaf spots are a fungal disease and so some fungicide spraying may be useful, although the season is about over with fall at hand. Your County Extension Office may be able to suggest an appropriate product. If you cannot get such a suggestion I would try something with chlorothalonil in it, such as the product Daconil.
Next spring start fertilizing lightly every 4-6 weeks and continue through mid summer. Keep the soil moist but not soggy wet as they don't tolerate such conditions.
Thanks again and feel free to stop in any time we can help!
Q&A Library Searching Tips