Thanks for all the advise guys. I cut off the infected leaves and moved the plant to a sunnier spot. The fungus hasn't come back but it has a little sunburn now. I decided to just leave it alone now and see what happens.
"I don't care about spots on my apples- leave me the birds and the bees"