Answer: Gardenias are very fussy houseplants, as you are finding out. Providing ample humidity is one of the most important things you can do, next is to make sure it is receiving enough light. To start the budding process the plant needs a daytime temperature that is about ten degrees warmer than the night temperature, within a range of about 60 to 65 at night to 70 to 75 degrees during the day day. Otherwise, avoid big temperature changes, drafts, and other disruptions.
Simply moving the plant often causes bud drop, as may any imbalance in the growing conditions. Buds that turn black and drop, and bottom leaves that are yellowed are sure signs that gardenias aren't getting enough light.
Yellow leaves can be a natural occurance since the old leaves normally turn yellow and fall off, or could indicate inadequate light, or can signal insufficient water or even the use of hard water. If your water is hard, you might try watering with rain water or distilled water to try to maintain the acidity level of the original soil mix.
Although leaf drop can be caused by improper soil pH (gardenias require 5 to 5.5 pH - on the acid side), yellowing and leaf drop are also possible signs of various soil nutrient deficiencies such as nitrogen, zinc, and iron. Using a low dosage of a water soluble fertilizer formulated for acid loving plants can help with this, check the label also to make sure it includes micronutrients or "minors" to assure a broad based supply of minerals. Follow the label instructions for use.
Finally, I am not sure if the bugs are aphids or spider mites (or possibly something else), however an insectidal soap applied -- and reapplied -- according to the label instructions should be effective on both of those pests. Be sure to apply the soap to tops and bottoms of the leaves and all over the stems since it is a contact insecticide with no residual effect. Note too that a high humidity level helps to combat spider mites.
I hope this helps you trouble shoot your plant. Don't be discouraged, this is a tough one to grow at home especially without a greenhouse.
Q&A Library Searching Tips