This sounds like a rock mulch. Are you able to add a pic of the Gardenia in its' setting? It's normal for the older leaves to turn yellow and fall off as the plant ages. Sometimes a bunch at once. Are the tips still lush and green? Knowing your general location would also help folks know what a Gardenia would normally be doing in your climate at a given time. I'm in AL near FL border and Gardenia is a dependable full-sun plant, with gorgeous glossy leaves and, of course, periodic blooms that smell awesome, as long as the soil does not get too dry.
There are multiple types of Gardenia. If you know any further info about what kind you have, the info you get could be more specific to that type.