Further to what Rubi said about plants which grow very well in gravel:
Below: part of the scree bed (all gravel in this case) at Merlin's Hollow, Aurora, On.
The purple one (some is pink too) is spring pea (Lathrys vernus) (May 9, 2015).
Certainly beyond my energy (and confidence) level, the owner, David Tomlinson has sunk, into the scree area, old paddling pools, filled them with muck, with water added each spring. This enabled him to grow wetland plants in the scree area. The primula here is Primula aurantiaca (July 4, 2009)