Is the space you intend to use covered? I lost plants to the monsoon rains last summer. 'Good dirt' can only do so much in the presence of constant rain, missing sun. If I tried this, dogs would run through it and ruin it immediately, stray cats would, well, you know. I can't probably imagine everything my son would think of doing, but... yeah. Aside from logistic (and 'pest') issues, if you like this look, you should absolutely do it! I like to get more close'n'personal with plants than it looks like this would allow. Can you walk around in the gray gravel? It looks like it would shift, and being as (NOT!) graceful as I am, I would fall onto those prickly plants!
