@ stush - yes most moss, not all , likes acid soil. I would guess that most cactus do as well but i have never had to adjust ph either way in cactus or succulent plantings so i don't really know.
And can you cut up newspaper and wet it and try to ' smother ' the moss by making a creative weed cloth around the seedlings? Just guessing
I assume you can not manually remove it. Water less is probably your best bet and keep a good airflow and use clean tap water. Good luck
