Well, they come from somewhere. Probably arrived with the plant. It can take a while for a plant with mites to show symptoms. To the extent you have mites on any of your plants (or might have them because for example you kept an offset from an affected plant) that is the biggest potential source of the problem. Zero tolerance is key. Clean the container well (ideally with bleach) after you empty it, and toss the plant in the trash, not the compost.
I don't think stress brings on the mites, to the extent I haven't done any actual experiments. The aloes in the ground here endure pretty serious stress every summer and that doesn't seem to provoke anything particular mite-wise. Some plants are much more susceptible than others to the aloe mite. Some of the more frequent hosts here are arborescens (esp. on the inflorescence), nobilis, spinosissima. Some plants are known mite magnets in cultivation, like longistyla. Some plants are known sources of mites in cultivation, like old tree aloes with affected branches high above the surrounding vegetation. Like this fabulous plant over a Pasadena nursery (can't see the mites in the photo, but they were definitely there):
I wouldn't even think of buying an aloe there. Whenever I see aloe mites in a nursery, I tell the people in charge and make a point of not buying any plants.
More on mites here (under Care):
The Aloes Database