To remove the bugs from the plant, I would wipe it clean with a paper towel soaked in 70% ethanol first. Start at the bottom of each leaf, holding the paper to the surface, and wipe upward toward the tip. Maybe change paper towels if the bugs start piling up. Since there are no spines on the leaf surfaces, this should be quick and easy. Then you can address any remaining visible bugs with ethanol spray, and repeat after a few days if you see more action.
That cottony stuff does look like mealybugs but mealybugs are not fast-moving animals in my experience, and they tend to concentrate their effort more toward the growth center, especially at the base of soft young leaves. I don't remember seeing them on the underside of aloe leaves, but I'm sure they end up there sometimes too. In any case it's really easy to wipe your plant clean... but like Will says, just be vigilant for a while afterwards (including watching your other plants) to make sure the cotton fuzz doesn't come back.