I read through this and am late to the party, but, FWIW, this kind of bug (which I think is a millipede,) is not a pest of the plant, just the soil in the pot. The fact that they were leaving the pot probably indicated that there wasn't enough sustenance for them. If they were eating some other critter, they must have eaten all of them. A potted plant is not inherently a suitable environment for any soil-dwellers. Finding them (in the pots of plants that are inside) should be cause to investigate things such as the appropriateness of the soil, watering habits.
Dunking a pot in a bucket of water so the soil is completely submerged should cause anything that breathes to evacuate, or drown. I do that before bringing plants inside.
