My vote is for millipedes. If you don't want to use insecticides you can submerge the whole pot under water. The millipedes will pretty much all float to the top of the water and you can dispose of them. You might have to do this every 5-7 days until no more are found. Of course insecticidal drenches will work. Just be sure the label lists millipedes/centipedes.
