I woke up this morning to find a number of my potted herbs in my kitchen are crawling with these little white bugs. I monitor these guys every day or so and have not seem these before today.
Some of them move and some aren't but they are very small and everywhere

Any suggestions as to what these are and how I can get rid of them? I am thinking a heavy rain or a shower could hopefully do the trick. As these are herbs, I would like to avoid spraying them with anything that is not organic.

Ummm, eggs hatched. I use self rising flour for caterpillar and bug control, sift lightly on the soil around the plant base. I cannot tell what your bugs are tho...could be aphids. Aphids won't die with most insecticides, it can't affect them, if you use a water sprayer or mister, add a bit of non residue soap to it. Let dry then rinse with clean water and see if it helps. Another method is 1 part non scented bleach to 9 parts water, soak above the brim for 10 to 15 minutes, then rinse well with clear water. (for clay or plastic pots)
white color, whiteflys? aphid? Good Luck

Try neem oil. Spray it on each and every leaf cover front and back side of the leaf. Spray it in evening when the temperature is down.

I always use neem oil on my herbs and veggies . It works for me.

Your bug problem is due to keeping the soil too wet. Allow the top of the soil to dry well between waterings. You can put all the stuff you want on your plant but if soil conditions stay the same you will keep having bugs.