Answer: All of the plants you list are frost-tender and they should all be taken indoors over the winter months. Instead of planting them in the ground and then digging them up every year, why not keep them in containers and set the containers out in the garden? You can even bury the containers in the ground if you'd rather not have a collection of containers sitting around your garden. Then it will be easy to move the plants indoors when the weather starts to get cold. Coleus is generally an annual and probably won't perform well indoors, but you can certainly try!
There are lots and lots of books available on growing herbs and on successfully growing houseplants. There are so many, in fact, that it might be best to thumb through them at your local library or bookstore and choose the ones that most appeal to you.
Best wishes with your herbs!
Q&A Library Searching Tips