Ants generally don't harm plants. If they are nesting in the pot though, you might have an invasion on your hands in a week or two. I'd put a little bit of diatomaceous earth powder around the bases of the plants to discourage them. They will pick up and move on.
You can get d.e. in a bag at any big box store. It's completely inert, but is sort of like tiny pieces of broken glass to the ants and if they walk through or over it they dessicate and die. Be careful not to breathe the dust as it can make you cough.
