The key is whether or not they are able to make seeds (whether or not the plant is sterile.) If a plant can't make seeds, it can't spread and is harmless.
My Mom put a seed making one in her yard (same town as me) and it was not well behaved. After a few yrs, there was a patch of them, I dug them out.
