Let's not conflate two terms that have different meanings. "Invasive" is a imprecisely defined word, but gardeners generally use it to describe plants that spread around their yards more than they would like. "Noxious weed" is a legal term applied to plants that cause significant ecological and/or economic damage. Laws vary from state to state, but it's generally illegal to grow plants that have been designated as noxious, in which a case, how it behaves in a given person's yard is irrelevant.