Answer: It won't hurt your wisteria to move it so soon. In fact, it will probably be easier to move it now than to wait until it expands its root system. I expect that the rootmass will be in exactly the same place it was when you first planted it. To lessen stress, soak the area the day before the planned move. Moist soil is easier to dig and you can probably take the entire root system without knocking off any soil.
Best wishes with your garden.
Q&A Library Searching Tips