HaHa, I did have to use a magnifying glass to confirm that my Common Milkweed is indeed 'fuzzy'! Couldn't tell by feel.
It wasn't until the six plants I rescued from a landscape crew started to branch that I even found out there was a Milkweed look-alike called Dogbane. So, I agree with Sallyg about the branching being a major difference. I think Dogbane is the prettier of the two plants with its red tinged stems and structural form (and no orange oleander aphids!) Easier to dig and transplant successfully than Common MW a shovel depth all around plant and I got most of it.
I didn't watch this video but it may be helpful:
https://www.youtube.com/watch?v=YvQnJSSvRLA
I'd also say that last year's stalks could be a clue as the Common would be the largest. My three + year old Dogbane is still more slender than a pencil. The Dogbane here emerges a week or two earlier than Common MW in same area.
Good luck Brendan
@bhart90. Hope you are able to figure something out.