Thanks both of you for your suggestions. It's definitely not common milkweed and I don't think it is the hairy balls, especially here in zone 4 Canada. It reminded me somewhat of borage but the leaves are narrower and the flowers not blue but the buds did look somewhat fuzzy. It looked like some of the flowers that were open had a green back. I so wished I could have got over the fence to have a better look at it.
