Answer: Your zip code places you in zone 5A or the coldest part of zone 5, but in an exposed windy location it certainly could be the equivalent of a zone 4 spot.
Clematis are the most reliable flowering vine for zone 4. This plant really needs sun to bloom well and most are not fragrant. However, as an exception to the rule, clematis terniflora is fragrant and will tolerate partial shade, so it might be a possibility. Another option might be to place containers -- such as a window box across the top -- on your shaded arbor and grow shade tolerant plants that way.
Deer seem to eat nearly everything. Clematis are listed as deer resistant on some lists and as deer candy on others. It will depend what your local deer have learned to eat so far and also on how severe the browse pressure is. Based on my own experience, if you have trouble with deer you need to consider installing a tall fence as the most reliable long term protection for your garden areas.
Q&A Library Searching Tips