Latin's no help at all in the case of clematis. It comes from a Greek word, κλῆμα, which actually is pronounced KLEE-ma.
You're right that people who insist on the "correct" Latin pronunciation are pretentious or uninformed. I took Latin at a Catholic school and a public school. The pronunciations were radically different. Ecclesiastical Latin sounds nothing like Classical Latin, and we don't follow either in our pronunciation of many Latin words and names. We pronounce Cicero as SISS-uh-row, but Classical and Ecclesiastical Latin would pronounce the name as KEE-ke-ro or CHEE-che-ro.
The Romance languages are no help because most of the "correct" pronunciations don't follow the Latin rules or the rules of most Romance languages. If they did, our pronunciation of Campanula would sound like kahm-pah-NOO-lah, but the pronunciation offered in most botanical dictionaries is kam-PAN-yoo-lah.