It wasn't you who was called an oxymoron.
It was your statement about what he was saying about what you were doing and that became the oxymoron and no it isn't some kinda ubermoron unless that makes you feel better in which case you can be one if you want to be. One.