You're right that it's not lavender unless there's a type of lavender that I've never seen. Plus, lavender has a recognizable scent.
Don't know what you actually have. (Sorry. Not helpful.) I just want to point out 2 things. The first is that the leaves are similar to most mints. Thus your friends could easily think that it's mint.
Second, and forgive me if you already know this, but there are many many many types of mint. Spearmint, apple mint, chocolate mint, mojito mint, orange mint, lemon mint, etc. Sometimes you have to rub the leaves to really determine if it's mint.
Those little flowers are throwing me off.
So, while waiting for a mint expert to actually answer your question, let me ask you a few.
Had anything been growing in this soil prior to the mystery plant?
Why did you think you planted lavender? Did you have seeds? Or did you get a small plant from a big box store or local nursery that might have been mislabeled?
Let's hope that a mint expert will jump in. I'm sure one will.