We put our bananas in the fridge, unless they are green green. I prefer the flavor of bananas when they are still slightly firm and a tad green. Right now it is tomatoes from the garden that attract fruit flies, but it's only a handful of the insects.
I remember one year it was terrible in the middle of winter, but that was completely my fault. At that time, I knew nothing about vegetable storage and put a bag of potatoes on top of the fridge, then forgot about them. Oh, that was horrible!
It's like compost buckets and fruit flies go together.
I keep mine under the sink so it isn't too bad