The warm soil, the connection with the roots of a plant (tenderly seperating them when I have to split them is easier with hands), and weeding are all better for using bare hands!
For the big stuff - wait, there Is No Big Stuff in a small garden......... so I never wear gloves unless I have to, and then it's not gardening.

Then it is just plain *work*!

oh, you've GOT it, and who likes that 4-letter word, Work! Gardening "in the raw" is fun! That means bare toes and bare fingers! Anyway, who likes a "funny tan line" where your gloves stop?