I do both, tend to wear them, but do take them off for some types of weeding. I probably buy a new pair of snug fitting deerskin ones, from a farm store, every other year. Will grab an extra pair if on sale! I worked as a nurse until retired last year, so tried to keep dirt from looking permanent for obvious reasons
Also, I have a tendency to get blisters just above palm side of thumb when using pruners.
I learned that when they get really muddy and dirty - I toss them in the laundry, let them air dry, remembering to leave them in kitchen drain basket - that way I see them and so pull them on a few times while damp and wiggle to make them more limber. But they usually feel fine after the first hour or two of use, even if they are stiff at first. Not sure why I never thought to do this years ago!