I've been gardening for over 40 years and no, i've never considered giving it up; but, hard as it might be for a lot of us here to understand, it isn't something that everyone wants to do... I did give up golfing so I'd have more time for the garden, though
It's a hobby -- if it isn't something you love doing, why not find something else to devote your time to?