It's is a bit harder when you work full time, but it actually forces you into being a more discipline plant caretaker... when I worked full time, my days were very long because I commuted into Manhattan everyday, and I had hundreds of orchids... and it's because I love growing orchids and gardening so much that I chose to work independently at home when I had the opportunity... now I can just go down to play with my orchids and hanging out in the garden any time I want:)
But everyone's situation and responsibilities are different... it's a choice that only you can make for yourself, and if now is not the time, just remember you will have plenty of time one of these days... as long as you are happy and enjoy the plants you can care for now:)