Your garden is for your enjoyment, so definitely do that which makes it more enjoyable to you, within the parameters of what the plants will tolerate, as related by some of the anecdotes others have generously shared. Somewhere in the range of doing nothing at all, to examining every dead leaf, you'll find the style that works for your enjoyment.