Where I am, my plants are usually still green when we are getting pretty close to having snow, so I wait as long as I can and then cut them off near the ground, and burn the tops. I don't like to leave them in the garden because I usually have some asparagus beetles and I don't want them to be overwintering in the garden.
I agree that a couple of clumps won't give you much more than a taste... you can buy more roots, or save those red berries and plant them; it takes about 3 years before you can start harvesting from seed-grown plants, but it's free and the time goes by a lot faster than you would think!