I have a worm tower (currently residing in heated garage for the winter) and, while I've never put coffee grounds in there (saving them for the outdoor compost pile), my red wigglers love bananas. They'll eat all but the outer most layer of the peel. Not sure if "outdoor" worms would be any different in their preferences. You might try bananas under a layer of shredded leaves.
