When I was a kid, my grandmother had an apple orchard with trees that were 100 years old (She claimed Johnny Appleseed had planted them). The trees were huge so we only got the apples that fell. Two rules:
Don't eat the bruise
Don't bite to the center of the apple
We picked the apples up as soon as they fell so never had to deal with mold. I suspect the mold will affect the taste of the apple though and you won't want to eat them. Worms are bitter - you won't want to eat them either.