Answer: Unfortunately, it sounds like your tree may be suffering from several problems at once. The consecutive holes might be caused by a bird drilling into the bark in search of insects, the defoliation could be caused by disease, pests or possibly a nutrient problem or even mechanical damage to the roots or trunk. It is almost impossible to make this kind of diagnosis long distance.
I would suggest you contact your county extension and get help to determine the most effective care program for your tree. In many cases a routine spray program is needed in order to produce quality fruit, but the program depends on the specific kind of apple tree you are growing and its resistance level to different diseases as well as the local growing conditions.
They should be able to help identify the problems causing your tree to produce poor crops and also give you the most up to date control recommendations as well as the optimum timing for any treatments. (Sometimes you need to start in the winter, with follow ups later in the season.) They should also be able to help you figure out the most appropriate fertilizing regime and suggest pruning techniques.
Good luck with your tree. I apologize for the delay in responding to your question.
Q&A Library Searching Tips