Check for damage at the base of the plant. A wound could have allowed a fungus infection to get into the wood. I would recommend you remove all the branches that have discolored leaves. Good luck, and let us know if the plant recovers! Skimmia is such a lovely shrub, I wish I could grow it in this Zone.