Answer: There are borers which will attack and threaten the life of a dogwood. However, your description sounds like the bark fell off first and borers entered the inner wood portion later. Such borers attack dead inner wood parts and are not a threat to the living bark and cambium tissues. Their tunnels can weaken the trunk a little however. Sometimes, you can poke a piece of wire into the hole, and impale the caterpillar(s). It is difficult to control borers with sprays, since they are inside the trunk.
Your County Extension office (910-593-8179) might be able to recommend a borer treatment as well as the proper application method and timing, if you decide to spray.
Q&A Library Searching Tips