Annie i can see your problem,..i tried looking for "how to remove pieces of wood handle remaining in the metal of a fork",..nada,..they all tell how to insert the handle and of course there were no pieces of wood preventing the handle from being inserted
I have come across friends burning out the remaining wood and it didn't weaken the metal,..seems like your idea is the only solution,..using the propane torch

best of luck.