I found this in an extension.org article about powderpost beetles in wood(not suggesting that's what this is):
"Active infestations are characterized by frass streaming from or accumulating around the exit hole on the wood’s surface. Adult powderpost beetles are rarely seen. Exit holes with no frass present is evidence of a prior infestation, but not necessarily one that is still active. "
I did read somewhere else on borers in plants that some may push debris/frass out of the hole to keep it from accumulating in the hollow where they're still feeding.
I would think if the culprit was long gone the debris/frass would have been blown or washed away. Perhaps, Leslieray, you could clear away the white stuff and see if more accumulates again, that might indicate whether it is still in there.
