Thank you both. I've been guessing that it's from a tree but I don't think any of my neighbors have cherry or crabapple trees. I have a Carolina Cherry Laurel (Prunus caroliniana) as does at least one of my neighbors.
Carolina Laurel Cherry (Prunus caroliniana)
If that's what I'm seeing, and I don't know that it is, then the birds and the wind must be dropping those seed everywhere, which is a possibility. I'll have to grab a couple more and hope for better pictures.