I suffer a similar malady. I gather seeds impulsively, many of which I don't really want, then find them in my car, on potting bench, in a drawer.
Only by leaving the chaff am I able to know what most of them are later, but it sounds like you are fastidious enough to label them anyway.