Marilyn, I'm a newbie at this (but it is going on my 2nd spring for some of the plants) and I noticed that the only time I've tossed out semps is if they've bloomed and rotted to the ground, or just rotted - but in both cases I check really carefully, because when I first was tossing browned semps I flung it over my shoulder and as it passed my head I saw some babies clinging to the rotten mushy brown plant! Been super careful ever since!!
It seems they want to populate! I find this so different from most plants - and exciting, to see them look so straggly and then have something come back!! Anyway, hang in there my friend! After this first winter you too will be an expert!