The shriveled up prunes are usually just as good as plump ones, some just shrivel more than others. I pinch test them when I crack open the pods then I do it again after drying and then again when I go back later and seperated them into individual bags for sale. If they don't crumble like a dried dead leaf when pinched they are usually good and I will keep those for myself and the plumper ones go into the for sale. If I know a cross is going to be just for me I don't bother drying them before going into the fridge. I use the glassine envelopes so no moisture can get in and I make sure the fridge is cold enough. I have never had a problem with mold in these or even when they are in the plastic baggies.
One key thing that I do is spray my hands with bleach water and keep a paper towel damp with the bleach water to occasionally wipe my hands when handling the seeds.