That is the most aggravating thing about hardy hibiscus. What I do when I collect Hardy Hibiscus seeds is make sure they are totally dry then put them in a small plastic bag with a bit of Diatomaceous Earth (totally inorganic and has absolutely no poison in it unless it's the baited kind). The DE will kill anything that hatches. After about 2 weeks, remove the seeds and all the seeds that have round holes in them - that's where the larva was and they aren't viable.
I finally found a way to keep this from happening. I hand pollinate whatever flower I want to pollinate, cut off the petals and slip an organza bag over it and don't remove the bag until the pod is mature and starts to crack. I did that for the first time this year and I'm delighted to say, not one bug hatched out.
I get the organza bags at the Dollar Tree in the Baby Shower section.