We can learn so much from "failures", however they come about.
So did you end up removing the seed when the pod was spongy? Not really knowing anything about this, I'm not sure when the best time is. I hear it said to wait "X" number of weeks from pollination, but it must be quite variable from lily type to lily type, and thus not a good standard to use. As I understand, the predominant problem is that the seed embryo and endosperm are incompatible, having different origins. One would surmise that a more mature embryo would be advantageous, as long as the physicality of extraction could still be performed. So would this be at the first detection of the pod's loss of turgidity?