Ouch.
These won't be solutions, but might reduce the annoyance: sticky yellow traps, and top-dressing the pots with something that sheds water and stays dry: coarse bark chunks or Perlite or very coarse grit.
It sounds like you need a harsher insecticide. If your approach is "anything but that!", you might consider re-potting the plants affected ... but that's a blind shot in the dark.
If they were fungus gnats, I might urge you to use a LESS organic fertilizer, if you do use compost or organics that might be decomposing in the pot and encouraging fungus. But I think the soap would have killed fungus gnats.
I hope you get some useful suggestions!
