Jennifer, if you haven't already bought the insecticidal soap, a better solution might be some Bt - Thuricide is the brand name that comes to mind.
It's a naturally occurring bacterium that makes caterpillars get sick and die, but doesn't kill off every other bug it hits. Just the caterpillars that ingest it.
