I found this website to confirm that those small brown things are "catapiller (looper) poop." That's helpful. And yes, shaking the plant at night with a flashlight helps shake the critters out.
I also learned that the loopers' eggs are laid by moths. When I shake the plant before the loopers show up, moths fly out from the plant.
Anyone know something preventative that would discourage the moths showing up?