That's the ticket, Carol. Go out with a flashlight and look on the undersides of the leaves at night. I'll bet you see some little green caterpillars. These are not butterfly caterpillars so you can feel fine about picking them off the leaves and throwing them outside, or whatever else works for you. I get them on my brugmansias too. Bt (an organic insecticide) will eliminate them but hand picking will work if you only have a couple of plants. Just be thorough.
If it was a hornworm, there wouldn't be any leaves left at all, jmho. They're voracious, get big really fast and when they're full grown are the size of your finger. Green and white diagonal stripes and a red 'horn' on the back end, very distinctive.
