Joan, it's always better to provide habitat than not. Out of every 100 eggs laid, only 1 or 2 will make it to adulthood. Caterpillars are a pretty important link in the food chain, and birds don't usually discriminate and eat only the ugly ones.
I'm sure there are situations where they run out of food in the wild, too. Just having the host plants for them is giving them a chance.