I'm able to leave most hot peppers on the plant till they fully ripen and have no trouble with them not setting more...more cayenne plant produced around 40-50 peppers this year, my habanero plant almost as many. Bell peppers were a bit slow this year but I still got 5 or 6 per plant. I had several Yellow Admiral pepper plants and I would take those off the plant as soon as I saw a bit of yellow and let them finish inside.