Answer: The rotting at the stem end of the fruit may be caused by bacterial soft rot, which happens more often in humid weather. Make sure you are harvesting the peppers regularly and avoid bruising them. To minimize damage to the plant and the fruit, cut them off the plant rather than pulling them off.
Without knowing what insect is bothering the plants it is difficult to make a specific recommendation. There are a number of pests that may attach peppers including armyworms, weevils, and hornworms. If you can find the pest, you could handpick it off -- this is very effective with a hornworm problem because the hornworms are large and just one can do a lot of damage.
However, neem oil based sprays seem to have a repellent action so perhaps that would help. You may also want to consult with your local Cornell cooperative extension for help in identifying the problems more specifically.
Q&A Library Searching Tips