It's always hard to pull fruit off of plants but they'll be all the better for it. When people tell me they won't cut their flowers for bouquets I remind them that it will produce more flowers if they do. Example, I cut ALL the daisies that are in blossom every week and the next week this is what they look like. Same thing with the row of zinnias.
I had terrible germination of my sunflowers and I would do a second planting but I used all my seeds!
next year.