Alex, the buds won't hurt your plant. I'd probably let them bloom. However, I wouldn't let them set seed. However, I would cut the blooms off as fast as they start to fade. In my experience, it's the setting of seed that zaps a plant more. Have fun, you have a great selection.