The big fuzzy buds you see are next year's flower buds. If you prune them off, you will have no flowers next spring. So, although you wouldn't hurt the plant, pruning at this time is not recommended because of this.
When the debate is lost, slander becomes the tool of the losers. - Socrates