You can do any of the things you mentioned. For aesthetic purposes I might suggest removing bloom stalks back to the first leaf below the stalk. I would not see a need to remove the entire stalk to the ground until winter freezes them back.
Thanks for the question. Please stop in again soon!
Q&A Library Searching Tips