I'd leave them outside in your zone. Regarding the leaves, if they are dry and the ground is dry, then they should come off very easily. It you have to tug at them hard, I'd leave them alone until they come off easy. Because of the rot problem with all the rain we had early and mid summer, I moved a lot of my plants to areas where they were protected from the rain - saw the same problems your picture shows as the late summer was very dry. (You did notice it's now Fall?) Now with the cooler temps and some rain, my plants, recently moved back, are all showing some nice growth and fall colors. I've been picking off the dried leaves as opportunity permits. Just be careful that the soil and plant are dry when you do this, so any wounds get a chance to heal.