<<<I wait until the leaves fall off/after the first couple frosts? >>> in my part leaves start falling off after the first light frosts which it can tolerate. But I would place under cover when there's threat of the lower 20ºF's since its still a very young plant.I would say not lower than 28ºF. Then in should go.
Arturo
