I would try to give them some cover - like move the window box to a sheltered corner and make some framework over it, then stretch plastic film over that.
BUT a warm, sunny day could cook them or start them growing again in time to die during a freeze.
Maybe make the plastic film only a partial cover - leave lots of vents open at all times. If the goal is only to keep the window box from going below -40F (USDA Zone 2), keeping it out of the wind and up against a wall might be enough.
You might dig out a few of each and give them to a friend, with the understanding that, if all yours die, you'll ask for half of them back.
They sound too small to take cuttings from.
