If by any chance your temps will hover below 40 like what happens in my area where we hit very low 30's to 21F in January-Feb then I will bring them in. I have learned my lesson very well about it..lost some even while being protected by my growcamp.
In the high 40's to low 50's that is still manageable, as mentioned by Elaine already. My Phals are always indoors, so they are okay. If your rescues are still very young or fairly new rescues, I would consider bringing them in now. They are already stressed.