I think humidity outdoors can be a problem if it is high enough and cause rot in the semps if one doesn't provide enough drainage in the soil and ventilation of fresh air. Bugs are an everyday problem for any kind of plant and so one must just be diligent about checking for them before bringing the plant indoors.
With your frequent moving the semp in and out because of these factors, would you consider another type of succulent that is also beautiful in their rosette shape and coloring? Echeverias can be kept indoors near sunny windows and don't necessarily need to be outdoors. They are tender succulents of different sizes and coloring that survive mostly within the 45 degree and above temps and can be placed outdoor in the semi shade in warmer temps but it would not be necessary. Semps don't do well indoors and you seem to have a couple of factors outdoors that may prevent your being happy with your semps outside.
