I can't speak to the ones you have under lights—no experience with that—but depending on the species or variety of lavender you have (some are more hardy that others) the outdoor plants might be more at risk of rotting under all that mulch than the cold. 40 degrees shouldn't be a problem but extended freezing and below could be harmful.
If they are very young plants, they will need protection, but it's important that you don't allow the soil or plant to remain soggy underneath the mulch or insulation. Dry leaves are good. When they get wet, pull them away from the base of the plant to allow some drying. Rocks or gravel as mulch could be effective. Even if the soft growth dies back, it will resprout if the roots aren't frozen or rotted.
This first winter could be touch and go. When the plants mature they'll withstand winters better. Do research your type of lavender for hardiness.
Good luck