Change in light is certainly not the sole reason for Ficus leaf drop, just the most common. Improper watering is another common cause. In addition, these trees when grown indoors can support only a finite number of leaves determined by the available light. Once they max out, which happens when they are allowed to grow unchecked, they will start to shed older leaves so that they can continue to support new growth. This is why many inadequately pruned indoor Ficus trees look rather sparse in their interior portions.
In general, the larger the indoor plant, the more maintenance is involved. If you don't like the extra work, stick with smaller plants.