Several possibilities for you to evaluate and rule out:
Silver birch trees (Betula pendula) are known for having shallow, spreading roots. These roots tend to grow near the surface of the soil and extend outward, often reaching beyond the tree's canopy. This could lead to nutrient deficiencies but you would need a soil test to confirm it.
Weather events: highly fluctuating temperatures (sustained warm but then temperatures quickly crash below freezing). Aim for uniform moisture and soil that is not too wet or too dry. Moist, well mulched (3 to 4 inches or organic mulch past the drip line) soil helps when temperatures are about to crash.
Protect from camellia bud mites, which cause buds to develop slowly and either open late or fall off before opening. A single spray of a miticide in usually prevents injury but a second spray applied in 10 - 14 days may be necessary to kill those mites which have hatched from eggs.
Camellias that drop their buds year after year may have a varietal problem that is weather related or a problem of location that can be solved by transplanting. Sometimes you also have to replace them by a variety that blooms earlier or later.