It would be worth the wait. I love these. We grow the smaller Brunfelsia pauciflora and it's in brick planter. very slow to grow but blooms nicely year to year. I would think in zone ten is would grow quickly. I think the one in the your photo is B. grandiflora.
@Dutchlady1 is growing that one in Florida and would be able to answer your question on how quickly it grows there.
