Mixing plants, particularly when one will go fully dormant and have no need for moisture, can be problematical. Growing everything outside year 'round, is a whole other world. I am not sure that pea gravel makes any difference way up there in New York, but if you have concern, you can always use a mulch, such as clean cypress. I buy this by the bag and it is light in color. It would not only look good but also help keep the potting soil cooler and more moist.
I never have worried about adding soil to an existing plant, at least within reason. An inch or two won't make a bit of difference. Plumeria are kind of different trees in that regard. Speaking of watering, I know some folks say to let the top inch of soil dry before (thinking about) watering, but because we are so hot here in the spring and certainly the summer, I do my best to keep the potting soil slightly moist. My plumeria thrive when I can do that. They are water and fertilizer "hogs".