Your soil is being washed away. Do you have some kind of non-porous barrier between you & a neighbor? I can always tell where my soil & mulch go based 100% on what part of my concrete edgers are buried deeper than others. Plus, most bagged "soil" isn't actually "soil" (dirt) at all but rather finely ground up wood products, which decays quickly.
