I don't have much trouble with the ones in pots, but I have had trouble with tags with those planted out in the gardens so I made a map of what is planted where. I did this by taking a picture and then editing the photo with the name of the plant over top of the plant.
I also try and plant contrasting semps next to each other so that when they grow together I can really tell which one is which. I also try and take a LOT of pictures. Besides showing the individual characteristics of the semps, the picture shows what pot it's in or where in the garden it is.
