Another shrub fruit I just love are red currants. My mom used to grow red currants when I was a kid and I would go and eat them right off the shrubs when ripe. Mom would pick and make a sort of jelly. I still grow the red currants today.
Here is a view showing the tops of the red currant shrubs that grow near some white shrub roses. That is my old cherry tree trunk in the picture. Just shows you can mix edibles and ornamentals.