Alyssa, you're not supposed to plant different colors of raspberries together, something to do with a disease that transmits, I think. The red and yellow are all right together, it's the "black" or purple raspberries that can cause problems.
But blackberries and elderberries should be fine alongside your raspberries.