I know I've read about harvesting the greens and cooking them a certain way to avoid the toxins. You have to boil them twice to get rid of the toxins, and I'm pretty sure I read you should stick to the younger leaves.
You'll find most everything except vegetables in a garden is poisonous if ingested. Who leaves toddlers alone where they're not being watched that would even give them an opportunity to eat berries and flowers and leaves? And any child older is warned to not eat unknown things. If you've got a weird dog that likes to eat everything in sight, I'd say don't grow it. But I've never had a pet that ate anything other than a few chomps of grass, I just don't see it as anything they'd be interested in.