I did some research and found this information from Wikimedia
Bee nectar collection from Datura plants in Mexico and Hungary, belladonna flowers, henbane (Hyoscamus niger) plants from Hungary, Serjania lethalis from Brazil, Gelsemium sempervirens from the American Southwest, and Coriaria arborea from New Zealand[42] can all result in toxic honey,[43] as can honey made from other toxic plants such as oleander.[44] Narcotic opium honey has also been reported from honey made in areas where opium poppy cultivation is widespread.[45]
Various beekeeping discussion groups seem to be divided on the issue. Some say it's toxic to humans, others say if it's pure datura honey it would be toxic but small amounts mixed in with other nectar may not be toxic.