I've heard that a major issue with milkweed stands in areas like Texas is the growing use of herbicide-resistant crops, which mean that stronger and more potent herbicides can be used on them, which means that a lot of other native plants and weeds--including milkweed--are destroyed in the process.
http://www.mlmp.org/results/fi...