Weed seeds are adapted to growing in almost anything. That's why they're weeds. Vegetable seeds are not as forgiving. If I scatter tomato seeds on my lawn, I won't get tomatoes. If I scatter weed seeds there, BOOM. Compost can have salt, and bizarrely high nitrogen (as well as micronutrients). It's hard to keep it suitably wet, and once the plants are growing, offers only minimal support to the plant. If you've been successful germinating in compost, let's just call it lucky.