I noticed, that Cleome could even sprout, between sections of concrete! I think the issue may be partly due, to if they are from your own plants, versus from a seed packet. But, with it considered being a 'warm season' annual, may cause one to delay sowing & found out long ago, it did better,, if sown much earlier than one might imagine. As suggested above, it seems to expect a cool moist period, followed by a warming trend before it wants to germinate. A short stratification & when the temperature is just right, they all of a sudden appear to be sprouting at about the same time. When that is happening, they do need to be kept moist, at the very least until the pair of seed leaves have developed.
