They should be dug when the foliage has yellowed and died. Also the bulbs should have developed a brownish tunic (outer skin). If the bulbs look light in color with a poorly developed tunic it usually is a bit early to dig.
This will give you the optimal quality and these bulbs will be excellent to dry and store for autumn planting as well, if that is your plan.