What might be eating the columbine leaves. This seemed to happen overnight.

If it happened overnight it's probably a noctuid moth caterpillar.
The big problem with them is they are very mobile caterpillars so you never know if they have stuck around or have moved on. BTK (Bacillum thuringiensis var. kurstaki) is effective and harmless to pollinators but some noctuid caterpillars and armyworms are immune to it, so weight your options carefully.