I've had gnats in my chilli pots before! More of a nuisance than a problem! I used soapy water to start, it took a couple of goes but it worked, after most are gone you want to let the medium dry out! they don't like the dry! So if possible let the soil stay dry as long as you can and any thats still there will clear off! Once the soil is dry you could try placing another pot filled with damp soil beside it or near it, they will leave the dry soil and lay there eggs in the damp soil! You can just ditch it then!
