I have gotten good results using a combination of sticky flypaper and insecticidal soap (used as a drench for the soil). I believe that is the least toxic of any of the options mentioned so far in this thread. I use Safer brand insecticidal soap, and buy it as a concentrate. I am careful to drench the soil thoroughly when I water with it (which requires watering in more than one pass, waiting a few minutes in between, to fully hydrate the soil).
You cannot control gnats by only focusing on the adults (using flypaper). You have to use a combination of approaches that trap the adults and kill the larvae. Use flypaper placed near soil level as a way to monitor how many bugs are in the area, and assess how well your treatment is working. Treat every single plant in the area when you kill the larvae. Leaving 1 plant without treatment will create a reservoir for future infestation. Remember that all it takes is a couple of flying gnats to start up the problem again. Look at where they might be coming from: soil, other plants you already have, plants outside, new plants that you acquire.
Avoid complicated modifications related to top dressing. What you have described will only trap moisture in the soil, and promote the growth of more gnats.
Ideally the top layer of soil should dry out every time in between watering. Reducing watering is probably the best preventative measure you can take with any serious gnat problem. Bottom watering should yield the same results, roughly, as top watering when you do it right (ie. allowing the bottom layer of soil to get properly wet and waiting long enough for the water to wick upwards).
Be prepared to repeat the soap treatment and replace the flypaper on a periodic basis, but wait in between treatments in order to assess the results by counting bugs on the flypaper. This is definitely a problem that you can solve with care and attention. Once you have beaten down the bugs, just leave a little square of flypaper somewhere near soil level to give you an alert if the situation takes a turn for the worse, before it gets really bad.
