Answer: First off you need to verify whether you are seeing flying ants or carpenter ants, or termites, or exactly what.
If they are carpenter ants, you need to determine where they are coming from. The flying ants are leaving their nest, so they do not represent an immediate problem unless they are relocating into a building. If they are nesting in a dead tree, for example, they might not need to be treated.
If they are nesting in your house foundation then you would want to remove the nest and make any needed repairs to the structure. Generally they nest in damp areas so you may also need to do some waterproofing and/or exterior drainage work as well to eliminate the damp conditions they like to try to avoid a reinfestation.
As far as an organic method, they can sometimes be treated with boric acid powder dusted into wall cavities and similar areas in addition to locating and removing the nest and making repairs to eliminate damp or rotted wood and other materials attractive to them.
Given the potential severity of the problem, I would strongly encourage you to work with your local county extension to obtain a specific identification of the insect and based on knowing that, determine how to proceed.
Q&A Library Searching Tips