In my experience most gardening clubs participate to improve their local community by creating plantings or renovating plantings; it's about good-deed-doing by using pretty flowering plants.
Most vegetable gardeners that I have known are not interested in leaving their yard to attend meetings. That being said, I have met lots of vegetable growers who enjoyed sharing their knowledge - I just walked up to their fence and said, "Hello; nice garden!".
One of the best ways to meet other vegetable gardeners is to join your local community garden. Some places have plots for free; others have a reasonable fee for the season.