I get quite a few "volunteer" rudbeckia in my garden. I let many of them grow, but I also am quick to pull the ones that are in the way of my intended plantings. Rudbeckia have a tendency to crowd out other plants like squash and daylilies. On the other hand, rudbeckia attract pollinators to the garden, which helps the crops, and they also attract birds that feast on the seeds, so I even leave the browning seed heads until the seeds are all picked out before I pull up the plant at the end of the season. Here is a photo from my garden in June. All of the big clumps of yellow flowers are volunteer rudbeckia.