"Good old fashion ground dirt" isn't necessarily a good choice as a potting medium, especially if it's on the clay side, even with compost and perlite. It's often too dense and doesn't drain well in a pot. Peat or coco coir would do better, plus coarse sand, pine bark and other ingredients
(Note: research the use of peat before choosing it..there are environmental considerations)
Much depends on your containers—5-gallon buckets? Raised beds? Some loamy or sandy soil could be fine in a raised bed, not so good in buckets.
Here's one article with several recipes
https://sfyl.ifas.ufl.edu/lawn...
And another
https://www.preen.com/learn/la...
but googling "homemade potting mix for (fill in type of container)" will reveal many ideas and instructions.