There's no doubt that there's great fertility in worm castings, but this might be a better addition to a ground garden. In the tiny, closed environment of a pot, it's virtually impossible to provide fertility & nutrients via soil for plantings that aren't temporary (like porch pots of annuals.) The microbes that make nutrients available to plant roots aren't able to survive in a pot, and tiny particles of any kind can cause roots to suffocate (and then rot, called "overwatering.") Tiny particles lodge closely together with no air between them, stopping the air circulation and evaporation process, and this can cause the suffocation. Roots need oxygen & moisture at the same time to function.