Answer: Based on your description I am not certain what is wrong. However, ferns should come back bigger and better every year and these should actually spread a bit. If the soil is not organic enough, or if the soil is not moist enough, they may dwindle away.
Ferns need soil that is organic and humusy, evenly moist and acidic. Usually the best approach is to prepare the planting bed by adding organic matter such as compost, old rotted leaves, or possibly very well aged and rotted table manure/bedding material. You would also check the pH to be sure it is on the acid side of neutral. Loosen the soil down about a foot when you encorporate the organic matter. After planting, mulch around the plants with organic mulch, it should be two to three inches thick year round. Add more as needed to maintain that depth. The mulch helps feed the soil and adds more organic matter as it breaks down over time.
If your soil is not well prepared, you could lift them, re-work the planting bed and then replant. Do not let them dry out while they are out of the ground -- set them in a shady area with mulch heaped over the root balls.
I hope this helps you trouble shoot.
Q&A Library Searching Tips