Welcome to NGA, Tumbiola. Sounds like a hydrangea macrophylla but I am not 100% sure without a picture. Make sure you are not pruning the hydrangea after it has developed the flower buds at the ends of the stems (they are "invisible", hidden inside the ends of the stems). Make sure that deer are not eating your flower buds; flower buds develop in late summer / early Fall and open in Spring or, in the case of reblooming hydrangeas, they also develop in late Spring and open in the summer. Thus, if deer, squirrels and rabbits are hungry, they may eat the ends of the stems with dire blooming consequences. Or if you prune at the wrong time, you could be cutting off the flower buds. Make sure that you are not adding too much nitrogen rich fertilizer as too much nitrogen can produce nice green foliage and no blooms. If your soil has no mineral deficiencies, you can stop fertilizing your hydrangea once it becomes established in your garden, provided that you maintain 2-4" organic mulch (no rocks) year around. It will feed off the decomposing mulch. To prevent ole man winter from zapping last year's stems that are full of invisible flower buds, winter protect the plant when it goes dormant and remove the winter protection around 1-2 weeks after your average date of last frost. (usually the last week of May). Last thing to consider is dense shade and reblooming ability. If this plant is supposed to produce only one flush of flower only (is not sold as a rebloomer hydrangea) then its stems may suffer dieback from winter and that will kill the flower buds, as they develop in late summer of the previous year (like azaleas). If the plant is sold as a rebloomer, dense shade may not allow the new stems that start growing in Spring 2020 to grow tall and old enough to produce a flush of blooms in late Fall.
Of those reasons, pruning at the wrong time and not winter protecting would be the ones I would concentrate on. In your zone, winter is probably killing the old stems and this hydrangea probably only blooms once -in Spring- and as a result, all growth is from new stem growth from the crown (where the stems originate from) and the plant does not attain much size year after year since it is always "starting from scratch". It is probably not a rebloomer hydrangea as a rebloomer hydrangea usually would have managed to produce some bloomage in late summer via the new stems that start growing in Spring. Since it has not done that in 5 years, there is a good chance that this type of hydrangea is not a rebloomer.
The comingling of roots between the violet and the hydrangea might cause competition for water and resources. I can see this causing blooming problems sometimes but maybe not for 5 years. But.... have caution working in this area... hydrangea roots are tiny & fibrous, usually in the top 4" of the soil only so, be careful when stepping in the area or when cutting/extracting the violet.
