What you describe as your patchwork garden is how I made 90% of my gardens! Except my problem is when I am at a nursery looking for that one plant on my wishlist that I have to have, not finding it but leaving with a bunch of plants that I have no idea where I'm going to plant! However I find even when I plan out stuff by height and bloom time it still doesn't always work out the way I hope. I think a holding garden helps us see a plants habit and decide where it will work in our gardens.