As
@Serpent points out, your trees (I see brown spots on your other tree as well) will have empty spots in them that won't fill in very quickly. If you are primarily concerned with appearances go ahead and pull them out.
But, its important to figure out why your tree got damaged before planting another. How often do you water? Do you fertilize? Anything else?
