Everything I've looked at concerning this plant leads me to believe it's Nyssa sylvatica as well.
One very specific trait of these trees is the main trunk formation. Does the main trunk go straight to the top with no "Y" or complex branch offshoots? (Is each main branch attached to the center trunk?)
Have you observed any cavities in this tree?
Are there any other specimens in the vicinity?
Are the leaves thick-ish and somewhat leathery; similar to, but a bit thinner than a magnolia leaf?
Do you get the feeling that this tree was planted there, or that it was started from natural occurrences?
Are there any shoots showing around the base of the trunk?