Musas require a warm, humid environment with light but rains. NYC weather is too variable to grow them outdoors. High temps and frequent downpours are what is damaging your plants. It would do much better if you kept it indoors in front of a sunny window. Indoors, temps are less extreme, but warm, and you control the watering rather than relying on the vicissitudes of mother nature.
