In nature, plants put down roots and stay in one location throughout their lives. Changing their location when potted tends to stress them out as they are forced to adapt to each change in light. In addition, plants tend to react to environmental changes very slowly, so they require patience. As much as anything, yours has suffered from too many experimental changes in location, especially light.
I recommend that you find a location in your home that provides north or east window light and keep that window completely uncovered throughout the daylight hours. Rotate it in place weekly so that it grows straight. Judging from the size of the pot, it probably needs water when the top half to one inch feels dry.
If you move it outside, you will introduce too many environmental variables and then you will have to move it back inside again when the weather turns cold forcing it to adjust yet again.
Finally, don't try to "fix it" with fertilizer or repotting. Just find a good location for it and exercise patience as it slowly adapts to that location.
