Good to know then it has drain holes, then just try and improve the media. Make it grittier.
It is also typical for older leaves to die out first, so those older lower leaves maybe manifesting that too.
What is the orientation of that window? Ideally, it would like as much light to direct sun. The more sun the better. No fertilizers for it is needed, just improve the cultural growing conditions it needs.
When I say grittier, this is how gritty I make my succulents media to be: apart from using cacti mix, I also mix in a good layer of pumice, chicken grit ( insoluble crushed granite), sometimes even finely crushed lava rock.
And aloes love the sun a lot!
