I grow some herbs here in sandy conditions, very little fertility, and they do fairly well. Although I've not had much success with the mints. I agree that your annual basil does prefer warmer weather. But in my opinion, most herbs really prefer poor soil conditions, so the fertilizing might be too much for them. Strange as that may sound.
