DnD, I'm pretty sure we can rule out the "garden genius" thing...
I start my seeds around April 1st, in 6-packs (8 packs to a flat), indoors, under lights. Around May 1st they get moved out to the greenhouse, and soon after that I move them up to 2-1/2 to 3-inch pots. Usually plant them outside around June 7th, and I cover them with "floating row cover" (like Reemay, Agribond, etc), which pretty much eliminates the need to harden them off because they're protected from the wind and have some shelter from the sun, as well. They get compost and a little triple-10 fertilizer, and once they start setting fruit I usually give them some Miracle-Gro.
My peppers mainly get planted directly in the ground, but I've also had good luck with them in containers -- I give each plant a pretty good-sized plastic pot (maybe 8" to 10"?), and just use Miracle-Gro potting soil (which I know a lot of people don't like but I've been using it since long before I ever heard there was better stuff out there, and since it's readily available here and I've been happy with my results, I've just continued to use it). Maybe your plant dried out too much in the clay pot?
I tend to "think" of peppers as being a hot-climate lover, but apparently that isn't true because even when we have unusually cool summers (like last year) they do well. One of my favorite things to grow because they seem so trouble free; the only problem I've really had is that the plants get so tall and have so many peppers on them that they need some support or they fall over!