There are several ways to propagate them. I grow ornamental sweet potatoes most years. I start in Feb to have nice plants in May so I am not sure about a timeline for regular sweet potatoes starting now. Yes, put your potato part way in water. I use a longer shallow container rather than the jar and toothpick but either will work. Basically the area in the water will grow roots and the eyes out of the water will grow slips (stems and leaves). Once the slips get to be several inches long I gently pull them off and put them into a jar of water till they grow roots and then plant.
Potatoes can produce slips for several weeks just remove what is long enough and put potatoes back in water to wait for more.
From the potatoes before root growth.
After a couple of weeks in water glass growing roots
and ready to pot up