Melanie, I have had several passion vines in my yard for years, the Gulf Frits visit in July-Aug-Sept. I have seen very few butterflies and no caterpillars at all. I do have this assassin looking bug that also likes the passion vine. I am wondering if his kind are a problem.
I saw a black swallowtail hovering around what's left of my parsley bush but it was much too quick for a photo.
Other than the parsley, do you have a preference for a host plant, I saw you mentioned "rue"-
thanks
John