Passiflora edulis is the edible passion fruit and does not do well here in my garden. However most people grow Passiflora caerulea which does seem to grow well on a south or south west facing fence/wall. We have no trouble with them at all. The frost can knock them back. But our springs are early and we get daffodils in February. The passion flowers shown are mine and keep flowering till the first cold nights We have Robins here that are not like your ones as they are small and have a redbreast, so I leave the yellow fruit on to fall, as the Robins will peck at them for the seeds.
Kiwi fruit also grows well here, and the evergreen clematis or Clematis armandii, which is said to be tender.
Regards.
Neil.