[PATCH 6.1 056/204] remoteproc: qcom_q6v5_pas: Make single-PD handling more robust