[PATCH 6.6 2/2] Revert "spi: spi-cadence-quadspi: Fix pm runtime unbalance"