[PATCH 5.12 097/384] soundwire: cadence: only prepare attached devices on clock stop