[PATCH 6.3 632/694] rtc: jz4740: Make sure clock provider gets removed