[PATCH 5.15 202/207] spi: spi-sun4i: fix early activation