[PATCH 5.10 446/563] spi: uniphier: Fix a bug that doesnt point to private data correctly