[PATCH 5.1 153/371] clocksource/drivers/tegra: Restore base address before cleanup