[PATCH 5.16 151/186] iavf: Add waiting so the port is initialized in remove