[PATCH 5.15 230/262] iavf: Add waiting so the port is initialized in remove