[PATCH 4.14 112/205] mlxsw: spectrum: Properly cleanup LAG uppers when removing port from LAG