Looking into this again.
Unfortunately, couldn't see how I would do bisection on the issue, as it appears with that 5.4.118 commit implicated by the call stack,and with tha tremoved,is obviously gone(that I tested, 5.4.135 with b56ad4febe67b8c0647c0a3e427e935a76dedb59 reverted runs smoothly for me, while the original 5.4.135 with that 5.4.118 time commit in, now on a dozen machines, throws the WARNING.
I got email on the side of someone (Igor, on Cc) who sees the same with DELL servers, a newer 5.10 kernel, for him running IPVS + he sees actual operational impact there.
I just had a look at Linus mainlinetree, and see there is this followup / further fix from Peter Zijlstra, https://github.com/torvalds/linux/commit/3a7956e25e1d7b3c148569e78895e1f3178... ; now I'm much too incompetent to try and backport that, as it looks more involved, but I imagine such a backport would be needed to fix the WARNING (or IPVS breakage of Igor) we see.
best regards Patrick