Arturo Borrero Gonzalez arturo@netfilter.org wrote:

[ cc stable@ ]

We experienced a major network outage today when upgrading kernels.

The affected servers run the VRF+conntrack+nftables combo. They are edge firewalls/NAT boxes, meaning most interesting traffic is not locally generated, but forwarded.

What we experienced is NATed traffic in the reply direction never being forwarded back to the original client.

Good kernel: 5.10.40 (debian 5.10.0-0.bpo.7-amd64) Bad kernel: 5.10.70 (debian 5.10.0-0.bpo.9-amd64)

I suspect the problem may be related to this patch: https://x-lore.kernel.org/stable/20210824165908.709932-58-sashal@kernel.org/

This commit has been reverted upstream:

55161e67d44fdd23900be166a81e996abd6e3be9 ("vrf: Revert "Reset skb conntrack connection...").

Sasha, Greg, it would be good if you could apply this revert to all stable trees that have a backport of 09e856d54bda5f288ef8437a90ab2b9b3eab83d1 ("vrf: Reset skb conntrack connection on VRF rcv").

Arturo, it would be good if you could check current linux.git or net.git -- those contain the revert + an alternate approach to address the problem that 09e856d54bda5f288ef8437a90ab2b9b3eab83d1 tried to fix.

If net.git is still broken for your use case it would be great if you could extend this test script:

https://patchwork.ozlabs.org/project/netfilter-devel/patch/20211018123813.17...

this would help to figure out what the issue is.