Hi,
The upstream commit below is backported to 5.10.186, 5.15.120 and 6.1.36:
""" commit ecdf985d7615356b78241fdb159c091830ed0380 Author: Eduard Zingerman eddyz87@gmail.com Date: Wed Feb 15 01:20:27 2023 +0200
bpf: track immediate values written to stack by BPF_ST instruction """
This commit is causing the following bpf:test_verifier kselftest to fail:
""" # #760/p precise: ST insn causing spi > allocated_stack FAIL """
Since this test didn't fail before ecdf985d76 backport, the question is if this is a test bug or if this commit introduced a regression.
I haven't checked if this failure is present in latest Linus tree because I was unable to build & run the bpf kselftests in an older distro.
Also, there some important details about running the bpf kselftests in 5.10 and 5.15:
* On 5.10, bpf kselftest build is broken. The following upstream commit needs to be cherry-picked for it to build & run:
""" commit 4237e9f4a96228ccc8a7abe5e4b30834323cd353 Author: Gilad Reti gilad.reti@gmail.com Date: Wed Jan 13 07:38:08 2021 +0200
selftests/bpf: Add verifier test for PTR_TO_MEM spill """
* On 5.15.120 there's one additional test that's failing, but I didn't debug this one:
""" #150/p calls: trigger reg2btf_ids[reg→type] for reg→type > __BPF_REG_TYPE_MAX FAIL FAIL """
* On 5.11 onwards, building and running bpf tests is disabled by default by commit 7a6eb7c34a78498742b5f82543b7a68c1c443329, so I wonder if we want to backport this to 5.10 as well?
Thanks!
- Luiz
On Mon, 2023-07-17 at 09:04 -0400, Luiz Capitulino wrote:
Hi,
The upstream commit below is backported to 5.10.186, 5.15.120 and 6.1.36:
""" commit ecdf985d7615356b78241fdb159c091830ed0380 Author: Eduard Zingerman eddyz87@gmail.com Date: Wed Feb 15 01:20:27 2023 +0200
bpf: track immediate values written to stack by BPF_ST instruction"""
This commit is causing the following bpf:test_verifier kselftest to fail:
""" # #760/p precise: ST insn causing spi > allocated_stack FAIL """
I can reproduce the error on 6.1.36 but don't understand what's causing it yet. The log is suspiciously different from master, will comment later today.
Since this test didn't fail before ecdf985d76 backport, the question is if this is a test bug or if this commit introduced a regression.
I haven't checked if this failure is present in latest Linus tree because I was unable to build & run the bpf kselftests in an older distro.
Also, there some important details about running the bpf kselftests in 5.10 and 5.15:
- On 5.10, bpf kselftest build is broken. The following upstream
commit needs to be cherry-picked for it to build & run:
""" commit 4237e9f4a96228ccc8a7abe5e4b30834323cd353 Author: Gilad Reti gilad.reti@gmail.com Date: Wed Jan 13 07:38:08 2021 +0200
selftests/bpf: Add verifier test for PTR_TO_MEM spill"""
- On 5.15.120 there's one additional test that's failing, but I didn't
debug this one:
""" #150/p calls: trigger reg2btf_ids[reg→type] for reg→type > __BPF_REG_TYPE_MAX FAIL FAIL """
- On 5.11 onwards, building and running bpf tests is disabled by
default by commit 7a6eb7c34a78498742b5f82543b7a68c1c443329, so I wonder if we want to backport this to 5.10 as well?
Thanks!
- Luiz
On 2023-07-17 10:55, Eduard Zingerman wrote:
On Mon, 2023-07-17 at 09:04 -0400, Luiz Capitulino wrote:
Hi,
The upstream commit below is backported to 5.10.186, 5.15.120 and 6.1.36:
""" commit ecdf985d7615356b78241fdb159c091830ed0380 Author: Eduard Zingerman eddyz87@gmail.com Date: Wed Feb 15 01:20:27 2023 +0200
bpf: track immediate values written to stack by BPF_ST instruction"""
This commit is causing the following bpf:test_verifier kselftest to fail:
""" # #760/p precise: ST insn causing spi > allocated_stack FAIL """
I can reproduce the error on 6.1.36 but don't understand what's causing it yet. The log is suspiciously different from master, will comment later today.
Thank you very much for the prompt reply, Eduard.
I'm available for further testing if needed.
- Luiz
Since this test didn't fail before ecdf985d76 backport, the question is if this is a test bug or if this commit introduced a regression.
I haven't checked if this failure is present in latest Linus tree because I was unable to build & run the bpf kselftests in an older distro.
Also, there some important details about running the bpf kselftests in 5.10 and 5.15:
- On 5.10, bpf kselftest build is broken. The following upstream
commit needs to be cherry-picked for it to build & run:
""" commit 4237e9f4a96228ccc8a7abe5e4b30834323cd353 Author: Gilad Reti gilad.reti@gmail.com Date: Wed Jan 13 07:38:08 2021 +0200
selftests/bpf: Add verifier test for PTR_TO_MEM spill"""
- On 5.15.120 there's one additional test that's failing, but I didn't
debug this one:
""" #150/p calls: trigger reg2btf_ids[reg→type] for reg→type > __BPF_REG_TYPE_MAX FAIL FAIL """
- On 5.11 onwards, building and running bpf tests is disabled by
default by commit 7a6eb7c34a78498742b5f82543b7a68c1c443329, so I wonder if we want to backport this to 5.10 as well?
Thanks!
- Luiz
On Mon, 2023-07-17 at 10:59 -0400, Luiz Capitulino wrote:
On 2023-07-17 10:55, Eduard Zingerman wrote:
On Mon, 2023-07-17 at 09:04 -0400, Luiz Capitulino wrote:
Hi,
The upstream commit below is backported to 5.10.186, 5.15.120 and 6.1.36:
""" commit ecdf985d7615356b78241fdb159c091830ed0380 Author: Eduard Zingerman eddyz87@gmail.com Date: Wed Feb 15 01:20:27 2023 +0200
bpf: track immediate values written to stack by BPF_ST instruction"""
This commit is causing the following bpf:test_verifier kselftest to fail:
""" # #760/p precise: ST insn causing spi > allocated_stack FAIL """
I can reproduce the error on 6.1.36 but don't understand what's causing it yet. The log is suspiciously different from master, will comment later today.
Thank you very much for the prompt reply, Eduard.
I'm available for further testing if needed.
A few observations: - this is not a bug, precision tracking works a bit differently; - the patches [1] and [2] (or just [1] but with a minor conflict) are sufficient to make behavior similar to master.
I will provided a more detailed explanation in 1-2hr, sorry.
[1] f63181b6ae79 ("bpf: stop setting precise in current state") [2] 7a830b53c17b ("bpf: aggressively forget precise markings during state checkpointing")
- Luiz
Since this test didn't fail before ecdf985d76 backport, the question is if this is a test bug or if this commit introduced a regression.
I haven't checked if this failure is present in latest Linus tree because I was unable to build & run the bpf kselftests in an older distro.
Also, there some important details about running the bpf kselftests in 5.10 and 5.15:
- On 5.10, bpf kselftest build is broken. The following upstream
commit needs to be cherry-picked for it to build & run:
""" commit 4237e9f4a96228ccc8a7abe5e4b30834323cd353 Author: Gilad Reti gilad.reti@gmail.com Date: Wed Jan 13 07:38:08 2021 +0200
selftests/bpf: Add verifier test for PTR_TO_MEM spill"""
- On 5.15.120 there's one additional test that's failing, but I didn't
debug this one:
""" #150/p calls: trigger reg2btf_ids[reg→type] for reg→type > __BPF_REG_TYPE_MAX FAIL FAIL """
- On 5.11 onwards, building and running bpf tests is disabled by
default by commit 7a6eb7c34a78498742b5f82543b7a68c1c443329, so I wonder if we want to backport this to 5.10 as well?
Thanks!
- Luiz
On Mon, 2023-07-17 at 10:59 -0400, Luiz Capitulino wrote:
On 2023-07-17 10:55, Eduard Zingerman wrote:
On Mon, 2023-07-17 at 09:04 -0400, Luiz Capitulino wrote:
Hi,
The upstream commit below is backported to 5.10.186, 5.15.120 and 6.1.36:
""" commit ecdf985d7615356b78241fdb159c091830ed0380 Author: Eduard Zingerman eddyz87@gmail.com Date: Wed Feb 15 01:20:27 2023 +0200
bpf: track immediate values written to stack by BPF_ST instruction"""
This commit is causing the following bpf:test_verifier kselftest to fail:
""" # #760/p precise: ST insn causing spi > allocated_stack FAIL """
I can reproduce the error on 6.1.36 but don't understand what's causing it yet. The log is suspiciously different from master, will comment later today.
Thank you very much for the prompt reply, Eduard.
I'm available for further testing if needed.
The test case in question looks as follows:
; old log new log 0: R1=ctx(off=0,imm=0) 0: r3 = r10 ; R3=fp0 R10=fp0 R3=fp0 R10=fp0 1: if r3 != 0x7b goto pc+0 ; R3=fp0 R3=fp0 2: *(u64 *)(r3 -8) = 0 ; R3=fp0 fp-8_w=mmmmmmmm R3=fp0 fp-8_w=00000000 3: r4 = *(u64 *)(r10 -8) ; R4_w=scalar() R10=fp0 R4_w=P0 R10=fp0 fp-8_w=00000000 4: r0 = -1 ; R0=-1 R0=-1 5: if r4 > r0 goto pc+0 6: exit
The test checks the log generated by __mark_chain_precision() at instruction #5: registers `r4` and `r0` should be marked precise (in that order).
The `r4` and `r0` are marked precise by check_cond_jmp_op() function because jump #5 is predicted (because comparison is unsigned `r4` is never greater than -1/`r0`).
Failure happens because only log for `r0` markings is present.
Patch [5] changes processing of instruction #2 ( *(u64 *)(r3 -8) = 0; ): - before patch fp[-8] is marked as mmmmmmmm (any 64-bit scalar value); - after patch fp[-8] is marked as 00000000.
Which in turn changes processing of instruction #3 ( r4 = *(u64 *)(r10 -8); ): - before patch r4 is marked as unbound scalar - after patch r4 is marked as precise zero (check_stack_read_fixed_off() has a special case for zero loads, DST registers of such loads are always marked precise).
If __mark_chain_precision() is called for a register that is already marked as precise no log is generated. Thus the log looks different when check_cond_jmp_op() calls __mark_chain_precision() for `r4` and then for `r0`: - before patch `r4` is not precise and log for it is generated; - after patch `r4` is precise and log for it is not generated.
Now a question: why does current master has log for both `r4` and `r0`, given that commit [5] is present in master?
This happens because of commit [2] (not back-ported to 6.1.36).
Commit [2] adds function mark_all_scalars_imprecise() that removes precision marks from scalar registers when new state is created in a state chain (and for this test new states are created before #1 and #5 because of the BPF_F_TEST_STATE_FREQ flag).
Effectively, before #5 is processed by check_cond_jmp_op() mark_all_scalars_imprecise() undoes `r4`'s precision mark assigned by check_stack_read_fixed_off() => log for both `r4` and `r0` is present.
I need some time to convince myself that such interaction between check_stack_read_fixed_off() and mark_all_scalars_imprecise() is safe, but that's what we have in master.
Commit [2] is a part of a series [4] by Andrii Nakryiko. This series had been partially back-ported already. Commits [0,1,2,3] are not yet back-ported and have to be picked altogether, otherwise there are numerous test failures.
Still, when I cherry-pick [0,1,2,3] `./test_progs -a setget_sockopt` is failing. I'll investigate this failure but don't think I'll finish today.
---
Alternatively, if the goal is to minimize amount of changes, we can disable or modify the 'precise: ST insn causing spi > allocated_stack'.
---
Commits (in chronological order): [0] be2ef8161572 ("bpf: allow precision tracking for programs with subprogs") [1] f63181b6ae79 ("bpf: stop setting precise in current state") [2] 7a830b53c17b ("bpf: aggressively forget precise markings during state checkpointing") [3] 4f999b767769 ("selftests/bpf: make test_align selftest more robust") [4] 07d90c72efbe ("Merge branch 'BPF verifier precision tracking improvements'") [5] ecdf985d7615 ("bpf: track immediate values written to stack by BPF_ST instruction")
- Luiz
Since this test didn't fail before ecdf985d76 backport, the question is if this is a test bug or if this commit introduced a regression.
I haven't checked if this failure is present in latest Linus tree because I was unable to build & run the bpf kselftests in an older distro.
Also, there some important details about running the bpf kselftests in 5.10 and 5.15:
- On 5.10, bpf kselftest build is broken. The following upstream
commit needs to be cherry-picked for it to build & run:
""" commit 4237e9f4a96228ccc8a7abe5e4b30834323cd353 Author: Gilad Reti gilad.reti@gmail.com Date: Wed Jan 13 07:38:08 2021 +0200
selftests/bpf: Add verifier test for PTR_TO_MEM spill"""
- On 5.15.120 there's one additional test that's failing, but I didn't
debug this one:
""" #150/p calls: trigger reg2btf_ids[reg→type] for reg→type > __BPF_REG_TYPE_MAX FAIL FAIL """
- On 5.11 onwards, building and running bpf tests is disabled by
default by commit 7a6eb7c34a78498742b5f82543b7a68c1c443329, so I wonder if we want to backport this to 5.10 as well?
Thanks!
- Luiz
On Tue, 2023-07-18 at 01:57 +0300, Eduard Zingerman wrote:
[...] Still, when I cherry-pick [0,1,2,3] `./test_progs -a setget_sockopt` is failing. I'll investigate this failure but don't think I'll finish today.
Alternatively, if the goal is to minimize amount of changes, we can disable or modify the 'precise: ST insn causing spi > allocated_stack'.
Commits (in chronological order): [0] be2ef8161572 ("bpf: allow precision tracking for programs with subprogs") [1] f63181b6ae79 ("bpf: stop setting precise in current state") [2] 7a830b53c17b ("bpf: aggressively forget precise markings during state checkpointing") [3] 4f999b767769 ("selftests/bpf: make test_align selftest more robust") [4] 07d90c72efbe ("Merge branch 'BPF verifier precision tracking improvements'") [5] ecdf985d7615 ("bpf: track immediate values written to stack by BPF_ST instruction")
I made a mistake, while resolving merge conflict for [0] yesterday. After correction the `./test_progs -a setget_sockopt` passes. I also noted that the following tests fail on v6.1.36:
./test_progs -a sk_assign,fexit_bpf2bpf
These tests are fixed by back-porting the following upstream commits: - 7ce878ca81bc ("selftests/bpf: Fix sk_assign on s390x") - 63d78b7e8ca2 ("selftests/bpf: Workaround verification failure for fexit_bpf2bpf/func_replace_return_code")
I pushed modified version of v6.1.36 to my github account, it has test_verifier, test_progs, test_progs-no_alu32 and test_maps passing (on my x86 setup):
https://github.com/eddyz87/bpf/commits/v6.1.36-with-fixes
Do you need any additional actions from my side?
On Tue, Jul 18, 2023 at 03:31:25PM +0300, Eduard Zingerman wrote:
On Tue, 2023-07-18 at 01:57 +0300, Eduard Zingerman wrote:
[...] Still, when I cherry-pick [0,1,2,3] `./test_progs -a setget_sockopt` is failing. I'll investigate this failure but don't think I'll finish today.
Alternatively, if the goal is to minimize amount of changes, we can disable or modify the 'precise: ST insn causing spi > allocated_stack'.
Commits (in chronological order): [0] be2ef8161572 ("bpf: allow precision tracking for programs with subprogs") [1] f63181b6ae79 ("bpf: stop setting precise in current state") [2] 7a830b53c17b ("bpf: aggressively forget precise markings during state checkpointing") [3] 4f999b767769 ("selftests/bpf: make test_align selftest more robust") [4] 07d90c72efbe ("Merge branch 'BPF verifier precision tracking improvements'") [5] ecdf985d7615 ("bpf: track immediate values written to stack by BPF_ST instruction")
I made a mistake, while resolving merge conflict for [0] yesterday. After correction the `./test_progs -a setget_sockopt` passes. I also noted that the following tests fail on v6.1.36:
./test_progs -a sk_assign,fexit_bpf2bpf
These tests are fixed by back-porting the following upstream commits:
- 7ce878ca81bc ("selftests/bpf: Fix sk_assign on s390x")
- 63d78b7e8ca2 ("selftests/bpf: Workaround verification failure for fexit_bpf2bpf/func_replace_return_code")
I pushed modified version of v6.1.36 to my github account, it has test_verifier, test_progs, test_progs-no_alu32 and test_maps passing (on my x86 setup):
https://github.com/eddyz87/bpf/commits/v6.1.36-with-fixes
Do you need any additional actions from my side?
I don't understand, what can I do with a github link? Can you send us the patches backported so we can apply them to the stable tree?
thanks,
greg k-h
On Tue, 2023-07-18 at 15:23 +0200, Greg KH wrote:
On Tue, Jul 18, 2023 at 03:31:25PM +0300, Eduard Zingerman wrote:
On Tue, 2023-07-18 at 01:57 +0300, Eduard Zingerman wrote:
[...] Still, when I cherry-pick [0,1,2,3] `./test_progs -a setget_sockopt` is failing. I'll investigate this failure but don't think I'll finish today.
Alternatively, if the goal is to minimize amount of changes, we can disable or modify the 'precise: ST insn causing spi > allocated_stack'.
Commits (in chronological order): [0] be2ef8161572 ("bpf: allow precision tracking for programs with subprogs") [1] f63181b6ae79 ("bpf: stop setting precise in current state") [2] 7a830b53c17b ("bpf: aggressively forget precise markings during state checkpointing") [3] 4f999b767769 ("selftests/bpf: make test_align selftest more robust") [4] 07d90c72efbe ("Merge branch 'BPF verifier precision tracking improvements'") [5] ecdf985d7615 ("bpf: track immediate values written to stack by BPF_ST instruction")
I made a mistake, while resolving merge conflict for [0] yesterday. After correction the `./test_progs -a setget_sockopt` passes. I also noted that the following tests fail on v6.1.36:
./test_progs -a sk_assign,fexit_bpf2bpf
These tests are fixed by back-porting the following upstream commits:
- 7ce878ca81bc ("selftests/bpf: Fix sk_assign on s390x")
- 63d78b7e8ca2 ("selftests/bpf: Workaround verification failure for fexit_bpf2bpf/func_replace_return_code")
I pushed modified version of v6.1.36 to my github account, it has test_verifier, test_progs, test_progs-no_alu32 and test_maps passing (on my x86 setup):
https://github.com/eddyz87/bpf/commits/v6.1.36-with-fixes
Do you need any additional actions from my side?
I don't understand, what can I do with a github link? Can you send us the patches backported so we can apply them to the stable tree?
Sorry, I'm not familiar with procedure for stable tree patches or who decides what's being picked. Looks like this situation is "Option 3" from [1], rigth? After reading that page I'm not sure: - can I bundle all the necessary commits as a patch-set? - a few commits need merging, others could be cherry-picked, is it possible to submit all of them with [ Upstream commit ... ] marks?
Also, as I wrote above, there are two possible solutions: - backport above mentioned patches - adjust the test log
[1] https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html
thanks,
greg k-h
On 2023-07-18 09:52, Eduard Zingerman wrote:
On Tue, 2023-07-18 at 15:23 +0200, Greg KH wrote:
On Tue, Jul 18, 2023 at 03:31:25PM +0300, Eduard Zingerman wrote:
On Tue, 2023-07-18 at 01:57 +0300, Eduard Zingerman wrote:
[...] Still, when I cherry-pick [0,1,2,3] `./test_progs -a setget_sockopt` is failing. I'll investigate this failure but don't think I'll finish today.
Alternatively, if the goal is to minimize amount of changes, we can disable or modify the 'precise: ST insn causing spi > allocated_stack'.
Commits (in chronological order): [0] be2ef8161572 ("bpf: allow precision tracking for programs with subprogs") [1] f63181b6ae79 ("bpf: stop setting precise in current state") [2] 7a830b53c17b ("bpf: aggressively forget precise markings during state checkpointing") [3] 4f999b767769 ("selftests/bpf: make test_align selftest more robust") [4] 07d90c72efbe ("Merge branch 'BPF verifier precision tracking improvements'") [5] ecdf985d7615 ("bpf: track immediate values written to stack by BPF_ST instruction")
I made a mistake, while resolving merge conflict for [0] yesterday. After correction the `./test_progs -a setget_sockopt` passes. I also noted that the following tests fail on v6.1.36:
./test_progs -a sk_assign,fexit_bpf2bpf
These tests are fixed by back-porting the following upstream commits:
- 7ce878ca81bc ("selftests/bpf: Fix sk_assign on s390x")
- 63d78b7e8ca2 ("selftests/bpf: Workaround verification failure for fexit_bpf2bpf/func_replace_return_code")
I pushed modified version of v6.1.36 to my github account, it has test_verifier, test_progs, test_progs-no_alu32 and test_maps passing (on my x86 setup):
https://github.com/eddyz87/bpf/commits/v6.1.36-with-fixes
Do you need any additional actions from my side?
I don't understand, what can I do with a github link? Can you send us the patches backported so we can apply them to the stable tree?
Sorry, I'm not familiar with procedure for stable tree patches or who decides what's being picked.
I'm by no means an authority here, but I'll try to help with what I would do myself.
Looks like this situation is "Option 3" from [1], rigth?
Right.
After reading that page I'm not sure:
- can I bundle all the necessary commits as a patch-set?
Yes.
- a few commits need merging, others could be cherry-picked, is it possible to submit all of them with [ Upstream commit ... ] marks?
Yes.
Also, as I wrote above, there are two possible solutions:
- backport above mentioned patches
- adjust the test log
I think we want to avoid deviating from upstream (Linus tree), but I'm not sure if there are valid exceptions.
- Luiz
[1] https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html
thanks,
greg k-h
On Tue, Jul 18, 2023 at 10:58:45AM -0400, Luiz Capitulino wrote:
On 2023-07-18 09:52, Eduard Zingerman wrote:
On Tue, 2023-07-18 at 15:23 +0200, Greg KH wrote:
On Tue, Jul 18, 2023 at 03:31:25PM +0300, Eduard Zingerman wrote:
On Tue, 2023-07-18 at 01:57 +0300, Eduard Zingerman wrote:
[...] Still, when I cherry-pick [0,1,2,3] `./test_progs -a setget_sockopt` is failing. I'll investigate this failure but don't think I'll finish today.
Alternatively, if the goal is to minimize amount of changes, we can disable or modify the 'precise: ST insn causing spi > allocated_stack'.
Commits (in chronological order): [0] be2ef8161572 ("bpf: allow precision tracking for programs with subprogs") [1] f63181b6ae79 ("bpf: stop setting precise in current state") [2] 7a830b53c17b ("bpf: aggressively forget precise markings during state checkpointing") [3] 4f999b767769 ("selftests/bpf: make test_align selftest more robust") [4] 07d90c72efbe ("Merge branch 'BPF verifier precision tracking improvements'") [5] ecdf985d7615 ("bpf: track immediate values written to stack by BPF_ST instruction")
I made a mistake, while resolving merge conflict for [0] yesterday. After correction the `./test_progs -a setget_sockopt` passes. I also noted that the following tests fail on v6.1.36:
./test_progs -a sk_assign,fexit_bpf2bpf
These tests are fixed by back-porting the following upstream commits:
- 7ce878ca81bc ("selftests/bpf: Fix sk_assign on s390x")
- 63d78b7e8ca2 ("selftests/bpf: Workaround verification failure for fexit_bpf2bpf/func_replace_return_code")
I pushed modified version of v6.1.36 to my github account, it has test_verifier, test_progs, test_progs-no_alu32 and test_maps passing (on my x86 setup):
https://github.com/eddyz87/bpf/commits/v6.1.36-with-fixes
Do you need any additional actions from my side?
I don't understand, what can I do with a github link? Can you send us the patches backported so we can apply them to the stable tree?
Sorry, I'm not familiar with procedure for stable tree patches or who decides what's being picked.
I'm by no means an authority here, but I'll try to help with what I would do myself.
Looks like this situation is "Option 3" from [1], rigth?
Right.
After reading that page I'm not sure:
- can I bundle all the necessary commits as a patch-set?
Yes.
- a few commits need merging, others could be cherry-picked, is it possible to submit all of them with [ Upstream commit ... ] marks?
Yes.
Also, as I wrote above, there are two possible solutions:
- backport above mentioned patches
- adjust the test log
I think we want to avoid deviating from upstream (Linus tree), but I'm not sure if there are valid exceptions.
backporting the mentioned patches is best, can someone send them to us in email so that we can apply them?
thanks,
greg k-h
On Fri, 2023-07-21 at 07:30 +0200, Greg KH wrote: [...]
backporting the mentioned patches is best, can someone send them to us in email so that we can apply them?
I'm investigating 5.15 test failures at the moment. Will send patches for linux-6.1.y by the end of the day.
Thanks, Eduard
On 2023-07-18 08:31, Eduard Zingerman wrote:
On Tue, 2023-07-18 at 01:57 +0300, Eduard Zingerman wrote:
[...] Still, when I cherry-pick [0,1,2,3] `./test_progs -a setget_sockopt` is failing. I'll investigate this failure but don't think I'll finish today.
Alternatively, if the goal is to minimize amount of changes, we can disable or modify the 'precise: ST insn causing spi > allocated_stack'.
Commits (in chronological order): [0] be2ef8161572 ("bpf: allow precision tracking for programs with subprogs") [1] f63181b6ae79 ("bpf: stop setting precise in current state") [2] 7a830b53c17b ("bpf: aggressively forget precise markings during state checkpointing") [3] 4f999b767769 ("selftests/bpf: make test_align selftest more robust") [4] 07d90c72efbe ("Merge branch 'BPF verifier precision tracking improvements'") [5] ecdf985d7615 ("bpf: track immediate values written to stack by BPF_ST instruction")
I made a mistake, while resolving merge conflict for [0] yesterday. After correction the `./test_progs -a setget_sockopt` passes. I also noted that the following tests fail on v6.1.36:
./test_progs -a sk_assign,fexit_bpf2bpf
These tests are fixed by back-porting the following upstream commits:
- 7ce878ca81bc ("selftests/bpf: Fix sk_assign on s390x")
- 63d78b7e8ca2 ("selftests/bpf: Workaround verification failure for fexit_bpf2bpf/func_replace_return_code")
I pushed modified version of v6.1.36 to my github account, it has test_verifier, test_progs, test_progs-no_alu32 and test_maps passing (on my x86 setup):
https://github.com/eddyz87/bpf/commits/v6.1.36-with-fixes
Do you need any additional actions from my side?
First, thank you very much for your work on this and getting the tests passing on 6.1.
In terms of action items, have you checked this situation in 5.10 and 5.15? For 5.10, we also need 4237e9f4a96228ccc8a7abe5e4b30834323cd353 otherwise the bpf tests don't even build there.
Also, would you know if something important is broken for users or is this just a small behavior difference between kernels?
- Luiz
On Tue, 2023-07-18 at 10:06 -0400, Luiz Capitulino wrote:
On 2023-07-18 08:31, Eduard Zingerman wrote:
On Tue, 2023-07-18 at 01:57 +0300, Eduard Zingerman wrote:
[...] Still, when I cherry-pick [0,1,2,3] `./test_progs -a setget_sockopt` is failing. I'll investigate this failure but don't think I'll finish today.
Alternatively, if the goal is to minimize amount of changes, we can disable or modify the 'precise: ST insn causing spi > allocated_stack'.
Commits (in chronological order): [0] be2ef8161572 ("bpf: allow precision tracking for programs with subprogs") [1] f63181b6ae79 ("bpf: stop setting precise in current state") [2] 7a830b53c17b ("bpf: aggressively forget precise markings during state checkpointing") [3] 4f999b767769 ("selftests/bpf: make test_align selftest more robust") [4] 07d90c72efbe ("Merge branch 'BPF verifier precision tracking improvements'") [5] ecdf985d7615 ("bpf: track immediate values written to stack by BPF_ST instruction")
I made a mistake, while resolving merge conflict for [0] yesterday. After correction the `./test_progs -a setget_sockopt` passes. I also noted that the following tests fail on v6.1.36:
./test_progs -a sk_assign,fexit_bpf2bpf
These tests are fixed by back-porting the following upstream commits:
- 7ce878ca81bc ("selftests/bpf: Fix sk_assign on s390x")
- 63d78b7e8ca2 ("selftests/bpf: Workaround verification failure for fexit_bpf2bpf/func_replace_return_code")
I pushed modified version of v6.1.36 to my github account, it has test_verifier, test_progs, test_progs-no_alu32 and test_maps passing (on my x86 setup):
https://github.com/eddyz87/bpf/commits/v6.1.36-with-fixes
Do you need any additional actions from my side?
First, thank you very much for your work on this and getting the tests passing on 6.1.
Thank you.
In terms of action items, have you checked this situation in 5.10 and 5.15? For 5.10, we also need 4237e9f4a96228ccc8a7abe5e4b30834323cd353 otherwise the bpf tests don't even build there.
Haven't checked 5.15/5.10, will take a look. Are there any time-frame limitations? (I'd like to work on this on Wednesday or Thursday)
Also, would you know if something important is broken for users or is this just a small behavior difference between kernels?
I think it's more like small behavior difference: - be2ef8161572, f63181b6ae79, 7a830b53c17b are verification scalability optimizations, with these patches it is possible to load a bit more complex programs (larger programs, or more complex branching patterns). - 4f999b767769, 7ce878ca81bc, 63d78b7e8ca2 - fixes for selftests, no new functionality.
Thanks, Eduard
On 2023-07-18 10:35, Eduard Zingerman wrote:
On Tue, 2023-07-18 at 10:06 -0400, Luiz Capitulino wrote:
On 2023-07-18 08:31, Eduard Zingerman wrote:
On Tue, 2023-07-18 at 01:57 +0300, Eduard Zingerman wrote:
[...] Still, when I cherry-pick [0,1,2,3] `./test_progs -a setget_sockopt` is failing. I'll investigate this failure but don't think I'll finish today.
Alternatively, if the goal is to minimize amount of changes, we can disable or modify the 'precise: ST insn causing spi > allocated_stack'.
Commits (in chronological order): [0] be2ef8161572 ("bpf: allow precision tracking for programs with subprogs") [1] f63181b6ae79 ("bpf: stop setting precise in current state") [2] 7a830b53c17b ("bpf: aggressively forget precise markings during state checkpointing") [3] 4f999b767769 ("selftests/bpf: make test_align selftest more robust") [4] 07d90c72efbe ("Merge branch 'BPF verifier precision tracking improvements'") [5] ecdf985d7615 ("bpf: track immediate values written to stack by BPF_ST instruction")
I made a mistake, while resolving merge conflict for [0] yesterday. After correction the `./test_progs -a setget_sockopt` passes. I also noted that the following tests fail on v6.1.36:
./test_progs -a sk_assign,fexit_bpf2bpfThese tests are fixed by back-porting the following upstream commits:
- 7ce878ca81bc ("selftests/bpf: Fix sk_assign on s390x")
- 63d78b7e8ca2 ("selftests/bpf: Workaround verification failure for fexit_bpf2bpf/func_replace_return_code")
I pushed modified version of v6.1.36 to my github account, it has test_verifier, test_progs, test_progs-no_alu32 and test_maps passing (on my x86 setup):
https://github.com/eddyz87/bpf/commits/v6.1.36-with-fixesDo you need any additional actions from my side?
First, thank you very much for your work on this and getting the tests passing on 6.1.
Thank you.
In terms of action items, have you checked this situation in 5.10 and 5.15? For 5.10, we also need 4237e9f4a96228ccc8a7abe5e4b30834323cd353 otherwise the bpf tests don't even build there.
Haven't checked 5.15/5.10, will take a look. Are there any time-frame limitations? (I'd like to work on this on Wednesday or Thursday)
Since, as you explain below, this is not a impactful regression to stable users, I think that's fine.
Also, would you know if something important is broken for users or is this just a small behavior difference between kernels?
I think it's more like small behavior difference:
- be2ef8161572, f63181b6ae79, 7a830b53c17b are verification scalability optimizations, with these patches it is possible to load a bit more complex programs (larger programs, or more complex branching patterns).
- 4f999b767769, 7ce878ca81bc, 63d78b7e8ca2 - fixes for selftests, no new functionality.
Thanks for clarifying it!
- Luiz
Thanks, Eduard
linux-stable-mirror@lists.linaro.org
-
Eduard Zingerman -
Greg KH -
Luiz Capitulino