This patch fixes an encoding bug in emit_stx for BPF_B when the source register is BPF_REG_FP.
The current implementation for BPF_STX BPF_B in emit_stx saves one REX byte when the operands can be encoded using Mod-R/M alone. The lower 8 bits of registers %rax, %rbx, %rcx, and %rdx can be accessed without using a REX prefix via %al, %bl, %cl, and %dl, respectively. Other registers, (e.g., %rsi, %rdi, %rbp, %rsp) require a REX prefix to use their 8-bit equivalents (%sil, %dil, %bpl, %spl).
The current code checks if the source for BPF_STX BPF_B is BPF_REG_1 or BPF_REG_2 (which map to %rdi and %rsi), in which case it emits the required REX prefix. However, it misses the case when the source is BPF_REG_FP (mapped to %rbp).
The result is that BPF_STX BPF_B with BPF_REG_FP as the source operand will read from register %ch instead of the correct %bpl. This patch fixes the problem by fixing and refactoring the check on which registers need the extra REX byte. Since no BPF registers map to %rsp, there is no need to handle %spl.
Fixes: 622582786c9e0 ("net: filter: x86: internal BPF JIT") Signed-off-by: Xi Wang xi.wang@gmail.com Signed-off-by: Luke Nelson luke.r.nels@gmail.com --- arch/x86/net/bpf_jit_comp.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index 5ea7c2cf7ab4..42b6709e6dc7 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -158,6 +158,19 @@ static bool is_ereg(u32 reg) BIT(BPF_REG_AX)); }
+/* + * is_ereg_8l() == true if BPF register 'reg' is mapped to access x86-64 + * lower 8-bit registers dil,sil,bpl,spl,r8b..r15b, which need extra byte + * of encoding. al,cl,dl,bl have simpler encoding. + */ +static bool is_ereg_8l(u32 reg) +{ + return is_ereg(reg) || + (1 << reg) & (BIT(BPF_REG_1) | + BIT(BPF_REG_2) | + BIT(BPF_REG_FP)); +} + static bool is_axreg(u32 reg) { return reg == BPF_REG_0; @@ -598,9 +611,8 @@ static void emit_stx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off) switch (size) { case BPF_B: /* Emit 'mov byte ptr [rax + off], al' */ - if (is_ereg(dst_reg) || is_ereg(src_reg) || - /* We have to add extra byte for x86 SIL, DIL regs */ - src_reg == BPF_REG_1 || src_reg == BPF_REG_2) + if (is_ereg(dst_reg) || is_ereg_8l(src_reg)) + /* Add extra byte for eregs or SIL,DIL,BPL in src_reg */ EMIT2(add_2mod(0x40, dst_reg, src_reg), 0x88); else EMIT1(0x88);
This patch adds a test to test_verifier that writes the lower 8 bits of R10 (aka FP) using BPF_B to an array map and reads the result back. The expected behavior is that the result should be the same as first copying R10 to R9, and then storing / loading the lower 8 bits of R9.
This test catches a bug that was present in the x86-64 JIT that caused an incorrect encoding for BPF_STX BPF_B when the source operand is R10.
Signed-off-by: Xi Wang xi.wang@gmail.com Signed-off-by: Luke Nelson luke.r.nels@gmail.com --- .../selftests/bpf/verifier/stack_ptr.c | 40 +++++++++++++++++++ 1 file changed, 40 insertions(+)
diff --git a/tools/testing/selftests/bpf/verifier/stack_ptr.c b/tools/testing/selftests/bpf/verifier/stack_ptr.c index 7276620ef242..8bfeb77c60bd 100644 --- a/tools/testing/selftests/bpf/verifier/stack_ptr.c +++ b/tools/testing/selftests/bpf/verifier/stack_ptr.c @@ -315,3 +315,43 @@ }, .result = ACCEPT, }, +{ + "store PTR_TO_STACK in R10 to array map using BPF_B", + .insns = { + /* Load pointer to map. */ + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8), + BPF_ST_MEM(BPF_DW, BPF_REG_2, 0, 0), + BPF_LD_MAP_FD(BPF_REG_1, 0), + BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 2), + BPF_MOV64_IMM(BPF_REG_0, 2), + BPF_EXIT_INSN(), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_0), + /* Copy R10 to R9. */ + BPF_MOV64_REG(BPF_REG_9, BPF_REG_10), + /* Pollute other registers with unaligned values. */ + BPF_MOV64_IMM(BPF_REG_2, -1), + BPF_MOV64_IMM(BPF_REG_3, -1), + BPF_MOV64_IMM(BPF_REG_4, -1), + BPF_MOV64_IMM(BPF_REG_5, -1), + BPF_MOV64_IMM(BPF_REG_6, -1), + BPF_MOV64_IMM(BPF_REG_7, -1), + BPF_MOV64_IMM(BPF_REG_8, -1), + /* Store both R9 and R10 with BPF_B and read back. */ + BPF_STX_MEM(BPF_B, BPF_REG_1, BPF_REG_10, 0), + BPF_LDX_MEM(BPF_B, BPF_REG_2, BPF_REG_1, 0), + BPF_STX_MEM(BPF_B, BPF_REG_1, BPF_REG_9, 0), + BPF_LDX_MEM(BPF_B, BPF_REG_3, BPF_REG_1, 0), + /* Should read back as same value. */ + BPF_JMP_REG(BPF_JEQ, BPF_REG_2, BPF_REG_3, 2), + BPF_MOV64_IMM(BPF_REG_0, 1), + BPF_EXIT_INSN(), + BPF_MOV64_IMM(BPF_REG_0, 42), + BPF_EXIT_INSN(), + }, + .fixup_map_array_48b = { 3 }, + .result = ACCEPT, + .retval = 42, + .prog_type = BPF_PROG_TYPE_SCHED_CLS, +},
On Sat, Apr 18, 2020 at 4:27 PM Luke Nelson lukenels@cs.washington.edu wrote:
This patch fixes an encoding bug in emit_stx for BPF_B when the source register is BPF_REG_FP.
The current implementation for BPF_STX BPF_B in emit_stx saves one REX byte when the operands can be encoded using Mod-R/M alone. The lower 8 bits of registers %rax, %rbx, %rcx, and %rdx can be accessed without using a REX prefix via %al, %bl, %cl, and %dl, respectively. Other registers, (e.g., %rsi, %rdi, %rbp, %rsp) require a REX prefix to use their 8-bit equivalents (%sil, %dil, %bpl, %spl).
The current code checks if the source for BPF_STX BPF_B is BPF_REG_1 or BPF_REG_2 (which map to %rdi and %rsi), in which case it emits the required REX prefix. However, it misses the case when the source is BPF_REG_FP (mapped to %rbp).
The result is that BPF_STX BPF_B with BPF_REG_FP as the source operand will read from register %ch instead of the correct %bpl. This patch fixes the problem by fixing and refactoring the check on which registers need the extra REX byte. Since no BPF registers map to %rsp, there is no need to handle %spl.
Fixes: 622582786c9e0 ("net: filter: x86: internal BPF JIT") Signed-off-by: Xi Wang xi.wang@gmail.com Signed-off-by: Luke Nelson luke.r.nels@gmail.com
Applied. Thanks for the fix. It's questionable whether the verifier should have allowed such insn in the first place, but JIT fix is good regardless.
linux-kselftest-mirror@lists.linaro.org