On Sun, Aug 31, 2025 at 9:17 PM Yonghong Song yonghong.song@linux.dev wrote:
On 8/29/25 10:19 PM, Ian Rogers wrote:
On Thu, Jun 5, 2025 at 2:00 PM patchwork-bot+netdevbpf@kernel.org wrote:
Hello:
This series was applied to bpf/bpf-next.git (master) by Andrii Nakryiko andrii@kernel.org:
On Tue, 3 Jun 2025 13:37:00 -0700 you wrote:
The BTF dumper code currently displays arrays of characters as just that - arrays, with each character formatted individually. Sometimes this is what makes sense, but it's nice to be able to treat that array as a string.
This change adds a special case to the btf_dump functionality to allow 0-terminated arrays of single-byte integer values to be printed as character strings. Characters for which isprint() returns false are printed as hex-escaped values. This is enabled when the new ".emit_strings" is set to 1 in the btf_dump_type_data_opts structure.
[...]
Here is the summary with links:
- [v3,1/2] libbpf: add support for printing BTF character arrays as strings https://git.kernel.org/bpf/bpf-next/c/87c9c79a02b4
- [v3,2/2] Tests for the ".emit_strings" functionality in the BTF dumper. https://git.kernel.org/bpf/bpf-next/c/a570f386f3d1
You are awesome, thank you!
I believe this patch is responsible for segvs occurring in v6.17 in various perf tests when the perf tests run in parallel. There's lots
Could you share the command line to reproduce this failure? This will help debugging. Thanks!
Sure: ``` $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git ... $ cd linux $ mkdir /tmp/perf $ make -C tools/perf O=/tmp/perf make: Entering directory 'linux/tools/perf' BUILD: Doing 'make -j28' parallel build Warning: Kernel ABI header differences: diff -u tools/arch/arm64/include/asm/cputype.h arch/arm64/include/asm/cputype.h
Auto-detecting system features: ... libdw: [ on ] ... glibc: [ on ] ... libelf: [ on ] ... libnuma: [ on ] ... numa_num_possible_cpus: [ on ] ... libperl: [ on ] ... libpython: [ on ] ... libcapstone: [ on ] ... llvm-perf: [ on ] ... zlib: [ on ] ... lzma: [ on ] ... get_cpuid: [ on ] ... bpf: [ on ] ... libaio: [ on ] ... libzstd: [ on ]
GEN /tmp/perf/common-cmds.h ... LINK /tmp/perf/perf GEN /tmp/perf/python/perf.cpython-313-x86_64-linux-gnu.so make: Leaving directory 'linux/tools/perf' $ git describe v6.17-rc4 $ sudo /tmp/perf/perf test -v 1: vmlinux symtab matches kallsyms : Skip 2: Detect openat syscall event : Ok ... 79: build id cache operations : Ok --- start --- test child forked, pid 2546565 test daemon list FAILED: wrong name FAILED: wrong run FAILED: wrong base FAILED: wrong output FAILED: wrong control FAILED: wrong ack test daemon reconfig FAILED: Timeout waiting for daemon to ping ---- end(-1) ---- 80: daemon operations : FAILED! 81: perf diff tests : Ok 82: DRM PMU : Ok 83: perf ftrace tests : Ok --- start --- test child forked, pid 2546573 Test perf header file perf: Segmentation fault linux/tools/perf/tests/shell/header.sh: line 51: 2546664 Segmentation fault (core dumped ) perf record -o "${perfdata}" -- perf test -w noploop Unexpected signal in test_file ---- end(-1) ---- 84: perf header tests : FAILED! 85: perf list tests : Ok ... 92: perf sched tests : Ok --- start --- test child forked, pid 2546593 DB test perf: Segmentation fault /tmp/x/linux/tools/perf/tests/shell/script.sh: line 35: 2546822 Segmentation fault (core dumped ) perf record $cmd_flags -o "${perfdatafile}" true --- Cleaning up --- ---- end(-1) ---- 93: perf script tests : FAILED! ... $ /tmp/perf/perf version --build-options perf version 6.17.rc4.gb320789d6883 aio: [ on ] # HAVE_AIO_SUPPORT bpf: [ on ] # HAVE_LIBBPF_SUPPORT bpf_skeletons: [ on ] # HAVE_BPF_SKEL debuginfod: [ on ] # HAVE_DEBUGINFOD_SUPPORT dwarf: [ on ] # HAVE_LIBDW_SUPPORT dwarf_getlocations: [ on ] # HAVE_LIBDW_SUPPORT dwarf-unwind: [ on ] # HAVE_DWARF_UNWIND_SUPPORT auxtrace: [ on ] # HAVE_AUXTRACE_SUPPORT libbfd: [ OFF ] # HAVE_LIBBFD_SUPPORT ( tip: Deprecated, license incompatibility, u se BUILD_NONDISTRO=1 and install binutils-dev[el] ) libbpf-strings: [ on ] # HAVE_LIBBPF_STRINGS_SUPPORT libcapstone: [ on ] # HAVE_LIBCAPSTONE_SUPPORT libdw-dwarf-unwind: [ on ] # HAVE_LIBDW_SUPPORT libelf: [ on ] # HAVE_LIBELF_SUPPORT libnuma: [ on ] # HAVE_LIBNUMA_SUPPORT libopencsd: [ OFF ] # HAVE_CSTRACE_SUPPORT libperl: [ on ] # HAVE_LIBPERL_SUPPORT libpfm4: [ on ] # HAVE_LIBPFM libpython: [ on ] # HAVE_LIBPYTHON_SUPPORT libslang: [ on ] # HAVE_SLANG_SUPPORT libtraceevent: [ on ] # HAVE_LIBTRACEEVENT libunwind: [ OFF ] # HAVE_LIBUNWIND_SUPPORT ( tip: Deprecated, use LIBUNWIND=1 and ins tall libunwind-dev[el] to build with it ) lzma: [ on ] # HAVE_LZMA_SUPPORT numa_num_possible_cpus: [ on ] # HAVE_LIBNUMA_SUPPORT zlib: [ on ] # HAVE_ZLIB_SUPPORT zstd: [ on ] # HAVE_ZSTD_SUPPORT ```
Thanks, Ian
of BPF things happening in parallel in the test but the failures are happening in a shell and I did get to attach a debugger. I've not seen this problem earlier as the patches weren't in the perf-tools-next tree. Through bisection I was able to blame the patch and I came up with this minimal fix:
diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h index ccfd905f03df..71e198b30c5f 100644 --- a/tools/lib/bpf/btf.h +++ b/tools/lib/bpf/btf.h @@ -326,10 +326,10 @@ struct btf_dump_type_data_opts { bool compact; /* no newlines/indentation */ bool skip_names; /* skip member/type names */ bool emit_zeroes; /* show 0-valued fields */ - bool emit_strings; /* print char arrays as strings */ + //bool emit_strings; /* print char arrays as strings */ size_t :0; }; -#define btf_dump_type_data_opts__last_field emit_strings +#define btf_dump_type_data_opts__last_field emit_zeroes LIBBPF_API int btf_dump__dump_type_data(struct btf_dump *d, __u32 id, diff --git a/tools/lib/bpf/btf_dump.c b/tools/lib/bpf/btf_dump.c index f09f25eccf3c..c7b5a376642f 100644 --- a/tools/lib/bpf/btf_dump.c +++ b/tools/lib/bpf/btf_dump.c @@ -2599,7 +2599,7 @@ int btf_dump__dump_type_data(struct btf_dump *d, __u32 id, d->typed_dump->compact = OPTS_GET(opts, compact, false); d->typed_dump->skip_names = OPTS_GET(opts, skip_names, false); d->typed_dump->emit_zeroes = OPTS_GET(opts, emit_zeroes, false); - d->typed_dump->emit_strings = OPTS_GET(opts, emit_strings, false); + d->typed_dump->emit_strings = true; // OPTS_GET(opts, emit_strings, false); ret = btf_dump_dump_type_data(d, NULL, t, id, data, 0, 0);So I think the problem relates to modifying struct btf_dump_type_data_opts. Given I'm statically linking libbpf into perf I'm not sure on the exact route of the segv, no doubt this report will be enough for someone else to figure it out.
Given this is a regression what should the fix be?
Thanks, Ian