Hello all,
This patch series targets a long-standing BPF usability issue - the lack of general cross-compilation support - by enabling cross-endian usage of libbpf and bpftool, as well as supporting cross-endian build targets for selftests/bpf.
Benefits include improved BPF development and testing for embedded systems based on e.g. big-endian MIPS, more build options e.g for s390x systems, and better accessibility to the very latest test tools e.g. 'test_progs'.
The series touches many functional areas: BTF.ext handling; object access, introspection, and linking; generation of normal and "light" skeletons.
Initial development and testing used mips64, since this arch makes switching the build byte-order trivial and is thus very handy for A/B testing. However, it lacks some key features (bpf2bpf call, kfuncs, etc) making for poor selftests/bpf coverage.
Final testing takes the kernel and selftests/bpf cross-built from x86_64 to s390x, and runs the result under QEMU/s390x. That same configuration could also be used on kernel-patches/bpf CI for regression testing endian support or perhaps load-sharing s390x builds across x86_64 systems.
This thread includes some background regarding testing on QEMU/s390x and the generally favourable results: https://lore.kernel.org/bpf/ZsEcsaa3juxxQBUf@kodidev-ubuntu/
Earlier versions and related discussion of the series are here:
v1: https://lore.kernel.org/bpf/cover.1724216108.git.tony.ambardar@gmail.com/ v2: https://lore.kernel.org/bpf/cover.1724313164.git.tony.ambardar@gmail.com/ v3: https://lore.kernel.org/bpf/cover.1724843049.git.tony.ambardar@gmail.com/ v4: https://lore.kernel.org/bpf/cover.1724976539.git.tony.ambardar@gmail.com/ v5: https://lore.kernel.org/bpf/cover.1725347944.git.tony.ambardar@gmail.com/
Feedback and suggestions are welcome!
Best regards, Tony
Changelog: --------- v5 -> v6: (comments from Andrii, Alexei, Eduard) - clarify info_blob_bswap() by making it explicitly conditional on non-native target endianness, and merge a pair of related debug statements - reformat debug statement in bpf_object_bswap_progs() on single line - update existing info setup functions to validate and parse info section metadata prior to any byte-swapping, and drop earlier added validation checks - rework cross-endian BTF.ext handling by using callback functions to byte-swap different types of info records, but after initial parsing - fix a bug always outputting BTF.ext raw data in native endianness - include v5 "Acked-by:" from Alexei, Yonghong
v4 -> v5: (feedback from Andrii and Eduard) - add separate functions to byte-swap info metadata and records, and ensure ordering so record bswaps occur when metadata is native endian - use new and existing macros to iterate through info sections/records, and check embedded record sizes match that of info structs used - drop use of <cough> evil callbacks - move setting swapped_endian flag to after byte-swapping functions are called during initialization, allowing funcs to infer endianness and drop a 'bool native' call parameter - simplify byte-swapping macro used to generate light skeleton, and use internal lib funcs to swap info records instead of assuming all __u32 - change info bswap library funcs to void return - rework/consolidate new debug statements to reduce their number - remove some unneeded handling of impossible errors, and drop a safety check already handled elsewhere - add and clarify some comments
v3 -> v4: - fix a use-after-free ELF data-handling error causing rare CI failures - move bswap functions for func/line/core-relo records to internal header - use bswap functions also for info blobs in light skeleton
v2 -> v3: (feedback from Andrii) - improve some log and commit message formatting - restructure BTF.ext endianness safety checks and byte-swapping - use BTF.ext info record definitions for swapping, require BTF v1 - follow BTF API implementation more closely for BTF.ext - explicitly reject loading non-native endianness program into kernel - simplify linker output byte-order setting - drop redundant safety checks during linking - simplify endianness macro and improve blob setup code for light skel - no unexpected test failures after cross-compiling x86_64 -> s390x
v1 -> v2: - fixed a light skeleton bug causing test_progs 'map_ptr' failure - simplified some BTF.ext related endianness logic - remove an 'inline' usage related to CI checkpatch failure - improve some formatting noted by checkpatch warnings - unexpected 'test_progs' failures drop 3 -> 2 (x86_64 to s390x cross)
Tony Ambardar (8): libbpf: Improve log message formatting libbpf: Fix header comment typos for BTF.ext libbpf: Fix output .symtab byte-order during linking libbpf: Support BTF.ext loading and output in either endianness libbpf: Support opening bpf objects of either endianness libbpf: Support linking bpf objects of either endianness libbpf: Support creating light skeleton of either endianness selftests/bpf: Support cross-endian building
tools/lib/bpf/bpf_gen_internal.h | 1 + tools/lib/bpf/btf.c | 284 +++++++++++++++++++++------ tools/lib/bpf/btf.h | 3 + tools/lib/bpf/btf_dump.c | 2 +- tools/lib/bpf/btf_relocate.c | 2 +- tools/lib/bpf/gen_loader.c | 187 +++++++++++++----- tools/lib/bpf/libbpf.c | 56 ++++-- tools/lib/bpf/libbpf.map | 2 + tools/lib/bpf/libbpf_internal.h | 45 ++++- tools/lib/bpf/linker.c | 80 ++++++-- tools/lib/bpf/relo_core.c | 2 +- tools/lib/bpf/skel_internal.h | 3 +- tools/testing/selftests/bpf/Makefile | 7 +- 13 files changed, 527 insertions(+), 147 deletions(-)