Fix a few issues reported by 0Day/LKP during runing selftests/bpf.
Changelog: V2: - folded previous similar standalone patch to [1/5], and add acked tag from Song Liu - add acked tag to [2/5], [3/5] from Song Liu - [4/5]: move test_bpftool.py to TEST_PROGS_EXTENDED, files in TEST_GEN_PROGS_EXTENDED are generated by make. Otherwise, it will break out-of-tree install: 'make O=/kselftest-build SKIP_TARGETS= V=1 -C tools/testing/selftests install INSTALL_PATH=/kselftest-install' - [5/5]: new patch
Li Zhijian (5): selftests/bpf: enlarge select() timeout for test_maps selftests/bpf: make test_doc_build.sh work from script directory selftests/bpf: add default bpftool built by selftests to PATH selftests/bpf: add missing files required by test_bpftool.sh for installing selftests/bpf: exit with KSFT_SKIP if no Makefile found
tools/testing/selftests/bpf/Makefile | 4 +++- tools/testing/selftests/bpf/test_bpftool.sh | 6 ++++++ tools/testing/selftests/bpf/test_bpftool_build.sh | 2 +- tools/testing/selftests/bpf/test_doc_build.sh | 10 ++++++++-- tools/testing/selftests/bpf/test_maps.c | 2 +- 5 files changed, 19 insertions(+), 5 deletions(-)
0Day robot observed that it's easily timeout on a heavy load host. ------------------- # selftests: bpf: test_maps # Fork 1024 tasks to 'test_update_delete' # Fork 1024 tasks to 'test_update_delete' # Fork 100 tasks to 'test_hashmap' # Fork 100 tasks to 'test_hashmap_percpu' # Fork 100 tasks to 'test_hashmap_sizes' # Fork 100 tasks to 'test_hashmap_walk' # Fork 100 tasks to 'test_arraymap' # Fork 100 tasks to 'test_arraymap_percpu' # Failed sockmap unexpected timeout not ok 3 selftests: bpf: test_maps # exit=1 # selftests: bpf: test_lru_map # nr_cpus:8 ------------------- Since this test will be scheduled by 0Day to a random host that could have only a few cpus(2-8), enlarge the timeout to avoid a false NG report.
In practice, i tried to pin it to only one cpu by 'taskset 0x01 ./test_maps', and knew 10S is likely enough, but i still perfer to a larger value 30.
Reported-by: kernel test robot lkp@intel.com Signed-off-by: Li Zhijian lizhijian@cn.fujitsu.com Acked-by: Song Liu songliubraving@fb.com
--- V2: update to 30 seconds 3S is not enough sometimes on a very busy host taskset 1,1 ./test_maps 9 Fork 1024 tasks to 'test_update_delete' Fork 1024 tasks to 'test_update_delete' Fork 100 tasks to 'test_hashmap' Fork 100 tasks to 'test_hashmap_percpu' Fork 100 tasks to 'test_hashmap_sizes' Fork 100 tasks to 'test_hashmap_walk' Fork 100 tasks to 'test_arraymap' Fork 100 tasks to 'test_arraymap_percpu' Failed sockmap unexpected timeout
taskset 1,1 ./test_maps 10 Fork 1024 tasks to 'test_update_delete' Fork 1024 tasks to 'test_update_delete' Fork 100 tasks to 'test_hashmap' Fork 100 tasks to 'test_hashmap_percpu' Fork 100 tasks to 'test_hashmap_sizes' Fork 100 tasks to 'test_hashmap_walk' Fork 100 tasks to 'test_arraymap' Fork 100 tasks to 'test_arraymap_percpu' Fork 1024 tasks to 'test_update_delete' Fork 1024 tasks to 'test_update_delete' Fork 100 tasks to 'test_hashmap' Fork 100 tasks to 'test_hashmap_percpu' Fork 100 tasks to 'test_hashmap_sizes' Fork 100 tasks to 'test_hashmap_walk' Fork 100 tasks to 'test_arraymap' Fork 100 tasks to 'test_arraymap_percpu' test_array_map_batch_ops:PASS test_array_percpu_map_batch_ops:PASS test_htab_map_batch_ops:PASS test_htab_percpu_map_batch_ops:PASS test_lpm_trie_map_batch_ops:PASS test_sk_storage_map:PASS test_maps: OK, 0 SKIPPED
taskset 0x01 ./test_maps 9 Fork 1024 tasks to 'test_update_delete' Fork 1024 tasks to 'test_update_delete' Fork 100 tasks to 'test_hashmap' Fork 100 tasks to 'test_hashmap_percpu' Fork 100 tasks to 'test_hashmap_sizes' Fork 100 tasks to 'test_hashmap_walk' Fork 100 tasks to 'test_arraymap' Fork 100 tasks to 'test_arraymap_percpu' Fork 1024 tasks to 'test_update_delete' Fork 1024 tasks to 'test_update_delete' Fork 100 tasks to 'test_hashmap' Fork 100 tasks to 'test_hashmap_percpu' Fork 100 tasks to 'test_hashmap_sizes' Fork 100 tasks to 'test_hashmap_walk' Fork 100 tasks to 'test_arraymap' Fork 100 tasks to 'test_arraymap_percpu' test_array_map_batch_ops:PASS test_array_percpu_map_batch_ops:PASS test_htab_map_batch_ops:PASS test_htab_percpu_map_batch_ops:PASS test_lpm_trie_map_batch_ops:PASS test_sk_storage_map:PASS test_maps: OK, 0 SKIPPED
taskset 0x01 ./test_maps 10 Fork 1024 tasks to 'test_update_delete' Fork 1024 tasks to 'test_update_delete' Fork 100 tasks to 'test_hashmap' Fork 100 tasks to 'test_hashmap_percpu' Fork 100 tasks to 'test_hashmap_sizes' Fork 100 tasks to 'test_hashmap_walk' Fork 100 tasks to 'test_arraymap' Fork 100 tasks to 'test_arraymap_percpu' Fork 1024 tasks to 'test_update_delete' Fork 1024 tasks to 'test_update_delete' Fork 100 tasks to 'test_hashmap' Fork 100 tasks to 'test_hashmap_percpu' Fork 100 tasks to 'test_hashmap_sizes' Fork 100 tasks to 'test_hashmap_walk' Fork 100 tasks to 'test_arraymap' Fork 100 tasks to 'test_arraymap_percpu' test_array_map_batch_ops:PASS test_array_percpu_map_batch_ops:PASS test_htab_map_batch_ops:PASS test_htab_percpu_map_batch_ops:PASS test_lpm_trie_map_batch_ops:PASS test_sk_storage_map:PASS test_maps: OK, 0 SKIPPED --- tools/testing/selftests/bpf/test_maps.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c index 30cbf5d98f7d..de58a3070eea 100644 --- a/tools/testing/selftests/bpf/test_maps.c +++ b/tools/testing/selftests/bpf/test_maps.c @@ -985,7 +985,7 @@ static void test_sockmap(unsigned int tasks, void *data)
FD_ZERO(&w); FD_SET(sfd[3], &w); - to.tv_sec = 1; + to.tv_sec = 30; to.tv_usec = 0; s = select(sfd[3] + 1, &w, NULL, NULL, &to); if (s == -1) {
Previously, it fails as below: ------------- root@lkp-skl-d01 /opt/rootfs/v5.14-rc4/tools/testing/selftests/bpf# ./test_doc_build.sh ++ realpath --relative-to=/opt/rootfs/v5.14-rc4/tools/testing/selftests/bpf ./test_doc_build.sh + SCRIPT_REL_PATH=test_doc_build.sh ++ dirname test_doc_build.sh + SCRIPT_REL_DIR=. ++ realpath /opt/rootfs/v5.14-rc4/tools/testing/selftests/bpf/./../../../../ + KDIR_ROOT_DIR=/opt/rootfs/v5.14-rc4 + cd /opt/rootfs/v5.14-rc4 + for tgt in docs docs-clean + make -s -C /opt/rootfs/v5.14-rc4/. docs make: *** No rule to make target 'docs'. Stop. + for tgt in docs docs-clean + make -s -C /opt/rootfs/v5.14-rc4/. docs-clean make: *** No rule to make target 'docs-clean'. Stop. -----------
Reported-by: kernel test robot lkp@intel.com Signed-off-by: Li Zhijian lizhijian@cn.fujitsu.com Acked-by: Song Liu songliubraving@fb.com --- tools/testing/selftests/bpf/test_doc_build.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/bpf/test_doc_build.sh b/tools/testing/selftests/bpf/test_doc_build.sh index ed12111cd2f0..d67ced95a6cf 100755 --- a/tools/testing/selftests/bpf/test_doc_build.sh +++ b/tools/testing/selftests/bpf/test_doc_build.sh @@ -4,9 +4,10 @@ set -e
# Assume script is located under tools/testing/selftests/bpf/. We want to start # build attempts from the top of kernel repository. -SCRIPT_REL_PATH=$(realpath --relative-to=$PWD $0) +SCRIPT_REL_PATH=$(realpath $0) SCRIPT_REL_DIR=$(dirname $SCRIPT_REL_PATH) -KDIR_ROOT_DIR=$(realpath $PWD/$SCRIPT_REL_DIR/../../../../) +KDIR_ROOT_DIR=$(realpath $SCRIPT_REL_DIR/../../../../) +SCRIPT_REL_DIR=$(dirname $(realpath --relative-to=$KDIR_ROOT_DIR $SCRIPT_REL_PATH)) cd $KDIR_ROOT_DIR
for tgt in docs docs-clean; do
For 'make run_tests': selftests will build bpftool into tools/testing/selftests/bpf/tools/sbin/bpftool by default.
================== root@lkp-skl-d01 /opt/rootfs/v5.14-rc4# make -C tools/testing/selftests/bpf run_tests make: Entering directory '/opt/rootfs/v5.14-rc4/tools/testing/selftests/bpf' MKDIR include MKDIR libbpf MKDIR bpftool [...] GEN /opt/rootfs/v5.14-rc4/tools/testing/selftests/bpf/tools/build/bpftool/profiler.skel.h CC /opt/rootfs/v5.14-rc4/tools/testing/selftests/bpf/tools/build/bpftool/prog.o GEN /opt/rootfs/v5.14-rc4/tools/testing/selftests/bpf/tools/build/bpftool/pid_iter.skel.h CC /opt/rootfs/v5.14-rc4/tools/testing/selftests/bpf/tools/build/bpftool/pids.o LINK /opt/rootfs/v5.14-rc4/tools/testing/selftests/bpf/tools/build/bpftool/bpftool INSTALL bpftool GEN vmlinux.h [...] # test_feature_dev_json (test_bpftool.TestBpftool) ... ERROR # test_feature_kernel (test_bpftool.TestBpftool) ... ERROR # test_feature_kernel_full (test_bpftool.TestBpftool) ... ERROR # test_feature_kernel_full_vs_not_full (test_bpftool.TestBpftool) ... ERROR # test_feature_macros (test_bpftool.TestBpftool) ... Error: bug: failed to retrieve CAP_BPF status: Invalid argument # ERROR # # ====================================================================== # ERROR: test_feature_dev_json (test_bpftool.TestBpftool) # ---------------------------------------------------------------------- # Traceback (most recent call last): # File "/opt/rootfs/v5.14-rc4/tools/testing/selftests/bpf/test_bpftool.py", line 57, in wrapper # return f(*args, iface, **kwargs) # File "/opt/rootfs/v5.14-rc4/tools/testing/selftests/bpf/test_bpftool.py", line 82, in test_feature_dev_json # res = bpftool_json(["feature", "probe", "dev", iface]) # File "/opt/rootfs/v5.14-rc4/tools/testing/selftests/bpf/test_bpftool.py", line 42, in bpftool_json # res = _bpftool(args) # File "/opt/rootfs/v5.14-rc4/tools/testing/selftests/bpf/test_bpftool.py", line 34, in _bpftool # return subprocess.check_output(_args) # File "/usr/lib/python3.7/subprocess.py", line 395, in check_output # **kwargs).stdout # File "/usr/lib/python3.7/subprocess.py", line 487, in run # output=stdout, stderr=stderr) # subprocess.CalledProcessError: Command '['bpftool', '-j', 'feature', 'probe', 'dev', 'dummy0']' returned non-zero exit status 255. # ==================
Signed-off-by: Li Zhijian lizhijian@cn.fujitsu.com Acked-by: Song Liu songliubraving@fb.com --- tools/testing/selftests/bpf/test_bpftool.sh | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/tools/testing/selftests/bpf/test_bpftool.sh b/tools/testing/selftests/bpf/test_bpftool.sh index 66690778e36d..6b7ba19be1d0 100755 --- a/tools/testing/selftests/bpf/test_bpftool.sh +++ b/tools/testing/selftests/bpf/test_bpftool.sh @@ -2,4 +2,9 @@ # SPDX-License-Identifier: GPL-2.0 # Copyright (c) 2020 SUSE LLC.
+SCRIPT_DIR=$(dirname $(realpath $0)) + +# 'make -C tools/testing/selftests/bpf' will install to BPFTOOL_INSTALL_PATH +BPFTOOL_INSTALL_PATH="$SCRIPT_DIR"/tools/sbin +export PATH=$BPFTOOL_INSTALL_PATH:$PATH python3 -m unittest -v test_bpftool.TestBpftool
test_bpftool.sh relies on bpftool and test_bpftool.py.
'make install' will install bpftool to INSTALL_PATH/bpf/bpftool, and export it to PATH so that it can be used after installing.
Signed-off-by: Li Zhijian lizhijian@cn.fujitsu.com --- V2: move test_bpftool.py to TEST_PROGS_EXTENDED, files in TEST_GEN_PROGS_EXTENDED are generated by make. Otherwise, it will break out-of-tree install: 'make O=/kselftest-build SKIP_TARGETS= V=1 -C tools/testing/selftests install INSTALL_PATH=/kselftest-install' --- tools/testing/selftests/bpf/Makefile | 4 +++- tools/testing/selftests/bpf/test_bpftool.sh | 3 ++- 2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index f405b20c1e6c..43956afe36ae 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -79,7 +79,7 @@ TEST_PROGS := test_kmod.sh \
TEST_PROGS_EXTENDED := with_addr.sh \ with_tunnels.sh \ - test_xdp_vlan.sh + test_xdp_vlan.sh test_bpftool.py
# Compile but not part of 'make run_tests' TEST_GEN_PROGS_EXTENDED = test_sock_addr test_skb_cgroup_id_user \ @@ -187,6 +187,8 @@ $(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) BPFOBJ=$(BPFOBJ) BPF_INCLUDE=$(INCLUDE_DIR) && \ cp $(SCRATCH_DIR)/runqslower $@
+TEST_GEN_PROGS_EXTENDED += $(DEFAULT_BPFTOOL) + $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED): $(OUTPUT)/test_stub.o $(BPFOBJ)
$(OUTPUT)/test_dev_cgroup: cgroup_helpers.c diff --git a/tools/testing/selftests/bpf/test_bpftool.sh b/tools/testing/selftests/bpf/test_bpftool.sh index 6b7ba19be1d0..718f59692ccb 100755 --- a/tools/testing/selftests/bpf/test_bpftool.sh +++ b/tools/testing/selftests/bpf/test_bpftool.sh @@ -2,9 +2,10 @@ # SPDX-License-Identifier: GPL-2.0 # Copyright (c) 2020 SUSE LLC.
+# 'make -C tools/testing/selftests/bpf install' will install to SCRIPT_DIR SCRIPT_DIR=$(dirname $(realpath $0))
# 'make -C tools/testing/selftests/bpf' will install to BPFTOOL_INSTALL_PATH BPFTOOL_INSTALL_PATH="$SCRIPT_DIR"/tools/sbin -export PATH=$BPFTOOL_INSTALL_PATH:$PATH +export PATH=$SCRIPT_DIR:$BPFTOOL_INSTALL_PATH:$PATH python3 -m unittest -v test_bpftool.TestBpftool
This would happend when we run the tests after install kselftests root@lkp-skl-d01 ~# /kselftests/run_kselftest.sh -t bpf:test_doc_build.sh TAP version 13 1..1 # selftests: bpf: test_doc_build.sh perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LANGUAGE = (unset), LC_ALL = (unset), LC_ADDRESS = "en_US.UTF-8", LC_NAME = "en_US.UTF-8", LC_MONETARY = "en_US.UTF-8", LC_PAPER = "en_US.UTF-8", LC_IDENTIFICATION = "en_US.UTF-8", LC_TELEPHONE = "en_US.UTF-8", LC_MEASUREMENT = "en_US.UTF-8", LC_TIME = "en_US.UTF-8", LC_NUMERIC = "en_US.UTF-8", LANG = "en_US.UTF-8" are supported and installed on your system. perl: warning: Falling back to the standard locale ("C"). # skip: bpftool files not found! # ok 1 selftests: bpf: test_doc_build.sh # SKIP
Signed-off-by: Li Zhijian lizhijian@cn.fujitsu.com --- tools/testing/selftests/bpf/test_bpftool_build.sh | 2 +- tools/testing/selftests/bpf/test_doc_build.sh | 5 +++++ 2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/bpf/test_bpftool_build.sh b/tools/testing/selftests/bpf/test_bpftool_build.sh index ac349a5cea7e..b03a87571592 100755 --- a/tools/testing/selftests/bpf/test_bpftool_build.sh +++ b/tools/testing/selftests/bpf/test_bpftool_build.sh @@ -22,7 +22,7 @@ KDIR_ROOT_DIR=$(realpath $PWD/$SCRIPT_REL_DIR/../../../../) cd $KDIR_ROOT_DIR if [ ! -e tools/bpf/bpftool/Makefile ]; then echo -e "skip: bpftool files not found!\n" - exit 0 + exit 4 # KSFT_SKIP=4 fi
ERROR=0 diff --git a/tools/testing/selftests/bpf/test_doc_build.sh b/tools/testing/selftests/bpf/test_doc_build.sh index d67ced95a6cf..679cf968c7d1 100755 --- a/tools/testing/selftests/bpf/test_doc_build.sh +++ b/tools/testing/selftests/bpf/test_doc_build.sh @@ -10,6 +10,11 @@ KDIR_ROOT_DIR=$(realpath $SCRIPT_REL_DIR/../../../../) SCRIPT_REL_DIR=$(dirname $(realpath --relative-to=$KDIR_ROOT_DIR $SCRIPT_REL_PATH)) cd $KDIR_ROOT_DIR
+if [ ! -e $PWD/$SCRIPT_REL_DIR/Makefile ]; then + echo -e "skip: bpftool files not found!\n" + exit 4 # KSFT_SKIP=4 +fi + for tgt in docs docs-clean; do make -s -C $PWD/$SCRIPT_REL_DIR $tgt; done
linux-kselftest-mirror@lists.linaro.org
-
Li Zhijian