Successfully identified regression in *llvm* in CI configuration tcwg_bmk_llvm_tx1/llvm-release-aarch64-spec2k6-O2_LTO. So far, this commit has regressed CI configurations: - tcwg_bmk_llvm_tx1/llvm-release-aarch64-spec2k6-O2_LTO

Culprit: <cut> commit f8ed31cd991bf84372369409595707f6e3cebbce Author: Jonny Farley jonny.farley@sony.com Date: Tue Feb 16 16:34:35 2021 +0000

[Fuzzer][Test] Use %python substitution for trace-malloc-unbalanced.test

This test was found to fail for some of our downstream builds, on computers where python was not on the default $PATH. Therefore add a %python substitution to use sys.executable, based on similar solutions for python calls in tests elsewhere in LLVM.

Differential Revision: https://reviews.llvm.org/D96799 </cut>

Results regressed to (for first_bad == f8ed31cd991bf84372369409595707f6e3cebbce) # reset_artifacts: -10 # build_abe binutils: -9 # build_abe stage1 -- --set gcc_override_configure=--disable-libsanitizer: -8 # build_abe linux: -7 # build_abe glibc: -6 # build_abe stage2 -- --set gcc_override_configure=--disable-libsanitizer: -5 # build_llvm true: -3 # true: 0 # benchmark -- -O2_LTO artifacts/build-f8ed31cd991bf84372369409595707f6e3cebbce/results_id: 1 # 400.perlbench,perlbench_base.default regressed by 103

from (for last_good == cb2876800cc827431f4143c4fc5595c6c1191269) # reset_artifacts: -10 # build_abe binutils: -9 # build_abe stage1 -- --set gcc_override_configure=--disable-libsanitizer: -8 # build_abe linux: -7 # build_abe glibc: -6 # build_abe stage2 -- --set gcc_override_configure=--disable-libsanitizer: -5 # build_llvm true: -3 # true: 0 # benchmark -- -O2_LTO artifacts/build-cb2876800cc827431f4143c4fc5595c6c1191269/results_id: 1

Artifacts of last_good build: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-release-... Results ID of last_good: tx1_64/tcwg_bmk_llvm_tx1/bisect-llvm-release-aarch64-spec2k6-O2_LTO/4095 Artifacts of first_bad build: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-release-... Results ID of first_bad: tx1_64/tcwg_bmk_llvm_tx1/bisect-llvm-release-aarch64-spec2k6-O2_LTO/4070 Build top page/logs: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-release-...

Configuration details:

Reproduce builds: <cut> mkdir investigate-llvm-f8ed31cd991bf84372369409595707f6e3cebbce cd investigate-llvm-f8ed31cd991bf84372369409595707f6e3cebbce

git clone https://git.linaro.org/toolchain/jenkins-scripts

mkdir -p artifacts/manifests curl -o artifacts/manifests/build-baseline.sh https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-release-... --fail curl -o artifacts/manifests/build-parameters.sh https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-release-... --fail curl -o artifacts/test.sh https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-release-... --fail chmod +x artifacts/test.sh

# Reproduce the baseline build (build all pre-requisites) ./jenkins-scripts/tcwg_bmk-build.sh @@ artifacts/manifests/build-baseline.sh

# Save baseline build state (which is then restored in artifacts/test.sh) mkdir -p ./bisect rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/ --exclude /llvm/ ./ ./bisect/baseline/

cd llvm

# Reproduce first_bad build git checkout --detach f8ed31cd991bf84372369409595707f6e3cebbce ../artifacts/test.sh

# Reproduce last_good build git checkout --detach cb2876800cc827431f4143c4fc5595c6c1191269 ../artifacts/test.sh

cd .. </cut>

History of pending regressions and results: https://git.linaro.org/toolchain/ci/base-artifacts.git/log/?h=linaro-local/c...

Artifacts: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-release-... Build log: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-release-...

Full commit (up to 1000 lines): <cut> commit f8ed31cd991bf84372369409595707f6e3cebbce Author: Jonny Farley jonny.farley@sony.com Date: Tue Feb 16 16:34:35 2021 +0000

[Fuzzer][Test] Use %python substitution for trace-malloc-unbalanced.test

This test was found to fail for some of our downstream builds, on computers where python was not on the default $PATH. Therefore add a %python substitution to use sys.executable, based on similar solutions for python calls in tests elsewhere in LLVM.

Differential Revision: https://reviews.llvm.org/D96799 --- compiler-rt/test/fuzzer/lit.cfg.py | 2 ++ compiler-rt/test/fuzzer/trace-malloc-unbalanced.test | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/compiler-rt/test/fuzzer/lit.cfg.py b/compiler-rt/test/fuzzer/lit.cfg.py index 00871b8a94a3..d114ae7937fa 100644 --- a/compiler-rt/test/fuzzer/lit.cfg.py +++ b/compiler-rt/test/fuzzer/lit.cfg.py @@ -61,6 +61,8 @@ config.substitutions.append(('%build_dir', config.cmake_binary_dir)) libfuzzer_src_root = os.path.join(config.compiler_rt_src_root, "lib", "fuzzer") config.substitutions.append(('%libfuzzer_src', libfuzzer_src_root))

+config.substitutions.append(('%python', '"%s"' % (sys.executable))) + def generate_compiler_cmd(is_cpp=True, fuzzer_enabled=True, msan_enabled=False): compiler_cmd = config.clang extra_cmd = config.target_flags diff --git a/compiler-rt/test/fuzzer/trace-malloc-unbalanced.test b/compiler-rt/test/fuzzer/trace-malloc-unbalanced.test index c7b4632140cb..395386ab5cd2 100644 --- a/compiler-rt/test/fuzzer/trace-malloc-unbalanced.test +++ b/compiler-rt/test/fuzzer/trace-malloc-unbalanced.test @@ -8,10 +8,10 @@ RUN: %cpp_compiler %S/TraceMallocTest.cpp -o %t-TraceMallocTest

# Specify python because we can't use the shebang line on Windows. RUN: %run %t-TraceMallocTest -seed=1 -trace_malloc=1 -runs=200 2>&1 | \ -RUN: python %libfuzzer_src/scripts/unbalanced_allocs.py --skip=5 | FileCheck %s +RUN: %python %libfuzzer_src/scripts/unbalanced_allocs.py --skip=5 | FileCheck %s

RUN: %run %t-TraceMallocTest -seed=1 -trace_malloc=2 -runs=200 2>&1 | \ -RUN: python %libfuzzer_src/scripts/unbalanced_allocs.py --skip=5 | FileCheck %s --check-prefixes=CHECK,CHECK2 +RUN: %python %libfuzzer_src/scripts/unbalanced_allocs.py --skip=5 | FileCheck %s --check-prefixes=CHECK,CHECK2

CHECK: MallocFreeTracer: START # Behavior of the format string "%p" is implementation defined. Account for the </cut>