Hi Greg,
This appears to have been a fluke. Boot-testing succeeded before the merge and failed after. Boot-testing on allmodconfig doesn’t seem to be stable, so we are going to disable it.
Regards,
--
Maxim Kuvyrkov
https://www.linaro.org
> On 18 Aug 2021, at 08:38, Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> wrote:
>
> On Wed, Aug 18, 2021 at 05:22:07AM +0000, ci_notify(a)linaro.org wrote:
>> Successfully identified regression in *linux* in CI configuration tcwg_kernel/llvm-master-aarch64-lts-allmodconfig. So far, this commit has regressed CI configurations:
>> - tcwg_kernel/llvm-master-aarch64-lts-allmodconfig
>>
>> Culprit:
>> <cut>
>> commit 132a8267adabd645476b542b3b132c1b91988fe8
>> Author: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org>
>> Date: Thu Aug 12 13:22:21 2021 +0200
>>
>> Linux 5.10.58
>
> <snip>
>
> And what am I supposed to do with this information?
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe(a)googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/YRyczv2OCq51edQh%40kroa….
[TCWG CI] Regression caused by binutils: [gdb/testsuite] Add gdb.testsuite/dump-system-info.exp:
commit b4e4386a2e58ba6ce8d02b952f1bc6ceb8fc95d1
Author: Tom de Vries <tdevries(a)suse.de>
[gdb/testsuite] Add gdb.testsuite/dump-system-info.exp
Results regressed to
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_abe stage1 -- --set gcc_override_configure=--disable-libsanitizer --set gcc_override_configure=--disable-multilib --set gcc_override_configure=--with-cpu=cortex-m4 --set gcc_override_configure=--with-mode=thumb --set gcc_override_configure=--with-float=hard:
-8
# build_abe newlib:
-6
# build_abe stage2 -- --patch linaro-local/vect-metric-branch --set gcc_override_configure=--disable-libsanitizer --set gcc_override_configure=--disable-multilib --set gcc_override_configure=--with-cpu=cortex-m4 --set gcc_override_configure=--with-mode=thumb --set gcc_override_configure=--with-float=hard:
-5
# true:
0
# benchmark -- -O3_VECT_mthumb artifacts/build-b4e4386a2e58ba6ce8d02b952f1bc6ceb8fc95d1/results_id:
1
from
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_abe stage1 -- --set gcc_override_configure=--disable-libsanitizer --set gcc_override_configure=--disable-multilib --set gcc_override_configure=--with-cpu=cortex-m4 --set gcc_override_configure=--with-mode=thumb --set gcc_override_configure=--with-float=hard:
-8
# build_abe newlib:
-6
# build_abe stage2 -- --patch linaro-local/vect-metric-branch --set gcc_override_configure=--disable-libsanitizer --set gcc_override_configure=--disable-multilib --set gcc_override_configure=--with-cpu=cortex-m4 --set gcc_override_configure=--with-mode=thumb --set gcc_override_configure=--with-float=hard:
-5
# true:
0
# benchmark -- -O3_VECT_mthumb artifacts/build-baseline/results_id:
1
THIS IS THE END OF INTERESTING STUFF. BELOW ARE LINKS TO BUILDS, REPRODUCTION INSTRUCTIONS, AND THE RAW COMMIT.
This commit has regressed these CI configurations:
- tcwg_bmk_gnu_eabi_stm32/gnu_eabi-master-arm_eabi-coremark-O3_VECT
First_bad build: https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_ea…
Last_good build: https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_ea…
Baseline build: https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_ea…
Even more details: https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_ea…
Reproduce builds:
<cut>
mkdir investigate-binutils-b4e4386a2e58ba6ce8d02b952f1bc6ceb8fc95d1
cd investigate-binutils-b4e4386a2e58ba6ce8d02b952f1bc6ceb8fc95d1
# Fetch scripts
git clone https://git.linaro.org/toolchain/jenkins-scripts
# Fetch manifests and test.sh script
mkdir -p artifacts/manifests
curl -o artifacts/manifests/build-baseline.sh https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_ea… --fail
curl -o artifacts/manifests/build-parameters.sh https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_ea… --fail
curl -o artifacts/test.sh https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_ea… --fail
chmod +x artifacts/test.sh
# Reproduce the baseline build (build all pre-requisites)
./jenkins-scripts/tcwg_bmk-build.sh @@ artifacts/manifests/build-baseline.sh
# Save baseline build state (which is then restored in artifacts/test.sh)
mkdir -p ./bisect
rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/ --exclude /binutils/ ./ ./bisect/baseline/
cd binutils
# Reproduce first_bad build
git checkout --detach b4e4386a2e58ba6ce8d02b952f1bc6ceb8fc95d1
../artifacts/test.sh
# Reproduce last_good build
git checkout --detach 3814a9e1fe77c01c7e872c25afa198537d4ac780
../artifacts/test.sh
cd ..
</cut>
Full commit (up to 1000 lines):
<cut>
commit b4e4386a2e58ba6ce8d02b952f1bc6ceb8fc95d1
Author: Tom de Vries <tdevries(a)suse.de>
Date: Fri Sep 24 12:39:14 2021 +0200
[gdb/testsuite] Add gdb.testsuite/dump-system-info.exp
When interpreting the testsuite results, it's often relevant what kind of
machine the testsuite ran on. On a local machine one can just do
/proc/cpuinfo, but in case of running tests using a remote system
that distributes test runs to other remote systems that are not directly
accessible, that's not possible.
Fix this by dumping /proc/cpuinfo into the gdb.log, as well as lsb_release -a
and uname -a.
We could do this at the start of each test run, by putting it into unix.exp
or some such. However, this might be too verbose, so we choose to put it into
its own test-case, such that it get triggered in a full testrun, but not when
running one or a subset of tests.
We put the test-case into the gdb.testsuite directory, which is currently the
only place in the testsuite where we do not test gdb. [ Though perhaps this
could be put into a new gdb.info directory, since the test-case doesn't
actually test the testsuite. ]
Tested on x86_64-linux.
---
gdb/testsuite/gdb.testsuite/dump-system-info.exp | 48 ++++++++++++++++++++++++
1 file changed, 48 insertions(+)
diff --git a/gdb/testsuite/gdb.testsuite/dump-system-info.exp b/gdb/testsuite/gdb.testsuite/dump-system-info.exp
new file mode 100644
index 00000000000..bf181469bd5
--- /dev/null
+++ b/gdb/testsuite/gdb.testsuite/dump-system-info.exp
@@ -0,0 +1,48 @@
+# Copyright 2021 Free Software Foundation, Inc.
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# The purpose of this test-case is to dump /proc/cpuinfo and similar system
+# info into gdb.log.
+
+# Check if /proc/cpuinfo is available.
+set res [remote_exec target "test -r /proc/cpuinfo"]
+set status [lindex $res 0]
+set output [lindex $res 1]
+
+if { $status == 0 && $output == "" } {
+ verbose -log "Cpuinfo available, dumping:"
+ remote_exec target "cat /proc/cpuinfo"
+} else {
+ verbose -log "Cpuinfo not available"
+}
+
+set res [remote_exec target "lsb_release -a"]
+set status [lindex $res 0]
+set output [lindex $res 1]
+
+if { $status == 0 } {
+ verbose -log "lsb_release -a availabe, dumping:\n$output"
+} else {
+ verbose -log "lsb_release -a not available"
+}
+
+set res [remote_exec target "uname -a"]
+set status [lindex $res 0]
+set output [lindex $res 1]
+
+if { $status == 0 } {
+ verbose -log "uname -a availabe, dumping:\n$output"
+} else {
+ verbose -log "uname -a not available"
+}
</cut>
After llvm commit e8e2edd8ca88f8b0a7dba141349b2aa83284f3af
Author: Erich Keane <erich.keane(a)intel.com>
Fix test from 8dd42f, capitalization in test
the following benchmarks slowed down by more than 2%:
- 464.h264ref slowed down by 3% from 10973 to 11249 perf samples
- 464.h264ref:[.] FastFullPelBlockMotionSearch slowed down by 12% from 1446 to 1619 perf samples
Below reproducer instructions can be used to re-build both "first_bad" and "last_good" cross-toolchains used in this bisection. Naturally, the scripts will fail when triggerring benchmarking jobs if you don't have access to Linaro TCWG CI.
For your convenience, we have uploaded tarballs with pre-processed source and assembly files at:
- First_bad save-temps: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
- Last_good save-temps: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
- Baseline save-temps: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Configuration:
- Benchmark: SPEC CPU2006
- Toolchain: Clang + Glibc + LLVM Linker
- Version: all components were built from their tip of trunk
- Target: aarch64-linux-gnu
- Compiler flags: -O3
- Hardware: NVidia TX1 4x Cortex-A57
This benchmarking CI is work-in-progress, and we welcome feedback and suggestions at linaro-toolchain(a)lists.linaro.org . In our improvement plans is to add support for SPEC CPU2017 benchmarks and provide "perf report/annotate" data behind these reports.
THIS IS THE END OF INTERESTING STUFF. BELOW ARE LINKS TO BUILDS, REPRODUCTION INSTRUCTIONS, AND THE RAW COMMIT.
This commit has regressed these CI configurations:
- tcwg_bmk_llvm_tx1/llvm-master-aarch64-spec2k6-O3
First_bad build: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Last_good build: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Baseline build: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Even more details: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Reproduce builds:
<cut>
mkdir investigate-llvm-e8e2edd8ca88f8b0a7dba141349b2aa83284f3af
cd investigate-llvm-e8e2edd8ca88f8b0a7dba141349b2aa83284f3af
# Fetch scripts
git clone https://git.linaro.org/toolchain/jenkins-scripts
# Fetch manifests and test.sh script
mkdir -p artifacts/manifests
curl -o artifacts/manifests/build-baseline.sh https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-… --fail
curl -o artifacts/manifests/build-parameters.sh https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-… --fail
curl -o artifacts/test.sh https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-… --fail
chmod +x artifacts/test.sh
# Reproduce the baseline build (build all pre-requisites)
./jenkins-scripts/tcwg_bmk-build.sh @@ artifacts/manifests/build-baseline.sh
# Save baseline build state (which is then restored in artifacts/test.sh)
mkdir -p ./bisect
rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/ --exclude /llvm/ ./ ./bisect/baseline/
cd llvm
# Reproduce first_bad build
git checkout --detach e8e2edd8ca88f8b0a7dba141349b2aa83284f3af
../artifacts/test.sh
# Reproduce last_good build
git checkout --detach 77d200a546136c2855063613ff4bca1f682fb23a
../artifacts/test.sh
cd ..
</cut>
Full commit (up to 1000 lines):
<cut>
commit e8e2edd8ca88f8b0a7dba141349b2aa83284f3af
Author: Erich Keane <erich.keane(a)intel.com>
Date: Fri Sep 24 10:24:17 2021 -0700
Fix test from 8dd42f, capitalization in test
---
clang/test/CXX/drs/dr17xx.cpp | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/clang/test/CXX/drs/dr17xx.cpp b/clang/test/CXX/drs/dr17xx.cpp
index 42303c83ae3c..c8648908ebda 100644
--- a/clang/test/CXX/drs/dr17xx.cpp
+++ b/clang/test/CXX/drs/dr17xx.cpp
@@ -129,7 +129,7 @@ namespace dr1778 { // dr1778: 9
namespace dr1762 { // dr1762: 14
#if __cplusplus >= 201103L
float operator ""_E(const char *);
- // expected-error@+2 {{invalid suffix on literal; c++11 requires a space between literal and identifier}}
+ // expected-error@+2 {{invalid suffix on literal; C++11 requires a space between literal and identifier}}
// expected-warning@+1 {{user-defined literal suffixes not starting with '_' are reserved; no literal will invoke this operator}}
float operator ""E(const char *);
#endif
</cut>
Thanks, Stanislav,
FWIW, it will be, probably, easier for you to just rebuild the compiler, it is an x86_64-linux-gnu -> arm-linux-gnueabihf cross. This link has the build log [1].
cmake -G Ninja ../llvm/llvm '-DLLVM_ENABLE_PROJECTS=clang;lld' -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=True -DCMAKE_INSTALL_PREFIX=../llvm-install -DLLVM_TARGETS_TO_BUILD=ARM
Then compile the pre-processed source with plain -O2 or -O3 optimisation settings.
[1] https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tk1-llvm-master-…
Regards,
--
Maxim Kuvyrkov
https://www.linaro.org
> On 24 Sep 2021, at 20:30, Mekhanoshin, Stanislav <Stanislav.Mekhanoshin(a)amd.com> wrote:
>
> [AMD Official Use Only]
>
> I have reverted the whole change. There was yet another perf regression report.
>
> Stas
>
> From: Mekhanoshin, Stanislav
> Sent: Thursday, September 23, 2021 11:48
> To: Maxim Kuvyrkov <maxim.kuvyrkov(a)linaro.org>
> Cc: linaro-toolchain <linaro-toolchain(a)lists.linaro.org>
> Subject: RE: [TCWG CI] 456.hmmer slowed down by 6% after llvm: Allow rematerialization of virtual reg uses
>
> Thanks. I see the reload. There shall not be extra pressure since that is the whole idea, make pressure less. However, I see more spills in that specific file, fast_algorithms.s if I get it right.
> Can I get the IR for it? Something to feed llc.
>
> Stas
>
> From: Maxim Kuvyrkov <maxim.kuvyrkov(a)linaro.org>
> Sent: Thursday, September 23, 2021 2:31
> To: Mekhanoshin, Stanislav <Stanislav.Mekhanoshin(a)amd.com>
> Cc: linaro-toolchain <linaro-toolchain(a)lists.linaro.org>
> Subject: Re: [TCWG CI] 456.hmmer slowed down by 6% after llvm: Allow rematerialization of virtual reg uses
>
> [CAUTION: External Email]
>
> Thanks, Stanislav.
>
> I’ve looked into profile dumps, and 456.hmmer’s hot loop get several additional reloads. E.g., "ldr r1, [sp, #84]” generates 203 additional samples, which translates into 20 seconds of time just for that one instruction.
>
> See the attached profile dumps and the the screenshot with the hot loop highlighted.
>
> Maybe your patch increases register pressure too much?
>
> Regards,
>
> --
> Maxim Kuvyrkov
> https://www.linaro.org
>
> > On 22 Sep 2021, at 22:35, Mekhanoshin, Stanislav <Stanislav.Mekhanoshin(a)amd.com> wrote:
> >
> > [AMD Official Use Only]
> >
> > There are actually couple things worth to try if that is easy:
> >
> > https://reviews.llvm.org/D109077
> > https://reviews.llvm.org/differential/diff/374324/
> >
> > Both may slightly change spill weights and then spilling pattern.
> >
> > Stas
> >
> > -----Original Message-----
> > From: Mekhanoshin, Stanislav
> > Sent: Wednesday, September 22, 2021 12:09
> > To: Maxim Kuvyrkov <maxim.kuvyrkov(a)linaro.org>
> > Cc: linaro-toolchain <linaro-toolchain(a)lists.linaro.org>
> > Subject: RE: [TCWG CI] 456.hmmer slowed down by 6% after llvm: Allow rematerialization of virtual reg uses
> >
> > I assume some of the newly rematerialized instructions caused perf drops. Probably some very specific ones. I would appreciate if you could point them to me.
> > In addition I believe I would need to have a linked or optimized bitcode to feed into llc.
> >
> > Stas
> >
> > -----Original Message-----
> > From: Maxim Kuvyrkov <maxim.kuvyrkov(a)linaro.org>
> > Sent: Wednesday, September 22, 2021 12:06
> > To: Mekhanoshin, Stanislav <Stanislav.Mekhanoshin(a)amd.com>
> > Cc: linaro-toolchain <linaro-toolchain(a)lists.linaro.org>
> > Subject: Re: [TCWG CI] 456.hmmer slowed down by 6% after llvm: Allow rematerialization of virtual reg uses
> >
> > [CAUTION: External Email]
> >
> > Hi Stanislav,
> >
> > That's fair; I or someone from Linaro will try to analyze this and follow up here.
> >
> > On a more general note, what info would you like to see in these benchmarking regression reports?
> >
> > Thanks,
> >
> > --
> > Maxim Kuvyrkov
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linar…
> >
> >
> >> On Sep 22, 2021, at 9:40 PM, Mekhanoshin, Stanislav <Stanislav.Mekhanoshin(a)amd.com> wrote:
> >>
> >> [AMD Official Use Only]
> >>
> >> Hm... I'd really like to help, but I do not think I can do anything with megabytes of code in an asm which I do not understand and tons of differences in 48 asm files.
> >> What I can see there is overall less spilling code which was the intent in the first place: hmmer has 4 less spill opcodes overall and sphinx has 27 less of them.
> >> I doubt I could say much more without someone pointing to the actual root cause.
> >>
> >> Stas
> >>
> >> -----Original Message-----
> >> From: Maxim Kuvyrkov <maxim.kuvyrkov(a)linaro.org>
> >> Sent: Wednesday, September 22, 2021 5:16
> >> To: Mekhanoshin, Stanislav <Stanislav.Mekhanoshin(a)amd.com>
> >> Cc: linaro-toolchain <linaro-toolchain(a)lists.linaro.org>
> >> Subject: Re: [TCWG CI] 456.hmmer slowed down by 6% after llvm: Allow rematerialization of virtual reg uses
> >>
> >> [CAUTION: External Email]
> >>
> >> Hi Stanislav,
> >>
> >> Attached is a tarball with -save-temps output (pre-processed source and generated assembly) for first-bad run (your commit) and last-good run (immediate parent of your commit).
> >>
> >> --
> >> Maxim Kuvyrkov
> >> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linar…
> >>
> >>> On 20 Sep 2021, at 23:15, Mekhanoshin, Stanislav <Stanislav.Mekhanoshin(a)amd.com> wrote:
> >>>
> >>> [AMD Official Use Only]
> >>>
> >>> Thanks for letting me know. Some regressions are inevitable, however do you happen to have any analysis and dumps? I myself do not understand ARM ISA well...
> >>>
> >>> Stas
> >>>
> >>> -----Original Message-----
> >>> From: Maxim Kuvyrkov <maxim.kuvyrkov(a)linaro.org>
> >>> Sent: Wednesday, September 15, 2021 5:52
> >>> To: Mekhanoshin, Stanislav <Stanislav.Mekhanoshin(a)amd.com>
> >>> Cc: linaro-toolchain <linaro-toolchain(a)lists.linaro.org>
> >>> Subject: Re: [TCWG CI] 456.hmmer slowed down by 6% after llvm: Allow rematerialization of virtual reg uses
> >>>
> >>> [CAUTION: External Email]
> >>>
> >>> Hi Stanislav,
> >>>
> >>> FYI, your patch seems to be slowing down two of SPEC CPU2006 tests on 32-bit ARM at -O2 and -O3 optimization levels.
> >>>
> >>> --
> >>> Maxim Kuvyrkov
> >>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linar…
> >>>
> >>
> >>
> >
> <image001.png>