Recently I came across two excellent post about accelerating clang/llvm
build with different compiler/optimization [1] [2].
I tried some of author advices getting very good results. Basically I
moved to optimized clang build, changed to gold linker and used another
memory allocator than system glibc one. Results in build time for all
the clang/llvm toolchain is summarized below (my machine is a i7-4510U,
2C/4T, 8GB, 256GB SSD):
GCC 4.8.4 + gold (Ubuntu 14.04)
real 85m17.640s
user 257m1.976s
sys 11m35.284s
LLVM 3.6 + gold (Ubuntu 14.04)
real 34m4.909s
user 128m43.382s
sys 3m51.643s
LLVM 3.7 + gold + tcmalloc
real 32m56.707s
user 121m40.562s
sys 3m52.358s
Gold linker also shows a *much* less RSS usage, I am able to fully use make -j4
while linking in 8GB without issue any swapping.
Two things I would add/check for the posts:
1. Change from libc to tcmalloc showed me a 3-4% improvement. I tried jemalloc,
but tcmalloc is faster. I am using currently system version 2.2, but I have
pushed an aggressive decommit patch to enable as default for 2.4 that might
show lower RSS and latency (I will check it later).
2. First I try to accelerate my build by offloading compilation using distcc.
Results were good, although the other machine utilization (i7, 4C/8T, 8GB)
showed mixes cpu utilization. The problem was linking memory utilization
using ld.bfd, which generates a lot of swapping with higher job count. I
will try using distcc with clang.
[1] http://blogs.s-osg.org/an-introduction-to-accelerating-your-build-with-clan…
[2] http://blogs.s-osg.org/a-conclusion-to-accelerating-your-build-with-clang/
Benchmark automation - TCWG-360 [7/10]
* Arndales stopped booting
** Package servers for elderly filesystem had gone
** Investigated some approaches to creating more stable filesystems
** Realized I could just updated image to point at old-releases, so
did that for now
* _More_ time thinking about interactions with Jenkins & LAVA. Fathi
gave me some Jenkins jobs to prototype in.
* Brain-dumped some of the present state of things into Collaborate
Misc - [3/10]
=Plan=
Jenkins prototyping
>> Using Python to script GDB makes it much more efficient to do testing.
>> Having a Python-disabled build of GDB prevents this.
I use the example of the gdb-python scripts for the linux kernel.
They are very useful, these do not work when using GDB from windows.
-Duane
It seems the prebuilt windows releases of GDB do not enable Python.
Are there plans to release a python-enabled-gdb in the windows builds?
If not, what are the roadblocks to this?
Thanks
Example:
$ ./aarch64-linux-gnu-gdb.exe
GNU gdb (Linaro GDB 2015.02-3) 7.8-2014.09-1-git
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show
copying"
and "show warranty" for details.
This GDB was configured as "--host=i686-w64-mingw32
--target=aarch64-linux-gnu"
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.launchpad.net/gcc-linaro>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
(gdb) python
>
>Scripting in the "Python" language is not supported in this copy of GDB.
(gdb) quit
== This week ==
* Sprint recovery day (2/10)
* TCWG-140 - Transform end of loop conditions to min_expr (6/10)
- Investigated match.pd change added by Andrew Pinski
* TCWG-734 - GCC Autovectorization (1/10)
- EPIC Review
* Misc (1/10)
- Conference calls
== Next week ==
- Call with Charles and Prathamesh for Autovectorization planning
- Begin developing patch for TCWG-140
- Benchmark TCWG-146 patch
- TCWG-832 investigation
== This week ==
* cfgloop.h flatten (2/10)
- Rebased and tested on top of Andrew's patch
- bootstrapped on x86_64-unknown-linux-gnu
- building on all targets in config-list.mk in progress
* TCWG-777 (2/10)
- Problem is still present on trunk
- Original test-case: http://pastebin.com/UTv8g3aA
- Original test-case assembly for -O1: http://pastebin.com/815TgDjV
- Possibly reduced test-case: http://pastebin.com/UGUteX4a
- Assembly for reduced test-case for -O1: http://pastebin.com/edKu6CxH
- x86 assembly for reduced test-case for -O1: http://pastebin.com/9APbJeLr
- Workaround: To disable CSE in branch conditions make flags volatile.
This prevents
storing result of ands to temporary but leads to worse code-gen.
- Possible approaches from (https://bugs.launchpad.net/gcc-linaro/+bug/851258):
a) teach PRE not to fold in such cases ? I don't know how.
b) Write a new tree-ssa pass that "unfolds" cse on result of test conditions.
Check if operand in if (operand != 0) is a result of test condition.
eg:
_8 = flags & 1;
if (_8 != 0)
goto true_branch
else
goto false_branch
In this case we could add an attribute to _8 say "unfold".
If we again come across if (_8 != 0) at a later point,
and it's marked with "unfold",
we can create a new ssa var with definition of _8.
so it becomes:
_9 = flags & 1;
if (_9 != 0)
...
which will "undo" CSE on test conditions.
Does this sounds reasonable ?
Should we do it on all test conditions ?
I don't understand "suitable compare insn" a target should have in above link.
* TCWG-145 (2/10)
- Optimization is already implemented in PRE pass.
- Verified for various test-cases in eembc.
- Currently gcc does not convert global statics to locals
- remove-local-statics-7.c not handled currently by gcc:
(https://gcc.gnu.org/ml/gcc-patches/2008-07/msg01007.html)
- Desirable to have ipa dead-store analysis to avoid stores to static vars:
https://gcc.gnu.org/ml/gcc-patches/2008-07/msg01609.htmlhttps://gcc.gnu.org/ml/gcc-patches/2008-07/msg01602.html
- continuing to look for cases where static-to-locals wins over PRE.
* Misc (4/10)
- Built firefox on x86 non-LTO
- Read thru tree-ssa-dse.c, IPA, LTO doc
- US Visa application
- Setting up home office
== Issues ==
Not able to connect to #linaro-tcwg thru ircproxy.linaro.org , password invalid.
== Next Week ==
- TCWG-830
- TCWG-835
- TCWG-844
- continue investigating branch out of range error issue.
- Work towards committing cfgloop.h patch
== Progress ==
Sprint recovery [2/10]
(TCWG-775) NEON intrinsics error messages [7/10]
. familiarisation with differences between ARM and AArch64 NEON/SIMD backends
. started porting error reporting changes to ARM NEON backend
. pinged patch for AArch64 work
Misc [1/10]
== Plans ==
continue NEON error messages
vectorization work discussions
== Progress ==
* Maintenance (CARD-1833 1/10)
- Looking at AArch64 ADD->SUB a bit more
* Buildbots (CARD-1823 6/10)
- Moving LNT bot to CMake
- Setting up LLD and LLDB buildbots
- Investigating LNT instability / Perf buildbot
* Background (3/10)
- Code review, meetings, discussions, etc.
- Reviewing more stride vectorizer patches (back-end)
== Plan ==
* Try to get some development going back again...
# Progress #
* Linaro GDB [7/10]
** TCWG-805, aarch64 native debugging multi-arch support.
Aarch64 GDB works well debugging Aarch32 programs, except on handling
fork/vfork and watchpoint. Test harness uses ldd which doesn't
handle aarch32 too.
Report a kernel issue that it doesn't get TLS base right through
ptrace in compat mode. Got a patch to fix it.
* FSF GDB [3/10]
** Push in a fix to unbreak arm native gdb build.
** Post a patch to fix a bug on using PTRACE_GETREGSET on arm-linux.
** GDB 7.10 release. Branch isn't created. Test results of arm
and aarch64 look good.
# Plan #
* TCWG-805, start to push some preparatory patches upstream.
* Off on Thu.
--
Yao
