4 day week, on holiday monday
Submitted a Linaro Connect presentation.
[TCWG-1424] Investigate profile feedback for code-size
Frustrating week trying to plug profile feedback into builds and
benchmarks and get sensible results
- First attempt was to use clang 2 stage pgo build
-- New pass manager has a bug that prevents clang from linking when I
enable clang instrumented profiling (local ref into discarded comdat
group containing counters)
-- Sample based profiling requires building create_llvm_prof from
Google's autofdo repository. Does not build out of the box after it
switched to using git submodules for its dependencies, managed to fix
up makefile to get it built.
-- Tried a sample profiling run but with perf's sampling rate and
clang's running time mean that the number of samples is too small to
give meaningful results.
-- The version of perf on my machine doesn't support incremental
profiling so I can't accumulate data from multiple runs.
-- Inlining before my pass to mark functions for size optimisation
accounts for majority of code size increase.
- Second attempt was integrating clang/llvm profiling into spec
-- Spent way longer than I'd expected working out how to write a
config file that handles sample and instrumentation based profile
runs. Both forms require the profile generated from perf or an
instrumented build to be post-processed before being used as an input
for the next stage. The sample profiling also needs the path to the
training executable which isn't obvious how to do.
-- Got SpecInt running with all the various combinations that I want
to run, haven't had time to do full runs to get results yet
== Progress ==
* FDPIC
- GCC: posted patch series v2.
* GCC upstream validation:
- looking at some random noise in testing
- reported a couple of regressions
* Infrastructure:
- prototyped native toolchain build job
- misc cleanups
* misc (conf-calls, meetings, emails, ....)
== Next ==
Holidays for 2 weeks, back on July 30th
Progress:
* VIRT-65 [QEMU upstream maintainership]
+ tracked down and sent fix for bug in MPU handling of exception
frame unstacking that broke a Zephyr test case
+ sent some patches for minor GICv2 bugs
+ code review:
- large series adding virtualization support to GICv2
* VIRT-164 [improve Cortex-M emulation]
+ completed patchset that implements execution from small MPU regions
and MMIO regions, and sent it out for review
+ found and fixed a few bugs in the code I was modifying in the process
thanks
-- PMM
=== Work done during this week ===
* Continue work on TCWG-1428 (Support arithmetic on FileCheck regex
variable): external review
+ finish adding comments
+ add documentation and testcase
+ respond to feedback and send email to llvm-dev to discuss best
syntax for new feature
* Started looking into TCWG-1062 (MCF branch prediction overload due
to too many consecutive branches):
+ figure out what machines to run spec on, where to find spec and
how to use Linaro scripts to install/configure and run it (thanks to
Christophe Lyon)
+ try to build a native clang toolchain but throw it away since
clang does not link (perhaps due to lack of memory (4G machine))
+ learn how to build a cross LLVM toolchain (thanks to Peter Smith)
+ figure out how to get the right privilege to run perf (thanks to
Christophe Lyon)
* Misc:
+ LLVM buildbot babysitting with Peter Smith
+ attend presentation about Windows on Arm
+ start a full reorganization of my workflow
+ finish handover of my Arm tickets
=== Plan for week 28 ===
* DSGHACK-25 (Support arithmetic on FileCheck regex variable): address comments
+ rework patch once syntax is agreed
* Continue looking into MCF branch prediction overload due to too many
consecutive branches:
+ reproduce/confirm issue
+ start to investigate a solution
[VIRT-198 # QEMU: SVE Emulation Support ]
Fixed a couple of bugs uncovered by Alex's VQ16 traces.
[VIRT-249 # SVE System Mode ]
Tracked a phantom kernel crash for a while before it
magically went away with a qemu from-scratch rebuild.
(Plus other stuff, maybe? I tried a lot of things.)
Wrote a version of helper_sve_ldff1_bb_r that works
on pages, and works with softmmu.
time ./bld/tests/test-strlen
softmmu: 0m5.467s
linux-user (old): 0m2.016s
linux-user (new): 0m1.853s
Still need to generalize this to the other first-fault/
no-fault helpers. And possibly write some synthetic test
cases for them. Certainly they've not been tested so
far by anything.
[QEMU Upstream]
* tcg-next pull request.
* Review of some ppc fpu patches went too many rounds
before I broke down and wrote some patches myself.
* More nanomips review.
[GCC Upstream]
Fixed some problems with, and committed, the movprfx patches.
r~
[TCWG-1368] Tracked down libfuzzer buildbot failures on aarch64 and raised PR
[TCWG-1424] Use profile information for size optimisation
- Decided to use clang's 2-stage PGO build as next experiment with
compile times on the full test suite as the performance benchmarks
- Spent rather too much time trying to inject an extra c-flag into the
right stage of the bootstrap builds. As a consolation I know a little
bit more about the bootstrap builds.
- Just measuring code size the profile guided feedback builds of clang
are roughly 33% larger than standard clang, with cold functions
optimised for size this drops to 25% larger. Still waiting for
performance figures for all runs. It is looking like in its current
formulation the size optimisation might offer most of the benefits of
PGO but with a lower code size impact. It misses the original goal of
keeping the performance of non-PGO optimisation but lower overall
size.
[Misc] Tracked down a LLD static build failure with a recent glibc.
Looks like it won't affect Arm and AArch64 as it is related to X86
relaxed got relocations to ifunc resolvers.
Planned absences
- Holiday Monday 9th July
Progress:
* VIRT-65 [QEMU upstream maintainership]
+ softfreeze-related maintenance work
+ investigating reported bug with CMSDK timer device model:
couldn't reproduce hangs, but sent patches which fix
various issues
+ code review:
- various patches by the Outreachy/GSoC interns for
the v6M/microbit work
- a few OSX UI bugfix patches
+ scrub through some recently filed bugs to see what might be
worth investigating before the release
+ first pass through KVM Forum talk submissions
thanks
-- PMM
=== Work done during this week ===
* PR85434 / CVE-2018-12886 / TCWG-1379 (stack-protector failure on GCC
ARM): external review
+ finish testing and patch clean up (comments + changelog) and
submit for external review
* Continue work on TCWG-1428 (Support arithmetic on FileCheck regex variable):
+ finish implementation, fix compile issues and debug errors found
on simple testcase
+ run regression testsuite and fix another issue
+ start cleaning up patch
* Misc improvement to jenkins infrastructure
* Misc:
+ gave presentation on stack protector bug
https://docs.google.com/presentation/d/1ksqK-eTYjYhFXlTyVaPA2VIvCTsxwO5Xw-G…
+ Future of microprocessors presentation by Sophie Wilson
=== Plan for week 28 ===
* TCWG-1428 (Support arithmetic on FileCheck regex variable): submit
for external review
+ finish adding comments
+ add documentation and testcase
* Start looking into MCF branch prediction overload due to too many
consecutive branches
SVE Support ([VIRT-198])
========================
- wrote up [post on SVE development] with QEMU for for Linaro Blog
- originally planned to publish on QEMU 3.0 release
- HPC team would like it published before their workshop on 26th
July
- write up a abstract for HPC workshop and plan remote time :todo
[VIRT-198] https://projects.linaro.org/browse/VIRT-198
[post on SVE development]
https://docs.google.com/document/d/15v1asqk-6de2RtA7ZWdIQ29PkY92gl3nlwG7_Se…
RISU Support for SVE ([VIRT-199])
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- CLOSING NOTE [2018-07-06 Fri 12:37]
Changes merged upstream. Ticket closed.
- generated more traces at different VQ's
- VQ3 (found one error, actually x86 SIGFPE issue)
- VQ16 (found 2 errors in ptrue)
- initial run included errors in whilelo, whilels, wrffr, zip[12]
which went away
- I've parked the tests from ASL work pending legal feedback
[VIRT-199] https://projects.linaro.org/browse/VIRT-199
SVE Reviews
~~~~~~~~~~~
- started reviewing {RFC PATCH 00/16} KVM: arm64: Initial support for
SVE guests Message-Id:
<1529593060-542-1-git-send-email-Dave.Martin(a)arm.com>
QEMU ARMv8.3 Support ([VIRT-241])
=================================
- added remaining v8.1 stories to JIRA and re-organised under
- [ARMv8.1 Mandatory Features] and [ARMv8.1 optional features]
- pondering how much effort to do VHE
- it's mostly conditional re-direction of EL1/EL2 registers so
host kernel can pretend it's in EL1 when really in EL2
- some EL2 registers present slightly different formats when VHE
bit is set
[VIRT-241] https://projects.linaro.org/browse/VIRT-241
[ARMv8.1 Mandatory Features]
https://projects.linaro.org/browse/TCWG-1434
[ARMv8.1 optional features] https://projects.linaro.org/browse/TCWG-1435
Upstream Work ([VIRT-109])
==========================
- posted {PATCH v3 00/20} Travis, Code Coverage and Cross Build
updates Message-Id: <20180702143021.18864-1-alex.bennee(a)linaro.org>
- posted {PULL 00/20} Travis, Code Coverage and Cross Build updates
Message-Id: <20180703101444.23778-12-alex.bennee(a)linaro.org>
- with a follow-up of {PULL v2 00/20} Travis, Code Coverage and
Cross Build updates Message-Id:
<20180704090642.3469-1-alex.bennee(a)linaro.org>
- and {PULL v3 00/20} Travis, Code Coverage and Cross Build updates
Message-Id: <20180705160329.30386-1-alex.bennee(a)linaro.org>
- now merged \o/
[VIRT-109] https://projects.linaro.org/browse/VIRT-109
Testing and CI
==============
- spoke with LAVA team about build status and build button
- now have a qa-reports account to submit LAVA jobs via
QEMU CI Loop ([VIRT-187])
~~~~~~~~~~~~~~~~~~~~~~~~~
- investigating porting existing RISU tests via qa-reports
- add additional test patterns :todo
[VIRT-187] https://projects.linaro.org/browse/VIRT-187
KVM CI Loop ([VIRT-2])
~~~~~~~~~~~~~~~~~~~~~~
- need to sync-up on the current state of this work :todo
- started looking at Xiang's latest auto setup scripts
[VIRT-2] https://projects.linaro.org/browse/VIRT-2
[our wiki] https://wiki.linaro.org/Core/Virtualization
MTTCG tests ([VIRT-52])
~~~~~~~~~~~~~~~~~~~~~~~
- need to dust these off and get up-streamed if I'm going to add new
tests :todo
[VIRT-52] https://projects.linaro.org/browse/VIRT-52
Other Tasks
===========
- Spoke to Ilias Apalodimas (LEDGE SIG) about VM networking
- options for zero copy guest to guest networking
- experiments with userspace networking stacks
- lightweight VM+containers
- [Kata Containers] - follow on from Clear Containers work
- Crostini/[CrosVM] work, ChromeOS VM+Docker approach
- Create an Instrumentation EPIC :todo
[Kata Containers] https://katacontainers.io/
[CrosVM] https://chromium.googlesource.com/chromiumos/platform/crosvm/
Completed Reviews [5/5]
=======================
{PATCH v2 00/13} iommu: support txattrs, support TCG execution, implement TZ MPC
Message-Id: <20180604152941.20374-10-peter.maydell(a)linaro.org>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- CLOSING NOTE [2018-06-14 Thu 19:27]
Looks good.
{PATCH 0/8} Docker improvements
Message-Id: <20180628164643.9668-1-f4bug(a)amsat.org>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- CLOSING NOTE [2018-06-29 Fri 21:38]
Grabbed some patches, commented on others
{Qemu-arm} {PATCH v6 00/35} target/arm SVE patches
Message-Id: <20180627043328.11531-1-richard.henderson(a)linaro.org>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- CLOSING NOTE [2018-06-29 Fri 21:39]
Looks good
{Qemu-devel} {PATCH v5 00/35} target/arm SVE patches
Message-Id: <20180621015359.12018-1-richard.henderson(a)linaro.org>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- CLOSING NOTE [2018-06-29 Fri 21:39]
Looking good, stopped to move to v6
{PATCH 0/4} KVM: arm64: FPSIMD/SVE fixes for 4.17
Message-Id: <1528976039-25826-1-git-send-email-Dave.Martin(a)arm.com>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- CLOSING NOTE [2018-06-29 Fri 21:40]
Seems sane to me.
Absences
========
- YVR18 Connect (17th-21st September 2018)
- KVM Forum 2018 (24th-26th October 2018)
Current Review Queue
====================
* {RFC PATCH 00/16} KVM: arm64: Initial support for SVE guests
Message-Id: <1529593060-542-1-git-send-email-Dave.Martin(a)arm.com>
* {Qemu-devel} {RFC v2 0/2} Add BPF suuport to Qemu
Message-Id: <20180625110706.23332-1-sameeh(a)daynix.com>
* {PATCH v4 00/19} reverse debugging
Message-Id: <20180528071332.9424.27343.stgit@pasha-VirtualBox>
* {RFC PATCH 00/12} tests/tcg: Add TriCore tests
Message-Id: <20180501142222.19154-1-kbastian(a)mail.uni-paderborn.de>
* {PATCH 0/4} tests/vm: various trivial fixes
Message-Id: <20180628153535.1411-1-f4bug(a)amsat.org>
* {PATCH 0/6} docker: Port to Python 3
Message-Id: <20180627021423.18404-1-ehabkost(a)redhat.com>
--
Alex Bennée
