Hi,
could somebody help me understand why the total size of the
recorded ETM trace differs from run to run?
Is this something in my Juno machine setup, or do you also see this?
The maximum length that has been recorded is about 6MB on my setup.
When I am recording the trace of the same program on intel-pt:
$ perf record -e intel_pt//u ./sort
the amount of captured data is deterministic (around 296MB +/- a few KB.)
Thanks,
Sebastian
sort.c is from https://gcc.gnu.org/wiki/AutoFDO/Tutorial
+ gcc sort.c -o sort -O3
++ seq 1 10
+ for i in '$(seq 1 10)'
+ /root/etm/OpenCSD/tools/perf/perf record -e cs_etm/(a)20070000.etr/u
--per-thread ./sort
Bubble sorting array of 30000 elements
7779 ms
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.610 MB perf.data ]
+ for i in '$(seq 1 10)'
+ /root/etm/OpenCSD/tools/perf/perf record -e cs_etm/(a)20070000.etr/u
--per-thread ./sort
Bubble sorting array of 30000 elements
7789 ms
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.545 MB perf.data ]
+ for i in '$(seq 1 10)'
+ /root/etm/OpenCSD/tools/perf/perf record -e cs_etm/(a)20070000.etr/u
--per-thread ./sort
Bubble sorting array of 30000 elements
7797 ms
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.353 MB perf.data ]
+ for i in '$(seq 1 10)'
+ /root/etm/OpenCSD/tools/perf/perf record -e cs_etm/(a)20070000.etr/u
--per-thread ./sort
Bubble sorting array of 30000 elements
5949 ms
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.353 MB perf.data ]
+ for i in '$(seq 1 10)'
+ /root/etm/OpenCSD/tools/perf/perf record -e cs_etm/(a)20070000.etr/u
--per-thread ./sort
Bubble sorting array of 30000 elements
7807 ms
[ perf record: Woken up 2 times to write data ]
[ perf record: Captured and wrote 3.287 MB perf.data ]
+ for i in '$(seq 1 10)'
+ /root/etm/OpenCSD/tools/perf/perf record -e cs_etm/(a)20070000.etr/u
--per-thread ./sort
Bubble sorting array of 30000 elements
7772 ms
[ perf record: Woken up 3 times to write data ]
Warning:
AUX data lost 2 times out of 4!
[ perf record: Captured and wrote 0.126 MB perf.data ]
+ for i in '$(seq 1 10)'
+ /root/etm/OpenCSD/tools/perf/perf record -e cs_etm/(a)20070000.etr/u
--per-thread ./sort
Bubble sorting array of 30000 elements
7811 ms
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.001 MB perf.data ]
+ for i in '$(seq 1 10)'
+ /root/etm/OpenCSD/tools/perf/perf record -e cs_etm/(a)20070000.etr/u
--per-thread ./sort
Bubble sorting array of 30000 elements
7784 ms
[ perf record: Woken up 3 times to write data ]
Warning:
AUX data lost 2 times out of 4!
[ perf record: Captured and wrote 0.619 MB perf.data ]
+ for i in '$(seq 1 10)'
+ /root/etm/OpenCSD/tools/perf/perf record -e cs_etm/(a)20070000.etr/u
--per-thread ./sort
Bubble sorting array of 30000 elements
7770 ms
[ perf record: Woken up 2 times to write data ]
Warning:
AUX data lost 1 times out of 1!
[ perf record: Captured and wrote 0.002 MB perf.data ]
+ for i in '$(seq 1 10)'
+ /root/etm/OpenCSD/tools/perf/perf record -e cs_etm/(a)20070000.etr/u
--per-thread ./sort
Bubble sorting array of 30000 elements
5934 ms
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 1.236 MB perf.data ]
Hi Mathieu,
perf/Documentation/intel-pt.txt describes how to make autoFDO work
with Intel-PT recorded traces:
# perf record -e intel_pt//u ./sort
# perf inject -i perf.data -o inj --itrace=i100usl --strip
# create_gcov --binary=./sort --profile=inj --gcov=sort.gcov -gcov_version=1
# gcc -O3 -fauto-profile=sort.gcov sort.c -o sort_autofdo
On the ARM side, I was able to get an ETM trace, and I started working
with my colleague Brian Rzycki on the second step that translates the trace
into branch events.
Attached is the current state of the patch that adds functionality
from intel-pt.c
to cs-etm.c. We are still trying to get more than one branch recorded in the
branch stack before emitting an event, and it looks like what we need is to
decode more than a packet at a time in cs-etm-decoder.c like in
cs_etm_decoder__buffer_packet()
Comments on the early version of the patch are welcome.
Thanks,
Sebastian
Changed openCSD library version number from v0.4.2 to v0.5 and pumped
the kernel version to v4.9.
Signed-off-by: Mathieu Poirier <mathieu.poirier(a)linaro.org>
---
HOWTO.md | 36 ++++++++++++++++++------------------
1 file changed, 18 insertions(+), 18 deletions(-)
diff --git a/HOWTO.md b/HOWTO.md
index 239a2cd194c3..6f768ed8a72c 100644
--- a/HOWTO.md
+++ b/HOWTO.md
@@ -7,7 +7,7 @@ This HOWTO explains how to use the perf cmd line tools and the openCSD
library to collect and extract program flow traces generated by the
CoreSight IP blocks on a Linux system. The examples have been generated using
an aarch64 Juno-r0 platform. All information is considered accurate and tested
-using library version v0.4.2 and the latest perf branch `perf-opencsd-4.8`
+using library version v0.5 and the latest perf branch `perf-opencsd-4.9`
on the [OpenCSD github repository][1].
@@ -15,8 +15,8 @@ On Target Trace Acquisition - Perf Record
-----------------------------------------
All the enhancement to the Perf tools that support the new `cs_etm` pmu have
not been upstreamed yet. To get the required functionality branch
-`perf-opencsd-4.8` needs to be downloaded to the target system where
-traces are to be collected. This branch is an upstream v4.8 kernel
+`perf-opencsd-4.9` needs to be downloaded to the target system where
+traces are to be collected. This branch is an upstream v4.9 kernel
supplemented with modifications to the CoreSight framework and drivers to be
usable by the Perf core. The remaining out of tree patches are being
upstreamed incrementally.
@@ -261,14 +261,14 @@ the host's (which has nothing to do with the target) architecture:
Off Target Perf Tools Compilation
---------------------------------
As stated above not all the pieces of the solution have been upstreamed. To
-get all the components branch `perf-opencsd-4.8` needs to be
+get all the components branch `perf-opencsd-4.9` needs to be
obtained:
- linaro@t430:~/linaro/coresight$ git clone -b perf-opencsd-4.8 https://github.com/Linaro/OpenCSD.git perf-opencsd-4.8
+ linaro@t430:~/linaro/coresight$ git clone -b perf-opencsd-4.9 https://github.com/Linaro/OpenCSD.git perf-opencsd-4.9
...
...
- linaro@t430:~/linaro/coresight$ ls perf-opencsd-4.8/
+ linaro@t430:~/linaro/coresight$ ls perf-opencsd-4.9/
arch certs CREDITS Documentation firmware include ipc Kconfig lib Makefile net REPORTING-BUGS scripts sound usr
block COPYING crypto drivers fs init Kbuild kernel MAINTAINERS mm README samples security tools virt
@@ -279,12 +279,12 @@ successful, but handling of CoreSight trace data won't be supported.
**See perf-test-scripts below for assistance in creating a build and test enviroment.**
- linaro@t430:~/linaro/coresight$ cd perf-opencsd-4.8
- linaro@t430:~/linaro/coresight/perf-opencsd-4.8$ export CSTRACE_PATH=~/linaro/coresight/my-opencsd/decoder
- linaro@t430:~/linaro/coresight/perf-opencsd-4.8$ make -C tools/perf
+ linaro@t430:~/linaro/coresight$ cd perf-opencsd-4.9
+ linaro@t430:~/linaro/coresight/perf-opencsd-4.9$ export CSTRACE_PATH=~/linaro/coresight/my-opencsd/decoder
+ linaro@t430:~/linaro/coresight/perf-opencsd-4.9$ make -C tools/perf
...
...
- linaro@t430:~/linaro/coresight/perf-opencsd-4.8$ ls -l tools/perf/perf
+ linaro@t430:~/linaro/coresight/perf-opencsd-4.9$ ls -l tools/perf/perf
-rwxrwxr-x 1 linaro linaro 6276360 Mar 3 10:05 tools/perf/perf
@@ -323,7 +323,7 @@ to be sure everything is clean.
linaro@t430:~/linaro/coresight/sept20$ rm -rf ~/.debug
linaro@t430:~/linaro/coresight/sept20$ cp -dpR .debug ~/
linaro@t430:~/linaro/coresight/sept20$ export LD_LIBRARY_PATH=~/linaro/coresight/my-opencsd/decoder/lib/linux64/dbg/
- linaro@t430:~/linaro/coresight/sept20$ ../perf-opencsd-4.8/tools/perf/perf report --stdio
+ linaro@t430:~/linaro/coresight/sept20$ ../perf-opencsd-4.9/tools/perf/perf report --stdio
# To display the perf.data header info, please use --header/--header-only options.
#
@@ -367,7 +367,7 @@ to be sure everything is clean.
Additional data can be obtained, which contains a dump of the trace packets received using the command
- mjl@ubuntu-vbox:./perf-opencsd-4.8/coresight/tools/perf/perf report --stdio --dump
+ mjl@ubuntu-vbox:./perf-opencsd-4.9/coresight/tools/perf/perf report --stdio --dump
resulting a large amount of data, trace looking like:-
@@ -416,10 +416,10 @@ Trace Decoding with Perf Script
Working with perf scripts needs more command line options but yields
interesting results.
- linaro@t430:~/linaro/coresight/sept20$ export EXEC_PATH=/home/linaro/coresight/perf-opencsd-4.8/tools/perf/
+ linaro@t430:~/linaro/coresight/sept20$ export EXEC_PATH=/home/linaro/coresight/perf-opencsd-4.9/tools/perf/
linaro@t430:~/linaro/coresight/sept20$ export SCRIPT_PATH=$EXEC_PATH/scripts/python/
linaro@t430:~/linaro/coresight/sept20$ export XTOOL_PATH=/your/aarch64/toolchain/path/bin/
- linaro@t430:~/linaro/coresight/sept20$ ../perf-opencsd-4.8/tools/perf/perf --exec-path=${EXEC_PATH} script --script=python:${SCRIPT_PATH}/cs-trace-disasm.py -- -d ${XTOOL_PATH}/aarch64-linux-gnu-objdump
+ linaro@t430:~/linaro/coresight/sept20$ ../perf-opencsd-4.9/tools/perf/perf --exec-path=${EXEC_PATH} script --script=python:${SCRIPT_PATH}/cs-trace-disasm.py -- -d ${XTOOL_PATH}/aarch64-linux-gnu-objdump
7f89f24d80: 910003e0 mov x0, sp
7f89f24d84: 94000d53 bl 7f89f282d0 <free@plt+0x3790>
@@ -451,18 +451,18 @@ Kernel Trace Decoding
When dealing with kernel space traces the vmlinux file has to be communicated
explicitely to perf using the "--vmlinux" command line option:
- linaro@t430:~/linaro/coresight/sept20$ ../perf-opencsd-4.8/tools/perf/perf report --stdio --vmlinux=./vmlinux
+ linaro@t430:~/linaro/coresight/sept20$ ../perf-opencsd-4.9/tools/perf/perf report --stdio --vmlinux=./vmlinux
...
...
- linaro@t430:~/linaro/coresight/sept20$ ../perf-opencsd-4.8/tools/perf/perf script --vmlinux=./vmlinux
+ linaro@t430:~/linaro/coresight/sept20$ ../perf-opencsd-4.9/tools/perf/perf script --vmlinux=./vmlinux
When using scripts things get a little more convoluted. Using the same example
an above but for traces but for kernel traces, the command line becomes:
- linaro@t430:~/linaro/coresight/sept20$ export EXEC_PATH=/home/linaro/coresight/perf-opencsd-4.8/tools/perf/
+ linaro@t430:~/linaro/coresight/sept20$ export EXEC_PATH=/home/linaro/coresight/perf-opencsd-4.9/tools/perf/
linaro@t430:~/linaro/coresight/sept20$ export SCRIPT_PATH=$EXEC_PATH/scripts/python/
linaro@t430:~/linaro/coresight/sept20$ export XTOOL_PATH=/your/aarch64/toolchain/path/bin/
- linaro@t430:~/linaro/coresight/sept20$ ../perf-opencsd-4.8/tools/perf/perf --exec-path=${EXEC_PATH} script \
+ linaro@t430:~/linaro/coresight/sept20$ ../perf-opencsd-4.9/tools/perf/perf --exec-path=${EXEC_PATH} script \
--vmlinux=./vmlinux \
--script=python:${SCRIPT_PATH}/cs-trace-disasm.py -- \
-d ${XTOOLS_PATH}/aarch64-linux-gnu-objdump \
--
2.7.4
On 11 November 2016 at 08:32, Sebastian Pop <sebpop(a)gmail.com> wrote:
> Salut Matthieu,
Bon matin,
>
> I was following your patches to enable ETM support in the linux kernel
> last years, and I have just seen that you have a very nice blog on how
> to use perf record to gather execution traces.
Depending on the source you have read the information may be outdated.
Nowadays the very latest instructions can be found in the HOWTO.md on
the OpenCSD repository [1]. That documentation is maintained
regularly and is considered accurate.
[1]. https://github.com/Linaro/OpenCSD/tree/opencsd-0v003
> I am mainly interested to use this mechanism to enable collection of
> traces with "perf record -b" like Intel's Last Branch Record: this is
> used by AutoFDO in the compiler to know which code paths are hot and
> enable better inlining, register allocation, and code layout:
> http://research.google.com/pubs/pub45290.html
This looks interesting.
>
> I am interested to work with you to enable "perf record -b" on aarch64.
This is a sizeable project that will keep you occupied for a little
while :o) I am happy to give guidance and review patches. Some
people on the CoreSight mailing list (CC'ed here) are very proficient
with the CoreSight tracers and can help understand the more complex
features of the IP. I suggest CC'ing the list if/when you have
questions.
Best regards,
Mathieu
>
> Thanks,
> Sebastian
> ---------- Forwarded message ----------
> From: Yan Lin Aung <yan_lin_aung(a)yahoo.com>
> To: "coresight(a)lists.linaro.org" <coresight(a)lists.linaro.org>
> Cc:
> Date: Mon, 7 Nov 2016 03:45:45 +0000 (UTC)
> Subject: perf with CoreSight and OpenCSD on TC2 and Juno r2
> Hi Linaro Coresight Team,
>
> I came to know of "Hardware Assisted Tracing on ARM with CoreSight and OpenCSD" by Mathieu Poirier.
> In his presentation, he mentioned the reference platforms to evaluate perf with CoreSight and OpenCSD are Vexpress TC2 and Juno (Page 7 on his slide).
>
> I just checked the "HOWTO.MD" at OpenCSD github site.
> However, there is very limited info on how to get started with Vexpress TC2 and Juno.
>
> I have access to the TC2 and Juno r2 platforms.
> Please provide a rather detailed version of getting started guide to try out perf with CoreSight and OpenCSD on either TC2 or Juno r2.
Hello Yan Lin,
You are correct, the HOWTO.md on github concentrates on CoreSight and
doesn't address platform specifics - something like this would be out
of scope. I'm not exactly sure of what you are looking for in a
"getting started guide"... Both Juno and TC2 are well supported
upstream and can be booted with a mainline kernel. The choice of
bootloader and user space are entirely up to users and don't affect
the CoreSight suite nor its integration with the perf subsystem.
The fact that you have access to both platform leads me to believe you
are part of a large organisation. As such there is definitely people
around you with experience on how to set-up the platforms.
I can try to answer specific questions if you have any.
Thanks,
Mathieu
>
> Thanx.
>
> Regards,
> Yan Lin Aung
>
On 8 November 2016 at 12:27, Darius-Andrei Suciu
<dsuciu(a)cs.stonybrook.edu> wrote:
> Hello Mathieu,
Hi
>
> My name is Darius Suciu, a Phd. student at Stony Brook University.
Congratulation - a PhD is a serious endeavour.
> I am
> working on a project where I need to get tracing data from an Cortex A-9 cpu
> (I'm using an i.MX6 nitrogen max board). The Coresight architecure seems
> very complex to me.
CS is complex to anyone ramping up on it. Looking at the drivers in
the kernel tree most of the complexity comes from the integration with
perf. The drivers and the framework are pretty bare bone and the
sysFS interface very easy to understand.
> I am trying to figure out how I can read the ETB buffer
> from the kernel (without using a JTAG), and the Coresight driver seems to
> offer this functionality.
> On the comments on the Linux Coresight driver, it says it supports also PTM
> (even though the configuration seems mostly ETM specific).
In terms of configuration ETMv3 and PTM are quite similar. So far I
haven't found enough discrepancies that would mandate splitting the
ETMv3 driver.
> Could you please help me out with some questions?
Of course.
>
> Did you using the driver on a Cortex A-9 cpu ? Or do you know what changes
> are required to make it work ?
I never used CS on a cortex A9 but that is irrelevant. What is
important is the kind of tracer (ETMv3/4/PTM) that is coupled with
each CPU and we have support for all of them (except ITM). From there
you'll need to take care of the power domains, clock and the device
tree.
> Do you know if the ETB buffer configuration is the same for ETM and PTM
> architectures (can I reuse it)
ETB and the ETM/PTM are different IP blocks and don't affect each
other. The current ETB driver in the kernel tree will work with any
kind of tracer.
Mathieu
>
> Thank you for the help,
> Darius
>