Review of ETMV4 sysfs code resulted in a number of minor issues being
discovered. Patchset fixed these and updated docs.
Applies to coresight/next
Changes since v3
First 8 patches of v3 have been accepted onto coresight/next. The patch
series is now documents only
Docs .txt files changed to .rst by unrelated patch. This set reflects
this change and updates the added docs to match.
Indexing changed for new coresight docs directory.
Changes since v2 (reviews from Mathieu and Leo):-
Patch 0002 now adds stable tag. Tested on 4.9, 4.14, 4.19
Applies to coresight/next (5.4-rc1)
Documentation changed to .rst format to match recent updates that
converted other CoreSight .txt files.
Misc typo / comment changes.
Changes since v1 (from reviews by Mathieu and Leo):-
Usability patch split into 2 separate functional patches.
Docs patch split into 3 patches.
Misc style and comment typo fixes.
Mike Leach (4):
coresight: etm4x: docs: Update ABI doc for new sysfs name scheme.
coresight: etm4x: docs: Update ABI doc for new sysfs etm4 attributes
coresight: docs: Create common sub-directory for coresight trace.
coresight: etm4x: docs: Adds detailed document for programming etm4x.
.../testing/sysfs-bus-coresight-devices-etm4x | 183 ++--
.../{ => coresight}/coresight-cpu-debug.rst | 0
.../coresight/coresight-etm4x-reference.rst | 798 ++++++++++++++++++
.../trace/{ => coresight}/coresight.rst | 2 +-
Documentation/trace/coresight/index.rst | 9 +
Documentation/trace/index.rst | 3 +-
MAINTAINERS | 3 +-
7 files changed, 925 insertions(+), 73 deletions(-)
rename Documentation/trace/{ => coresight}/coresight-cpu-debug.rst (100%)
create mode 100644 Documentation/trace/coresight/coresight-etm4x-reference.rst
rename Documentation/trace/{ => coresight}/coresight.rst (99%)
create mode 100644 Documentation/trace/coresight/index.rst
--
2.17.1
Good day Jan,
Please CC the coresight mailing list when you have questions such as
this one. There is a lot of knowledgeable people on it that are also
be able to help you.
On Fri, 18 Oct 2019 at 07:42, Jan Hoogerbrugge <jan.hoogerbrugge(a)nxp.com> wrote:
>
> Hi Mathieu,
>
> I am trying to understand Coresight support in the Linux kernel. I am using a Xilinx Zynq
> Ultrascale+ system. I configured the kernel with coresight support enabled. When the
> system is running I see the /sys/bus/coresight directory but the devices directory in it
> stays empty. Also, I do not see messages about coresight reported when booting:
>
> root@xilinx-zcu102-2017_4:~# ls -R /sys/bus/coresight
> /sys/bus/coresight:
> devices drivers_autoprobe uevent
> drivers drivers_probe
>
> /sys/bus/coresight/devices:
>
> /sys/bus/coresight/drivers:
> root@xilinx-zcu102-2017_4:~# dmesg | grep -i coresight
> root@xilinx-zcu102-2017_4:~#
>
> Any idea what I am doing wrong?
My guess it that coresight devices for that processor have not been
specified in the device tree. If I'm not mistaking some people (also
on this list) from Xiling have been experiencing with coresight on
that specific platform - hopefully they will chime in.
>
> I want to use Coresight to obtain some TPUI/ETM traces so that I can experiment with them.
Note that a driver for the TPIU IP block is currently not available.
I never had time to write a driver and nobody has ever submitted one.
> I hope that I can dump some traces to file so that I can process them later. Do you know about
> publicly accessible archives with traces on the Internet? This might be then an alternative
> for me.
I do not know of any.
Thanks,
Mathieu
>
> Regards,
>
> Jan
>
> --
>
> Jan Hoogerbrugge
>
> Principal Security Architect
>
> Competence Center Crypto & Security
>
> NXP Semiconductors
>
> High Tech Campus 46, 5656AE Eindhoven, The Netherlands
>
> Phone: +31 6 57728704
Review of ETMV4 sysfs code resulted in a number of minor issues being
discovered.
Patch set fixes these issues:-
1) Update for ETM v4.4 archtecture.
2) Add missing single shot comparator API.
3) Misc fixes and improvements to sysfs API
4) Updated programmers documentation and reference.
Changes since v2 (reviews from Mathieu and Leo):-
Patch 0002 now adds stable tag. Tested on 4.9, 4.14, 4.19
Applies to coresight/next (5.4-rc1)
Documentation changed to .rst format to match recent updates that
converted other CoreSight .txt files.
Misc typo / comment changes.
Changes since v1 (from reviews by Mathieu and Leo):-
Usability patch split into 2 separate functional patches.
Docs patch split into 3 patches.
Misc style and comment typo fixes.
Mike Leach (11):
coresight: etm4x: Fixes for ETM v4.4 architecture updates.
coresight: etm4x: Fix input validation for sysfs.
coresight: etm4x: Add missing API to set EL match on address filters
coresight: etm4x: Fix issues with start-stop logic.
coresight: etm4x: Improve usability of sysfs - include/exclude addr.
coresight: etm4x: Improve usability of sysfs - CID and VMID masks.
coresight: etm4x: Add view comparator settings API to sysfs.
coresight: etm4x: Add missing single-shot control API to sysfs
coresight: etm4x: docs: Update ABI doc for sysfs features added.
coresight: docs: Create common sub-directory for coresight trace.
coresight: etm4x: docs: Adds detailed document for programming etm4x.
.../testing/sysfs-bus-coresight-devices-etm4x | 183 ++--
.../{ => coresight}/coresight-cpu-debug.rst | 0
.../coresight/coresight-etm4x-reference.rst | 798 ++++++++++++++++++
.../trace/{ => coresight}/coresight.rst | 2 +-
Documentation/trace/{ => coresight}/stm.rst | 0
MAINTAINERS | 3 +-
.../coresight/coresight-etm4x-sysfs.c | 312 ++++++-
drivers/hwtracing/coresight/coresight-etm4x.c | 32 +-
drivers/hwtracing/coresight/coresight-etm4x.h | 17 +-
9 files changed, 1245 insertions(+), 102 deletions(-)
rename Documentation/trace/{ => coresight}/coresight-cpu-debug.rst (100%)
create mode 100644 Documentation/trace/coresight/coresight-etm4x-reference.rst
rename Documentation/trace/{ => coresight}/coresight.rst (99%)
rename Documentation/trace/{ => coresight}/stm.rst (100%)
--
2.17.1
Hi Yabin,
On Tue, 17 Sep 2019 at 15:53, Yabin Cui <yabinc(a)google.com> wrote:
>
> Hi guys,
>
> I am trying to reduce ETM data lose. There seems to have two types of data lose:
> 1. caused by instruction trace buffer overflow, which generates an Overflow packet after recovering.
> 2. caused by ETR buffer overflow, which generates an PERF_RECORD_AUX with PERF_AUX_FLAG_TRUNCATED.
>
> In practice, the second one is unlikely to happen when I set ETR buffer size to 4M, and can be improved by setting higher buffer size.
> The first one happens much more frequently, can happen about 21K times when generating 5.3M etm data.
> So I want to know if there is any way to reduce that.
>
> I found in the ETM arch manual that the overflow seems controlled by TRCSTALLCTLR, a stall control register.
> But TRCIDR3.NOOVERFLOW is not supported on my experiment device, which seems using qualcomm sdm845.
> TRCSTALLCTLR.ISTALL seems can be set by writing to mode file in /sys. But I didn't find any way to set
> TRCSTALLCTLR.LEVEL in linux kernel. So I wonder if it is ever used.
>
> Do you guys have any suggestions?
In order to get a timely response to your questions I advise to CC the
coresight mailing list (which I have included). There is a lot of
knowledgeable people there that can also help, especially with
architecture specific configuration.
TRCSTALLCTLR.LEVEL is currently not accessible to users because we
simply never needed to use the feature. If using the ISTALL and LEVEL
parameters help with your use case send a patch that exposes the
entire TRCSTALLCTLR register (rather than just the LEVEL field) and
I'll be happy to fold it in.
Thanks,
Mathieu
>
> Best,
> Yabin
>
Hi coresight team,
In ARM ETM specification for ETM4.0 to 4.2, TRCAUTHSTATUS contains NSNID,
showing whether non-invasive debug is disabled.
And In ARM Coresight Specification 3.0, NSNID field is 0b10 (supported and
disabled) if (NIDEN | DBGEN) == FALSE.
And In ARM architecture manual for ARMv8-A, in K2.1:
DBGEN is for external debug enable.
NIDEN is for external profiling and trace enable.
So it seems if DBGEN and NIDEN is disabled by hardware, even if the
intention is to disable external debug interface, ETM is totally
disabled. And there is no way to use it for self-hosted tracing.
Is it true? And if yes, is there any plan to solve it in the future?
Thanks,
Yabin
Hello,
We are using coresight to dump compressed stream through ETR into a 16MB
buffer in RAM. (The platform is nVidia TX2) To gather data from the buffer
we are using the Linux dd command:
dd if=/dev/8050000.etr of=~/coresightdata.bin
The issue is that during the copying of the data, the coresight recording
becomes disabled (which shouldn't really be the case since its a circular
buffer), so we are having some blind spots in the recordings for the
duration of the copying of data from RAM into a file.
Is there any way to prevent this from happening? Maybe to set up some kind
of ping-pong scheme, or to specify somehow that coresight shouldn't stop
recording while accessing the ETR buffer?
Thank you.
Srdjan Stokic
Mobile: +389-78-835-505
This patch series adds support for thread stack and callchain.
Patch 01 is to refactor the instruction size calculation and it is a
preparation for patch 02.
Patch 02 is to add thread stack support, after applying this patch then
the option '-F,+callindent' can be used by perf script tool; patch 03
is to add branch filter thus the perf tool can only display function
calls and returns after enable the call indentation or call chain
related options. Patch 04 is the patch to synthesize call chain for the
instruction samples.
Patch 05 allows the instruction sample can be handled synchronously with
the thread stack, thus it fixes an error for the callchain generation.
This patch set has been tested on 96boards Hikey620.
Test for option '-F,+callindent':
Before:
# perf script -F,+callindent
main 2808 1 branches: coresight_test1 ffff8634f5c8 coresight_test1+0x3c (/root/coresight_test/libcstest.so)
main 2808 1 branches: printf@plt aaaaba8d37ec main+0x28 (/root/coresight_test/main)
main 2808 1 branches: printf@plt aaaaba8d36bc printf@plt+0xc (/root/coresight_test/main)
main 2808 1 branches: _init aaaaba8d3650 _init+0x30 (/root/coresight_test/main)
main 2808 1 branches: _dl_fixup ffff86373b4c _dl_runtime_resolve+0x40 (/lib/aarch64-linux-gnu/ld-2.28.so)
main 2808 1 branches: _dl_lookup_symbol_x ffff8636e078 _dl_fixup+0xb8 (/lib/aarch64-linux-gnu/ld-2.28.so)
[...]
After:
# perf script -F,+callindent
main 2808 1 branches: coresight_test1@plt aaaaba8d37d8 main+0x14 (/root/coresight_test/main)
main 2808 1 branches: _dl_fixup ffff86373b4c _dl_runtime_resolve+0x40 (/lib/aarch64-linux-gnu/ld-2.28.s
main 2808 1 branches: _dl_lookup_symbol_x ffff8636e078 _dl_fixup+0xb8 (/lib/aarch64-linux-gnu/ld-2.28.so)
main 2808 1 branches: do_lookup_x ffff8636a49c _dl_lookup_symbol_x+0x104 (/lib/aarch64-linux-gnu/ld-2.28.
main 2808 1 branches: check_match ffff86369bf0 do_lookup_x+0x238 (/lib/aarch64-linux-gnu/ld-2.28.so)
main 2808 1 branches: strcmp ffff86369888 check_match+0x70 (/lib/aarch64-linux-gnu/ld-2.28.so)
main 2808 1 branches: printf@plt aaaaba8d37ec main+0x28 (/root/coresight_test/main)
main 2808 1 branches: _dl_fixup ffff86373b4c _dl_runtime_resolve+0x40 (/lib/aarch64-linux-gnu/ld-2.28.s
main 2808 1 branches: _dl_lookup_symbol_x ffff8636e078 _dl_fixup+0xb8 (/lib/aarch64-linux-gnu/ld-2.28.so)
main 2808 1 branches: do_lookup_x ffff8636a49c _dl_lookup_symbol_x+0x104 (/lib/aarch64-linux-gnu/ld-2.28.
main 2808 1 branches: _dl_name_match_p ffff86369af0 do_lookup_x+0x138 (/lib/aarch64-linux-gnu/ld-2.28.so)
main 2808 1 branches: strcmp ffff8636f7f0 _dl_name_match_p+0x18 (/lib/aarch64-linux-gnu/ld-2.28.so)
[...]
Test for option '--itrace=g':
Before:
# perf script --itrace=g16l64i100
main 1579 100 instructions: ffff0000102137f0 group_sched_in+0xb0 ([kernel.kallsyms])
main 1579 100 instructions: ffff000010213b78 flexible_sched_in+0xf0 ([kernel.kallsyms])
main 1579 100 instructions: ffff0000102135ac event_sched_in.isra.57+0x74 ([kernel.kallsyms])
main 1579 100 instructions: ffff000010219344 perf_swevent_add+0x6c ([kernel.kallsyms])
main 1579 100 instructions: ffff000010214854 perf_event_update_userpage+0x4c ([kernel.kallsyms])
[...]
After:
# perf script --itrace=g16l64i100
main 1579 100 instructions:
ffff000010213b78 flexible_sched_in+0xf0 ([kernel.kallsyms])
ffff00001020c0b4 visit_groups_merge+0x12c ([kernel.kallsyms])
main 1579 100 instructions:
ffff0000102135ac event_sched_in.isra.57+0x74 ([kernel.kallsyms])
ffff0000102137a0 group_sched_in+0x60 ([kernel.kallsyms])
ffff000010213b84 flexible_sched_in+0xfc ([kernel.kallsyms])
ffff00001020c0b4 visit_groups_merge+0x12c ([kernel.kallsyms])
main 1579 100 instructions:
ffff000010219344 perf_swevent_add+0x6c ([kernel.kallsyms])
ffff0000102135f4 event_sched_in.isra.57+0xbc ([kernel.kallsyms])
ffff0000102137a0 group_sched_in+0x60 ([kernel.kallsyms])
ffff000010213b84 flexible_sched_in+0xfc ([kernel.kallsyms])
ffff00001020c0b4 visit_groups_merge+0x12c ([kernel.kallsyms])
[...]
Changes from v1:
* Added comments for task thread handling (Mathieu).
* Split patch 02 into two patches, one is for support thread stack and
another is for callchain support (Mathieu).
* Added a new patch to support branch filter.
Leo Yan (5):
perf cs-etm: Refactor instruction size handling
perf cs-etm: Support thread stack
perf cs-etm: Support branch filter
perf cs-etm: Support callchain for instruction sample
perf cs-etm: Correct callchain for instruction sample
tools/perf/util/cs-etm.c | 141 ++++++++++++++++++++++++++++++++-------
1 file changed, 118 insertions(+), 23 deletions(-)
--
2.17.1
Some hardware will ignore bit TRCPDCR.PU which is used to signal
to hardware that power should not be removed from the trace unit.
Let's mitigate against this by conditionally saving and restoring
the trace unit state when the CPU enters low power states.
This patchset introduces a firmware property named
'arm,coresight-loses-context-with-cpu' - when this is present the
hardware state will be conditionally saved and restored.
A module parameter 'pm_save_enable' is also introduced which can
be configured to override the firmware property. This parameter
also provides a means to save/restore state when external agents
are used.
The hardware state is only ever saved and restored when a coresight
session is present.
Changes since v5:
- Fix indentation, comment style and add implicit fallthrough comment
- Use NOTIFY_* for all return values in pm notifier callback
- Rename PARAM_PM_SAVE_EXTERNAL to PARAM_PM_SAVE_ALWAYS
- Update module parameter description
- Add comment to explain why we keep power on
- Rebased onto coresight/next c165d8947bc4 ("eeprom: Deprecate the legacy eeprom driver")
Changes since v4:
- Rename fwnode property to "arm,coresight-loses-context-with-cpu" as this
doesn't imply a software policy
- Update the device tree binding document to indicate that this property
isn't specific to ETMs - also provide a longer description more generic
description with an example of why it might be used
- Set the module parameter at probe based on the value determined by firmware.
The user can still override the firmware via the kernel command line, this
has the effect of hiding the PARAM_PM_SAVE_FIRMWARE option from the user -
though we still internally use it to allow us to determine if the user has
set the parameter.
- Remove unnecessary call to smp_processor_id
- Move etm4_needs_save_restore helper to coresight.c and rename
- Rebased onto coresight/next a04d8683f577 ("...ity of etm4_os_unlock comment")
- Drop Reviewed-By from Suzuki on "coresight: etm4x: save/restore st..." patch
as content changed too much
- Add module option to that keeps clocks/power enabled at probe and saves
state when external or self-hosted is in use.
Changes since v3:
- Only save/restore when self-hosted is being used and detect this
without relying on the coresight registers (which may not be
available)
- Only allocate memory for etmv4_save_state at probe time when
configuration indicates it may be used
- Set pm_save_enable param to 0444 such that it is static after
boot
- Save/restore TRCPDCR
- Add missing comments to struct etm4_drvdata documentation
- Rebased onto coresight/next (8f1f9857)
Changes since v2:
- Move the PM notifier block from drvdata to file static
- Add section names to document references
- Add additional information to commit messages
- Remove trcdvcvr and trcdvcmr from saved state and add a comment to
describe why
- Ensure TRCPDCR_PU is set after restore and add a comment to explain
why we bother toggling TRCPDCR_PU on save/restore
- Reword the pm_save_enable options and add comments
- Miscellaneous style changes
- Move device tree binding documentation to its own patch
Changes since v1:
- Rebased onto coresight/next
- Correcly pass bit number rather than BIT macro to coresight_timeout
- Abort saving state if a timeout occurs
- Fix completely broken pm_notify handling and unregister handler on error
- Use state_needs_restore to ensure state is restored only once
- Add module parameter description to existing boot_enable parameter
and use module_param instead of module_param_named
- Add firmware bindings for coresight-needs-save-restore
- Rename 'disable_pm_save' to 'pm_save_enable' which allows for
disabled, enabled or firmware
- Update comment on etm4_os_lock, it incorrectly indicated that
the code unlocks the trace registers
- Add comments to explain use of OS lock during save/restore
- Fix incorrect error description whilst waiting for PM stable
- Add WARN_ON_ONCE when cpu isn't as expected during save/restore
- Various updates to commit messages
Andrew Murray (3):
coresight: etm4x: save/restore state across CPU low power states
dt-bindings: arm: coresight: Add support for
coresight-loses-context-with-cpu
coresight: etm4x: save/restore state for external agents
.../devicetree/bindings/arm/coresight.txt | 9 +
drivers/hwtracing/coresight/coresight-etm4x.c | 351 +++++++++++++++++-
drivers/hwtracing/coresight/coresight-etm4x.h | 64 ++++
drivers/hwtracing/coresight/coresight.c | 8 +-
include/linux/coresight.h | 13 +
5 files changed, 443 insertions(+), 2 deletions(-)
--
2.21.0
Hello,
I have had some success streaming instruction trace to my Linux PC using RDDI (from ARM Development studio 2019) and a DStreamer ST. However I am also
having some problems:
1. Failure to stop trace cleanly
I have enabled streaming from an ETM attached to a Cortex-R5 core.
As described in the RDDI documentation, to stop streaming, I call StreamingTrace_Flush() and wait for
RDDI_STREAMING_TRACE_EVENT_TYPE_END_OF_DATA event.
Next i call StreamingTrace_Stop() which succeeds and then try StreamingTrace_Detach() which seems to fail with the following assert.
tpp.c:84: __pthread_tpp_change_priority: Assertion `new_prio == -1 || (new_prio >= fifo_min_prio && new_prio <= fifo_max_prio)' failed.
Aborted (core dumped)
If i try to avoid calling StreamingTrace_{Detach, Stop} then Debug_CloseConn to the core never returns and i get a seg fault.
2. Raw trace does not start with A-SYNC bits
I do see A-SYNC bits later but my understanding was ETM always sends the A-SYNC bits first.
Cheers,
raj
Some hardware will ignore bit TRCPDCR.PU which is used to signal
to hardware that power should not be removed from the trace unit.
Let's mitigate against this by conditionally saving and restoring
the trace unit state when the CPU enters low power states.
This patchset introduces a firmware property named
'arm,coresight-loses-context-with-cpu' - when this is present the
hardware state will be conditionally saved and restored.
A module parameter 'pm_save_enable' is also introduced which can
be configured to override the firmware property. This parameter
also provides a means to save/restore state when external agents
are used.
The hardware state is only ever saved and restored when a coresight
session is present.
The last patch should be considered as an RFC as further consideration is
required relating to where the pm_save_enable parameter lives and to
determine if the external agent support should be a pm_save_enable option
or a new kernel option.
Changes since v4:
- Rename fwnode property to "arm,coresight-loses-context-with-cpu" as this
doesn't imply a software policy
- Update the device tree binding document to indicate that this property
isn't specific to ETMs - also provide a longer description more generic
description with an example of why it might be used
- Set the module parameter at probe based on the value determined by firmware.
The user can still override the firmware via the kernel command line, this
has the effect of hiding the PARAM_PM_SAVE_FIRMWARE option from the user -
though we still internally use it to allow us to determine if the user has
set the parameter.
- Remove unnecessary call to smp_processor_id
- Move etm4_needs_save_restore helper to coresight.c and rename
- Rebased onto coresight/next a04d8683f577 ("...ity of etm4_os_unlock comment")
- Drop Reviewed-By from Suzuki on "coresight: etm4x: save/restore st..." patch
as content changed too much
- Add module option to that keeps clocks/power enabled at probe and saves
state when external or self-hosted is in use.
Changes since v3:
- Only save/restore when self-hosted is being used and detect this
without relying on the coresight registers (which may not be
available)
- Only allocate memory for etmv4_save_state at probe time when
configuration indicates it may be used
- Set pm_save_enable param to 0444 such that it is static after
boot
- Save/restore TRCPDCR
- Add missing comments to struct etm4_drvdata documentation
- Rebased onto coresight/next (8f1f9857)
Changes since v2:
- Move the PM notifier block from drvdata to file static
- Add section names to document references
- Add additional information to commit messages
- Remove trcdvcvr and trcdvcmr from saved state and add a comment to
describe why
- Ensure TRCPDCR_PU is set after restore and add a comment to explain
why we bother toggling TRCPDCR_PU on save/restore
- Reword the pm_save_enable options and add comments
- Miscellaneous style changes
- Move device tree binding documentation to its own patch
Changes since v1:
- Rebased onto coresight/next
- Correcly pass bit number rather than BIT macro to coresight_timeout
- Abort saving state if a timeout occurs
- Fix completely broken pm_notify handling and unregister handler on error
- Use state_needs_restore to ensure state is restored only once
- Add module parameter description to existing boot_enable parameter
and use module_param instead of module_param_named
- Add firmware bindings for coresight-needs-save-restore
- Rename 'disable_pm_save' to 'pm_save_enable' which allows for
disabled, enabled or firmware
- Update comment on etm4_os_lock, it incorrectly indicated that
the code unlocks the trace registers
- Add comments to explain use of OS lock during save/restore
- Fix incorrect error description whilst waiting for PM stable
- Add WARN_ON_ONCE when cpu isn't as expected during save/restore
- Various updates to commit messages
Andrew Murray (3):
coresight: etm4x: save/restore state across CPU low power states
dt-bindings: arm: coresight: Add support for
coresight-loses-context-with-cpu
coresight: etm4x: save/restore state for external agents
.../devicetree/bindings/arm/coresight.txt | 9 +
drivers/hwtracing/coresight/coresight-etm4x.c | 339 +++++++++++++++++-
drivers/hwtracing/coresight/coresight-etm4x.h | 64 ++++
drivers/hwtracing/coresight/coresight.c | 8 +-
include/linux/coresight.h | 13 +
5 files changed, 431 insertions(+), 2 deletions(-)
--
2.21.0