CoreSight

coresight@lists.linaro.org

2 participants
2382 discussions

by Jeremy Ng

Hi Mathieu, I will like to ask how I might be able to configure the options in trcconfig driver file, found in /sys/bus/coresight/devices/*.etm/mgmt/ . For example, the trcidr0 denotes that tssize = 64bits (which indicates that timestamps are supported), retstack, cc, bb are also supported. However, my trcconfig file shows that none of the options are activated. In the *.etm driver files, it has the following files in it that I thought might have been linked to trcconfig, or at least be related to activating some of the options that are currently activated: 1. bb_ctrl (obviously bb) 2. cntr_ctrl 3. cyc_threshold (cc?) 4. event_ts (ts) I am hoping that help in activating these options can be provided. On another note, I will also like to ask if there might be example files that I may refer to to using sequencer/resource/events. In the *.etm file, there are seq_event,seq_idx,seq_reset_event,seq_state/res_ctrl, res_idx/event,event_instren,event_ts,event_vinst in the respective order (seq/res/events). I will like to understand how I might use these features to improve my tracing results. As well as understanding the various values I could apply to the mode file. As quoted in sysfs-bus-coresight-devices-etm4x, What: /sys/bus/coresight/devices/<memory_map>.etm/mode Date: April 2015 KernelVersion: 4.01 Contact: Mathieu Poirier <mathieu.poirier(a)linaro.org> Description: (RW) Controls various modes supported by this ETM, for example P0 instruction tracing, branch broadcast, cycle counting and context ID tracing. However it does not explain how that file can control various modes. I have tried toying with this files and it went from useful tracing results to just kernel instruction tracing results. After applying 0x0 to the mode again (which was the default value when i start up my device), I am unable to re-obtain useful trace results and had to resort to re-power my device. Lastly, I understand that nr_ss_cmp meant that my etm can only have 1 single-shot comparator during a trace (correct me if I am wrong). This explains why when I applied 0x3 to addr_idx and set addr_start value, I am unable to set a value for addr_stop. On the other hand,i have 0x4 nr_addr_cmp. However, I do not understand how I might be able to use the other 2 addr comparator (i.e. when addr_idx = 0, I am able to set addr_start and addr_stop, which took up 2 addr_cmp. How then, might I be able to use the other 2 addr_cmp?) My apologies for many amateurish questions. I have been looking at the etm documentation (IHI0064F), particularly on comparators, bb, ts, events, resources but unfortunately unable to connect the dots on how I might be able to use them with the coresight drivers. Yours Sincerely, Jeremy Ng

5 years, 10 months

[PATCH v4 2/2] perf machine: arm/arm64: Improve completeness for kernel address space

by Leo Yan

Arm and arm64 architecture reserve some memory regions prior to the symbol '_stext' and these memory regions later will be used by device module and BPF jit. The current code misses to consider these memory regions thus any address in the regions will be taken as user space mode, but perf cannot find the corresponding dso with the wrong CPU mode so we misses to generate samples for device module and BPF related trace data. This patch parse the link scripts to get the memory size prior to start address and reduce this size from 'machine>->kernel_start', then can get a fixed up kernel start address which contain memory regions for device module and BPF. Finally, machine__get_kernel_start() can reflect more complete kernel memory regions and perf can successfully generate samples. The reason for parsing the link scripts is Arm architecture changes text offset dependent on different platforms, which define multiple text offsets in $kernel/arch/arm/Makefile. This offset is decided when build kernel and the final value is extended in the link script, so we can extract the used value from the link script. We use the same way to parse arm64 link script as well. If fail to find the link script, the pre start memory size is assumed as zero, in this case it has no any change caused with this patch. Below is detailed info for testing this patch: - Install or build LLVM/Clang; - Configure perf with ~/.perfconfig: root@debian:~# cat ~/.perfconfig # this file is auto-generated. [llvm] clang-path = /mnt/build/llvm-build/build/install/bin/clang kbuild-dir = /mnt/linux-kernel/linux-cs-dev/ clang-opt = "-g" dump-obj = true [trace] show_zeros = yes show_duration = no no_inherit = yes show_timestamp = no show_arg_names = no args_alignment = 40 show_prefix = yes - Run 'perf trace' command with eBPF event: root@debian:~# perf trace -e string \ -e $kernel/tools/perf/examples/bpf/augmented_raw_syscalls.c - Read eBPF program memory mapping in kernel: root@debian:~# echo 1 > /proc/sys/net/core/bpf_jit_kallsyms root@debian:~# cat /proc/kallsyms | grep -E "bpf_prog_.+_sys_[enter|exit]" ffff00000008a0d0 t bpf_prog_e470211b846088d5_sys_enter [bpf] ffff00000008c6a4 t bpf_prog_29c7ae234d79bd5c_sys_exit [bpf] - Launch any program which accesses file system frequently so can hit the system calls trace flow with eBPF event; - Capture CoreSight trace data with filtering eBPF program: root@debian:~# perf record -e cs_etm/@tmc_etr0/ \ --filter 'filter 0xffff00000008a0d0/0x800' -a sleep 5s - Decode the eBPF program symbol 'bpf_prog_f173133dc38ccf87_sys_enter': root@debian:~# perf script -F,ip,sym Frame deformatter: Found 4 FSYNCS 0 [unknown] ffff00000008a1ac bpf_prog_e470211b846088d5_sys_enter ffff00000008a250 bpf_prog_e470211b846088d5_sys_enter 0 [unknown] ffff00000008a124 bpf_prog_e470211b846088d5_sys_enter 0 [unknown] ffff00000008a14c bpf_prog_e470211b846088d5_sys_enter ffff00000008a13c bpf_prog_e470211b846088d5_sys_enter ffff00000008a14c bpf_prog_e470211b846088d5_sys_enter 0 [unknown] ffff00000008a180 bpf_prog_e470211b846088d5_sys_enter 0 [unknown] ffff00000008a1ac bpf_prog_e470211b846088d5_sys_enter ffff00000008a190 bpf_prog_e470211b846088d5_sys_enter ffff00000008a1ac bpf_prog_e470211b846088d5_sys_enter ffff00000008a250 bpf_prog_e470211b846088d5_sys_enter 0 [unknown] ffff00000008a124 bpf_prog_e470211b846088d5_sys_enter 0 [unknown] ffff00000008a14c bpf_prog_e470211b846088d5_sys_enter 0 [unknown] ffff00000008a180 bpf_prog_e470211b846088d5_sys_enter [...] Cc: Mathieu Poirier <mathieu.poirier(a)linaro.org> Cc: Alexander Shishkin <alexander.shishkin(a)linux.intel.com> Cc: Jiri Olsa <jolsa(a)redhat.com> Cc: Namhyung Kim <namhyung(a)kernel.org> Cc: Peter Zijlstra <peterz(a)infradead.org> Cc: Suzuki Poulouse <suzuki.poulose(a)arm.com> Cc: coresight(a)lists.linaro.org Cc: linux-arm-kernel(a)lists.infradead.org Signed-off-by: Leo Yan <leo.yan(a)linaro.org> --- tools/perf/Makefile.config | 22 ++++++++++++++++++++++ tools/perf/arch/arm/util/Build | 2 ++ tools/perf/arch/arm/util/machine.c | 17 +++++++++++++++++ tools/perf/arch/arm64/util/Build | 1 + tools/perf/arch/arm64/util/machine.c | 17 +++++++++++++++++ 5 files changed, 59 insertions(+) create mode 100644 tools/perf/arch/arm/util/machine.c create mode 100644 tools/perf/arch/arm64/util/machine.c diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config index e4988f49ea79..76e0ad0b4fd2 100644 --- a/tools/perf/Makefile.config +++ b/tools/perf/Makefile.config @@ -51,6 +51,17 @@ endif ifeq ($(SRCARCH),arm) NO_PERF_REGS := 0 LIBUNWIND_LIBS = -lunwind -lunwind-arm + PRE_START_SIZE := 0 + ifneq ($(wildcard $(srctree)/arch/$(SRCARCH)/kernel/vmlinux.lds),) + # Extract info from lds: + # . = ((0xC0000000)) + 0x00208000; + # PRE_START_SIZE := 0x00208000 + PRE_START_SIZE := $(shell egrep ' \. \= ${2}0x[0-9a-fA-F]+${2}' \ + $(srctree)/arch/$(SRCARCH)/kernel/vmlinux.lds | \ + sed -e 's/[(|)|.|=|+|<|;|-]//g' -e 's/ \+/ /g' -e 's/^[ \t]*//' | \ + awk -F' ' '{printf "0x%x", $$2}' 2>/dev/null) + endif + CFLAGS += -DARM_PRE_START_SIZE=$(PRE_START_SIZE) endif ifeq ($(SRCARCH),arm64) @@ -58,6 +69,17 @@ ifeq ($(SRCARCH),arm64) NO_SYSCALL_TABLE := 0 CFLAGS += -I$(OUTPUT)arch/arm64/include/generated LIBUNWIND_LIBS = -lunwind -lunwind-aarch64 + PRE_START_SIZE := 0 + ifneq ($(wildcard $(srctree)/arch/$(SRCARCH)/kernel/vmlinux.lds),) + # Extract info from lds: + # . = ((((((((0xffffffffffffffff)) - (((1)) << (48)) + 1) + (0)) + (0x08000000))) + (0x08000000))) + 0x00080000; + # PRE_START_SIZE := (0x08000000 + 0x08000000 + 0x00080000) = 0x10080000 + PRE_START_SIZE := $(shell egrep ' \. \= ${8}0x[0-9a-fA-F]+${2}' \ + $(srctree)/arch/$(SRCARCH)/kernel/vmlinux.lds | \ + sed -e 's/[(|)|.|=|+|<|;|-]//g' -e 's/ \+/ /g' -e 's/^[ \t]*//' | \ + awk -F' ' '{printf "0x%x", $$6+$$7+$$8}' 2>/dev/null) + endif + CFLAGS += -DARM_PRE_START_SIZE=$(PRE_START_SIZE) endif ifeq ($(SRCARCH),csky) diff --git a/tools/perf/arch/arm/util/Build b/tools/perf/arch/arm/util/Build index 296f0eac5e18..efa6b768218a 100644 --- a/tools/perf/arch/arm/util/Build +++ b/tools/perf/arch/arm/util/Build @@ -1,3 +1,5 @@ +perf-y += machine.o + perf-$(CONFIG_DWARF) += dwarf-regs.o perf-$(CONFIG_LOCAL_LIBUNWIND) += unwind-libunwind.o diff --git a/tools/perf/arch/arm/util/machine.c b/tools/perf/arch/arm/util/machine.c new file mode 100644 index 000000000000..db172894e4ea --- /dev/null +++ b/tools/perf/arch/arm/util/machine.c @@ -0,0 +1,17 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/types.h> +#include <linux/string.h> +#include <stdlib.h> + +#include "../../util/machine.h" + +void arch__fix_kernel_text_start(u64 *start) +{ + /* + * On arm, the 16MB virtual memory space prior to 'kernel_start' is + * allocated to device modules, a PMD table if CONFIG_HIGHMEM is + * enabled and a PGD table. To reflect the complete kernel address + * space, compensate the pre-defined regions for kernel start address. + */ + *start = *start - ARM_PRE_START_SIZE; +} diff --git a/tools/perf/arch/arm64/util/Build b/tools/perf/arch/arm64/util/Build index 3cde540d2fcf..8081fb8a7b3d 100644 --- a/tools/perf/arch/arm64/util/Build +++ b/tools/perf/arch/arm64/util/Build @@ -1,4 +1,5 @@ perf-y += header.o +perf-y += machine.o perf-y += sym-handling.o perf-$(CONFIG_DWARF) += dwarf-regs.o perf-$(CONFIG_LOCAL_LIBUNWIND) += unwind-libunwind.o diff --git a/tools/perf/arch/arm64/util/machine.c b/tools/perf/arch/arm64/util/machine.c new file mode 100644 index 000000000000..61058dca8c5a --- /dev/null +++ b/tools/perf/arch/arm64/util/machine.c @@ -0,0 +1,17 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/types.h> +#include <linux/string.h> +#include <stdlib.h> + +#include "../../util/machine.h" + +void arch__fix_kernel_text_start(u64 *start) +{ + /* + * On arm64, the root PGD table, device module memory region and + * BPF jit region are prior to 'kernel_start'. To reflect the + * complete kernel address space, compensate these pre-defined + * regions for kernel start address. + */ + *start = *start - ARM_PRE_START_SIZE; +} -- 2.17.1

5 years, 10 months

Re: coresight@lists.linaro.org

by Jeremy Ng

Hi Mike, Thank you for your reply! On Thu, Aug 1, 2019 at 1:09 AM Mike Leach <mike.leach(a)linaro.org> wrote: > > Hi Jeremy, > > The trc_pkt_lister app uses a mapping from core name to determine architecture version and core profile. A73 was not included inthe version of the library you are using, but it has been updated in v0.12.0 released today. > > Thank you for updating me on this information. I have tried it with Cortex-A73 and it works! > > On Thu, 25 Jul 2019 at 10:44, Jeremy Ng <ngyzj95(a)gmail.com> wrote: >> >> Dear Mike, >> >> I will like to ask you for guidance on using openCSD. >> >> I was trying to use and understand trc_pkt_lister from demos >> and notice that snapshot of the device is required >> (hikey960 device is used here). >> >> I have also tried looking at the programmer's guide for >> openCSD but I am confused with what I am required to do. >> >> Nevertheless, I tried running trc_pkt_lister in the hikey960 >> snapshot folder and it gave me this error: >> >> Trace Packet Lister : reading snapshot from path ./ >> Using ETR_0 as trace source >> ss2_dcdtree : 0x0025 (OCSD_ERR_TEST_SS_TO_DECODER) [test snapshot to decode tree conversion error]; Unrecognized Core name Cortex-A73. Cannot evaluate profile or architecture.ss2_dcdtree : 0x0025 (OCSD_ERR_TEST_SS_TO_DECODER) [test snapshot to decode tree conversion error]; Failed to create decoder for source ETM_4. >> ss2_dcdtree : 0x0025 (OCSD_ERR_TEST_SS_TO_DECODER) [test snapshot to decode tree conversion error]; Unrecognized Core name Cortex-A73. Cannot evaluate profile or architecture.ss2_dcdtree : 0x0025 (OCSD_ERR_TEST_SS_TO_DECODER) [test snapshot to decode tree conversion error]; Failed to create decoder for source ETM_5. >> ss2_dcdtree : 0x0025 (OCSD_ERR_TEST_SS_TO_DECODER) [test snapshot to decode tree conversion error]; Unrecognized Core name Cortex-A73. Cannot evaluate profile or architecture.ss2_dcdtree : 0x0025 (OCSD_ERR_TEST_SS_TO_DECODER) [test snapshot to decode tree conversion error]; Failed to create decoder for source ETM_6. >> ss2_dcdtree : 0x0025 (OCSD_ERR_TEST_SS_TO_DECODER) [test snapshot to decode tree conversion error]; Unrecognized Core name Cortex-A73. Cannot evaluate profile or architecture.ss2_dcdtree : 0x0025 (OCSD_ERR_TEST_SS_TO_DECODER) [test snapshot to decode tree conversion error]; Failed to create decoder for source ETM_7. >> Trace Packet Lister : Protocol printer ETMV4I on Trace ID 0x10 >> Trace Packet Lister : Protocol printer ETMV4I on Trace ID 0x12 >> Trace Packet Lister : Protocol printer ETMV4I on Trace ID 0x14 >> Trace Packet Lister : Protocol printer ETMV4I on Trace ID 0x16 >> Trace Packet Lister : Error : Unable to open trace buffer >> >> >> I am not exactly sure if there is a proper software to >> perform snapshots/kernel dump/reading registers values. >> I also thought it worthy to note that the snapshot files >> are made in reference to the juno-uname-002 demo project. >> >> >> On another note, I will like to ask if it is possible >> to use openCSD to decode just the trace binary dumps >> without the need for snapshots. > > > If you write your own application to use the library then you can specify whatever input format you wish to provide the correct data for the library. > > What level of decode are you looking for? If you want to simply convert the trace binaries into simple packet listings like ptmtohuman, then far less information is needed. For a full trace decode however both configuration information and program images are required for decode. > > The snapshots provide a format that manages this information and is easy to use for the library test programs such as trc_pkt_lister. May I inquire how I might go about doing that? I am have gotten a snapshot of my device, thanks to Leo Yan, however, I will like to understand how the information in the snapshot was acquired. Especially the following values: In cpu_0.ini: 1) ``` [regs] PC(size:64)=0xFFFFFFC000081000 SP(size:64)=0 SCTLR_EL1=0x1007 CPSR=0x1C5 ``` How was the value for PC, SP, SPCTLR_EL1 and CPSR acquired? 2) ``` [dump1] file=kernel_dump.bin address=0xFFFFFFC000081000 length=0x00050000 ``` How do I acquire kernel_dump.bin and how was the address and length decided? My apologies for the amateurish questions. Regards, Jeremy > > Regrds > > Mike > > >> >> Any assistance will be appreciated >> >> Cheers, >> Jeremy > > -- > Mike Leach > Principal Engineer, ARM Ltd. > Manchester Design Centre. UK

5 years, 10 months

Release of OpenCSD 0.12.0 and development branch 0.13.0-dev

by Mike Leach

Hi, The OpenCSD library is released today @ version 0.12.0 Key updates include:- i) Correction and enhancement of ETMv4 Exception packets. The ETMv4 exception packet contains a preferred return address. This address can also be a target address for the preceding branch if the exception was taken after a branch but before any instructions at the branch target was taken. This is implied by the ordering of the output generic elements in previous versions of the library - but this version has an explicit flag to indicate when this occurs. The intention here is to make it easier for clients that such as perf that are creating lists of branch source/target pairs to correctly spot this event. The preferred return address can also contain a context element in the circumstances that it is also a target address. This was not being emitted correctly by the library. This has been corrected. ii) Support for trace source captures that come from TPIU - these will include FSYNC bytes that need to be handled by the library. Update to test program and snapshot to demo this. iii) Various minor bugfixes and documentation updates to support the changes above. The OpenCSD dev-0v13v0 branch is created. This contains support not previously available in the library, to allow the processing of ETMv4 Q packets. This feature is still undergoing testing but is being made available should any users have a target that can enable these packets. Support for speculative trace will be made available on this branch in due course. Regards Mike -- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK

5 years, 11 months

[PATCH] coresight: acpi: Static funnel support

by Suzuki K Poulose

The ACPI bindings for CoreSight has been updated to add the device id for non-programmable CoreSight funnels (aka static funnels) as of v1.1 [0]. Add the ACPI id for static funnels in the driver. [0] https://static.docs.arm.com/den0067/a/DEN0067_CoreSight_ACPI_1.1.pdf Signed-off-by: Suzuki K Poulose <suzuki.poulose(a)arm.com> --- drivers/hwtracing/coresight/coresight-funnel.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/hwtracing/coresight/coresight-funnel.c b/drivers/hwtracing/coresight/coresight-funnel.c index fa97cb9ab4f9..0c99848a5d69 100644 --- a/drivers/hwtracing/coresight/coresight-funnel.c +++ b/drivers/hwtracing/coresight/coresight-funnel.c @@ -5,6 +5,7 @@ * Description: CoreSight Funnel driver */ +#include <linux/acpi.h> #include <linux/kernel.h> #include <linux/init.h> #include <linux/types.h> @@ -302,11 +303,19 @@ static const struct of_device_id static_funnel_match[] = { {} }; +#ifdef CONFIG_ACPI +static const struct acpi_device_id static_funnel_ids[] = { + {"ARMHC9FE", 0}, + {}, +}; +#endif + static struct platform_driver static_funnel_driver = { .probe = static_funnel_probe, .driver = { .name = "coresight-static-funnel", .of_match_table = static_funnel_match, + .acpi_match_table = ACPI_PTR(static_funnel_ids), .pm = &funnel_dev_pm_ops, .suppress_bind_attrs = true, }, -- 2.21.0

5 years, 11 months

[PATCH v3 0/6] coresight: etm4x: save/restore ETMv4 context across CPU low power states

by Andrew Murray

Some hardware will ignore bit TRCPDCR.PU which is used to signal to hardware that power should not be removed from the trace unit. Let's mitigate against this by conditionally saving and restoring the trace unit state when the CPU enters low power states. This patchset introduces a firmware property named 'arm,coresight-needs-save-restore' - when this is present the hardware state will be conditionally saved and restored. A module parameter 'pm_save_enable' is also introduced which can be configured to override the firmware property. The hardware state is only ever saved and restored when the claim tags indicate that coresight is in use. Changes since v2: - Move the PM notifier block from drvdata to file static - Add section names to document references - Add additional information to commit messages - Remove trcdvcvr and trcdvcmr from saved state and add a comment to describe why - Ensure TRCPDCR_PU is set after restore and add a comment to explain why we bother toggling TRCPDCR_PU on save/restore - Reword the pm_save_enable options and add comments - Miscellaneous style changes - Move device tree binding documentation to its own patch Changes since v1: - Rebased onto coresight/next - Correcly pass bit number rather than BIT macro to coresight_timeout - Abort saving state if a timeout occurs - Fix completely broken pm_notify handling and unregister handler on error - Use state_needs_restore to ensure state is restored only once - Add module parameter description to existing boot_enable parameter and use module_param instead of module_param_named - Add firmware bindings for coresight-needs-save-restore - Rename 'disable_pm_save' to 'pm_save_enable' which allows for disabled, enabled or firmware - Update comment on etm4_os_lock, it incorrectly indicated that the code unlocks the trace registers - Add comments to explain use of OS lock during save/restore - Fix incorrect error description whilst waiting for PM stable - Add WARN_ON_ONCE when cpu isn't as expected during save/restore - Various updates to commit messages Andrew Murray (6): coresight: etm4x: remove superfluous setting of os_unlock coresight: etm4x: use explicit barriers on enable/disable coresight: etm4x: use module_param instead of module_param_named coresight: etm4x: improve clarity of etm4_os_unlock comment coresight: etm4x: save/restore state across CPU low power states dt-bindings: arm: coresight: Add support for coresight-needs-save-restore .../devicetree/bindings/arm/coresight.txt | 3 + drivers/hwtracing/coresight/coresight-etm4x.c | 348 +++++++++++++++++- drivers/hwtracing/coresight/coresight-etm4x.h | 62 ++++ drivers/hwtracing/coresight/coresight.c | 2 +- include/linux/coresight.h | 8 + 5 files changed, 415 insertions(+), 8 deletions(-) -- 2.21.0

5 years, 11 months

Coresight Memory Mapped Addresses in Hikey960

by Jeremy Ng

Dear Leo Yan, I will like to ask if there are any better documentations that detail the memory mapped addresses for Coresight registers in Hikey960. I am currently referring to Chapter 2-9-1 in http://mirror.lemaker.org/HiKey960_SoC_Reference_Manual.pdf. This is all the information about coresight addresses that I could retrieve: 0xEC000000 0xED7FFFFF 24M CSSYS_APB I will like to check if there are any datasheet for hikey960 that is as detailed as that for Juno device (link here: http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0515b/CHDIFA… ). This is important to me as I am trying to acquire the device snapshot by using CSAL API. I am not sure if there are other ways to do this since the documentation hasn't been exactly straightforward. The reason for wanting the snapshot is to possibly run openCSD and decode my trace binary files. I have followed the CSAL build documentations and ran `csls` on my hikey960. The following output is seen but it crashed my device soon after: ```bash hikey960:/ # csls ** CSLS: listing CoreSight config... ** CSLS: Using default ROM address 0xEC000000 00EC031000: 2.1 908 00000037 00/0F type= 4 - LINK [FUNNEL: 7 in ports] 00EC032000: 1.1 912 000000A0 00/0F type= 8 - SINK PORT [TPIU] 00EC033000: 1.2 961 00001B40 00/0F type= 7 - SINK BUFFER(ETR r/w size: 4K) [TMC: ETR configuration] 00EC034000: 4.1 906 00040800 00/0F type=10 - CTI 00EC035000: 3.6 963 00010000 00/0F type= 3 - SOURCE SWSTIM(65536) [STM ext ports only, 64-bit, 128 masters] 00EC036000: 2.3 961 00000480 00/0F type= 6 - LINK SINK BUFFER(4K) [TMC: ETF configuration] 00EC037000: type=13 O TIMESTAMP 00EC038000: type=13 O TIMESTAMP ``` I will also like to understand why it might have crashed my device during the execution of csls. Please assist. Yours Sincerely, Jeremy

5 years, 11 months

[PATCH 0/5] coresight: tmc error handling and misc fixes

by Suzuki K Poulose

This series is a collection of fixes and cleanups I gathered from trying to get coresight up on a new platform. The TMC-ETR reports MemErr in the status register if there was an error in in the AXI transaction. So far we have ignored it and assumed that we are running on perfect platforms. Let us add the support for handling the MemErr reports and discard the buffer in such case. Also verify that the ETR can do non-secure transactions on the platform at probe time, in order to avoid presenting the user with a non-useable ETR. Suzuki K Poulose (5): coresight: Fix DEBUG_LOCKS_WARN_ON for uninitialized attribute coresight: funnel: Convert pr_warn to dev_warn for obsolete bindings coresight: etr_buf: Consolidate refcount initialization coresight: tmc-etr: Handle memory errors coresight: tmc-etr: Check if non-secure access is enabled .../hwtracing/coresight/coresight-etm-perf.c | 1 + .../hwtracing/coresight/coresight-funnel.c | 2 +- .../hwtracing/coresight/coresight-tmc-etr.c | 26 +++++++++++-------- drivers/hwtracing/coresight/coresight-tmc.c | 12 +++++++++ drivers/hwtracing/coresight/coresight-tmc.h | 4 +++ 5 files changed, 33 insertions(+), 12 deletions(-) -- 2.21.0

5 years, 11 months

Request for Guidance in Using sysFS for Coresight in HiKey960 (Cortex A53/A73)

by Student - Ng Yi Zher Jeremy

Dear Sir, My apologies for spamming you with more, perhaps ridiculous, questions... I have been struggling to understand the systematic approach to conduct system traces with sysFS. I have asked similar question to hwangcc23, but I feel this question is suited more for the Linaro team. I am having trouble choosing the options for the coresight drivers in my Hikey960 device with Cortex A53 and A73 processors, 4 CPUs each. I have created a simple program that is a quick implementation of a link list in C program. The following is the objdump of the main object: ```bash ... ... 00000000000009e8 <main>: 9e8: d10583ff sub sp, sp, #0x160 9ec: f900a3fc str x28, [sp,#320] 9f0: a9157bfd stp x29, x30, [sp,#336] 9f4: 910543fd add x29, sp, #0x150 9f8: 321e03e8 orr w8, wzr, #0x4 9fc: 52800009 mov w9, #0x0 // #0 a00: b27903e2 orr x2, xzr, #0x80 a04: d280000a mov x10, #0x0 // #0 a08: b000008b adrp x11, 11000 <init+0x100f4> a0c: f947e96b ldr x11, [x11,#4048] a10: d280260c mov x12, #0x130 // #304 a14: 8b0c016c add x12, x11, x12 a18: 9000000d adrp x13, 0 <note_android_ident-0x218> a1c: 913f21ad add x13, x13, #0xfc8 a20: d280130e mov x14, #0x98 // #152 a24: 8b0e016b add x11, x11, x14 a28: 9000000e adrp x14, 0 <note_android_ident-0x218> a2c: 913e41ce add x14, x14, #0xf90 a30: 9000000f adrp x15, 0 <note_android_ident-0x218> a34: 913f09ef add x15, x15, #0xfc2 a38: 90000010 adrp x16, 0 <note_android_ident-0x218> a3c: 913dfa10 add x16, x16, #0xf7e a40: 90000011 adrp x17, 0 <note_android_ident-0x218> a44: 913db231 add x17, x17, #0xf6c a48: 9102a3f2 add x18, sp, #0xa8 a4c: b81ec3bf stur wzr, [x29,#-20] a50: b81e83a0 stur w0, [x29,#-24] a54: f81e03a1 stur x1, [x29,#-32] a58: b81dc3a8 stur w8, [x29,#-36] a5c: f9003bf2 str x18, [sp,#112] a60: f90037f1 str x17, [sp,#104] a64: f90033f0 str x16, [sp,#96] a68: b9005fe9 str w9, [sp,#92] a6c: f9002be2 str x2, [sp,#80] a70: f90027ea str x10, [sp,#72] a74: f90023ed str x13, [sp,#64] a78: f9001fec str x12, [sp,#56] a7c: f9001beb str x11, [sp,#48] a80: f90017ee str x14, [sp,#40] a84: f90013ef str x15, [sp,#32] a88: 97ffff8e bl 8c0 <getpid@plt> a8c: b81d83a0 stur w0, [x29,#-40] a90: f9403be0 ldr x0, [sp,#112] a94: b9405fe8 ldr w8, [sp,#92] a98: 2a0803e1 mov w1, w8 a9c: f9402be2 ldr x2, [sp,#80] aa0: 97ffffa4 bl 930 <memset@plt> aa4: b27603e8 orr x8, xzr, #0x400 aa8: b27e03e9 orr x9, xzr, #0x4 aac: f90053e9 str x9, [sp,#160] ab0: f94053e9 ldr x9, [sp,#160] ab4: eb08013f cmp x9, x8 ab8: 1a9f27ea cset w10, cc abc: 3700004a tbnz w10, #0, ac4 <main+0xdc> ac0: 14000010 b b00 <main+0x118> ac4: 9102a3e8 add x8, sp, #0xa8 ac8: b27d03e9 orr x9, xzr, #0x8 acc: b27a03ea orr x10, xzr, #0x40 ad0: b24003eb orr x11, xzr, #0x1 ad4: b24017ec orr x12, xzr, #0x3f ad8: f94053ed ldr x13, [sp,#160] adc: 8a0c01ac and x12, x13, x12 ae0: 9acc216b lsl x11, x11, x12 ae4: f94053ec ldr x12, [sp,#160] ae8: 9aca098a udiv x10, x12, x10 aec: 9b0a7d29 mul x9, x9, x10 af0: 8b090108 add x8, x8, x9 af4: f9400109 ldr x9, [x8] af8: aa0b0129 orr x9, x9, x11 afc: f9000109 str x9, [x8] b00: 9102a3e2 add x2, sp, #0xa8 b04: b27903e1 orr x1, xzr, #0x80 b08: b85d83a0 ldur w0, [x29,#-40] b0c: 97ffff79 bl 8f0 <sched_setaffinity@plt> b10: b9009fe0 str w0, [sp,#156] b14: b9409fe0 ldr w0, [sp,#156] b18: 35000040 cbnz w0, b20 <main+0x138> b1c: 1400000c b b4c <main+0x164> b20: 320003e0 orr w0, wzr, #0x1 b24: b9409fe8 ldr w8, [sp,#156] b28: b9001fe0 str w0, [sp,#28] b2c: b9001be8 str w8, [sp,#24] b30: 97ffff60 bl 8b0 <__errno@plt> b34: b9401be8 ldr w8, [sp,#24] b38: b9000008 str w8, [x0] b3c: f94037e0 ldr x0, [sp,#104] b40: 97ffff64 bl 8d0 <perror@plt> b44: b9401fe0 ldr w0, [sp,#28] b48: 97ffff6e bl 900 <exit@plt> b4c: 9102a3e2 add x2, sp, #0xa8 b50: b27903e1 orr x1, xzr, #0x80 b54: b85d83a0 ldur w0, [x29,#-40] b58: 97ffff62 bl 8e0 <sched_getaffinity@plt> b5c: b9009be0 str w0, [sp,#152] b60: b9409be0 ldr w0, [sp,#152] b64: 35000040 cbnz w0, b6c <main+0x184> b68: 1400000c b b98 <main+0x1b0> b6c: 320003e0 orr w0, wzr, #0x1 b70: b9409be8 ldr w8, [sp,#152] b74: b90017e0 str w0, [sp,#20] b78: b90013e8 str w8, [sp,#16] b7c: 97ffff4d bl 8b0 <__errno@plt> b80: b94013e8 ldr w8, [sp,#16] b84: b9000008 str w8, [x0] b88: f94033e0 ldr x0, [sp,#96] b8c: 97ffff51 bl 8d0 <perror@plt> b90: b94017e0 ldr w0, [sp,#20] b94: 97ffff5b bl 900 <exit@plt> b98: b27603e8 orr x8, xzr, #0x400 b9c: b27e03e9 orr x9, xzr, #0x4 ba0: f9004be9 str x9, [sp,#144] ba4: f9404be9 ldr x9, [sp,#144] ba8: eb08013f cmp x9, x8 bac: 1a9f27ea cset w10, cc bb0: 3700004a tbnz w10, #0, bb8 <main+0x1d0> bb4: 14000016 b c0c <main+0x224> bb8: 9102a3e8 add x8, sp, #0xa8 bbc: d2800009 mov x9, #0x0 // #0 bc0: b27d03ea orr x10, xzr, #0x8 bc4: b27a03eb orr x11, xzr, #0x40 bc8: b24003ec orr x12, xzr, #0x1 bcc: b24017ed orr x13, xzr, #0x3f bd0: f9404bee ldr x14, [sp,#144] bd4: 9acb09cb udiv x11, x14, x11 bd8: 9b0b7d4a mul x10, x10, x11 bdc: 8b0a0108 add x8, x8, x10 be0: f9400108 ldr x8, [x8] be4: f9404bea ldr x10, [sp,#144] be8: 8a0d014a and x10, x10, x13 bec: 9aca218a lsl x10, x12, x10 bf0: 8a0a0108 and x8, x8, x10 bf4: eb09011f cmp x8, x9 bf8: 1a9f07ef cset w15, ne bfc: 320003f0 orr w16, wzr, #0x1 c00: 0a1001ef and w15, w15, w16 c04: b9000fef str w15, [sp,#12] c08: 14000003 b c14 <main+0x22c> c0c: 52800008 mov w8, #0x0 // #0 c10: b9000fe8 str w8, [sp,#12] c14: b9400fe8 ldr w8, [sp,#12] c18: b9008fe8 str w8, [sp,#140] c1c: b9408fe8 ldr w8, [sp,#140] c20: 35000048 cbnz w8, c28 <main+0x240> c24: 14000028 b cc4 <main+0x2dc> c28: b27c03e0 orr x0, xzr, #0x10 c2c: 321e03e3 orr w3, wzr, #0x4 c30: b85d83a2 ldur w2, [x29,#-40] c34: f9401be8 ldr x8, [sp,#48] c38: f90003e0 str x0, [sp] c3c: aa0803e0 mov x0, x8 c40: f94017e1 ldr x1, [sp,#40] c44: 97ffff33 bl 910 <fprintf@plt> c48: f94027e8 ldr x8, [sp,#72] c4c: f90043e8 str x8, [sp,#128] c50: f94003e1 ldr x1, [sp] c54: aa0103e0 mov x0, x1 c58: 97ffff32 bl 920 <malloc@plt> c5c: f90043e0 str x0, [sp,#128] c60: f94043e0 ldr x0, [sp,#128] c64: 940000aa bl f0c <init> c68: b9007fff str wzr, [sp,#124] c6c: 52849f08 mov w8, #0x24f8 // #9464 c70: 72a00028 movk w8, #0x1, lsl #16 c74: b9407fe9 ldr w9, [sp,#124] c78: 6b08013f cmp w9, w8 c7c: 1a9fa7e8 cset w8, lt c80: 37000048 tbnz w8, #0, c88 <main+0x2a0> c84: 14000009 b ca8 <main+0x2c0> c88: 52800c28 mov w8, #0x61 // #97 c8c: f94043e0 ldr x0, [sp,#128] c90: 2a0803e1 mov w1, w8 c94: 94000016 bl cec <push> c98: b9407fe8 ldr w8, [sp,#124] c9c: 11000508 add w8, w8, #0x1 ca0: b9007fe8 str w8, [sp,#124] ca4: 17fffff2 b c6c <main+0x284> ca8: f94013e0 ldr x0, [sp,#32] cac: 97fffefd bl 8a0 <printf@plt> cb0: f94043e8 ldr x8, [sp,#128] cb4: aa0803e0 mov x0, x8 cb8: 94000065 bl e4c <print_list> cbc: b81ec3bf stur wzr, [x29,#-20] cc0: 14000006 b cd8 <main+0x2f0> cc4: 321e03e3 orr w3, wzr, #0x4 cc8: b85d83a2 ldur w2, [x29,#-40] ccc: f9401fe0 ldr x0, [sp,#56] cd0: f94023e1 ldr x1, [sp,#64] cd4: 97ffff0f bl 910 <fprintf@plt> cd8: b85ec3a0 ldur w0, [x29,#-20] cdc: a9557bfd ldp x29, x30, [sp,#336] ce0: f940a3fc ldr x28, [sp,#320] ce4: 910583ff add sp, sp, #0x160 ce8: d65f03c0 ret ... ... ``` What i did was look at the start address (d10583ff) of the main object and echo it into /sys/bus/coresight/devices/<memory_map>.etm/addr_start. Even though I was able to write into addr_start in sysfs, I was unable to read the value back despite it being readable. This can be seen in the following output: ```bash -rw-r--r-- 1 root root 4096 1970-01-01 00:11 ecc40000.etm/addr_start hikey960:/sys/bus/coresight/devices # echo d10583ff > ecc40000.etm/addr_start 1|hikey960:/sys/bus/coresight/devices # cat ecc40000.etm/addr_start cat: ecc40000.etm/addr_start: Operation not permitted ``` I thought it was not important that the Operation was 'not permitted'. Nonetheless, I continued with the tutorial as depicted [here](https://github.com/torvalds/linux/blob/master/Documentation/trace/cor…. One thing I will like to note is that my `rwp` did move when i initially activated the sink and source. However, the 2nd time i checked the `rwp`, it went back `0x0`, the same position as that of my `rrp`. Regardless, I run the program in another kernel, performed CPU affinity to the program to point at the CPU in which the ETM is activated (in this case, CPU0). While the program is running, i looked at the `rwp/rrp` and see that neither of the pointers are moving. I thought perhaps if i perform `dd` operation, I might be able to get something. I performed `dd if=/dev/ec036000.etf of=/data/cs.bin bs=1M` but my device crashed and reboot itself without saving any files. More info about my device: `# uname -a` yields `Linux localhost 4.9.176-12953-g7c09ed7b46a4-dirty #13 SMP PREEMPT Tue Jun 4 10:16:26 +08 2019 aarch64` hi3660-coresight.dtsi yields the following device tree: ```bash /* * Copyright (C) 2016-2017 Hisilicon Ltd. * Author: shiwanglai <shiwanglai(a)hisilicon.com> * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License version 2 as * publishhed by the Free Software Foundation. */ / { amba { #address-cells = <2>; #size-cells = <2>; compatible = "arm,amba-bus"; ranges; /* A53 cluster internal coresight */ etm@0,ecc40000 { compatible = "arm,coresight-etm4x","arm,primecell"; reg = <0 0xecc40000 0 0x1000>; clocks = <&pclk>; clock-names = "apb_pclk"; cpu = <&cpu0>; port { etm0_out_port: endpoint { remote-endpoint = <&funnel0_in_port0>; }; }; }; etm@1,ecd40000 { compatible = "arm,coresight-etm4x","arm,primecell"; reg = <0 0xecd40000 0 0x1000>; clocks = <&pclk>; clock-names = "apb_pclk"; cpu = <&cpu1>; port { etm1_out_port: endpoint { remote-endpoint = <&funnel0_in_port1>; }; }; }; etm@2,ece40000 { compatible = "arm,coresight-etm4x","arm,primecell"; reg = <0 0xece40000 0 0x1000>; clocks = <&pclk>; clock-names = "apb_pclk"; cpu = <&cpu2>; port { etm2_out_port: endpoint { remote-endpoint = <&funnel0_in_port2>; }; }; }; etm@3,ecf40000 { compatible = "arm,coresight-etm4x","arm,primecell"; reg = <0 0xecf40000 0 0x1000>; clocks = <&pclk>; clock-names = "apb_pclk"; cpu = <&cpu3>; port { etm3_out_port: endpoint { remote-endpoint = <&funnel0_in_port3>; }; }; }; funnel0:funnel@0,ec801000 { compatible = "arm,coresight-funnel","arm,primecell"; reg = <0 0xec801000 0 0x1000>; clocks = <&pclk>; clock-names = "apb_pclk"; ports { #address-cells = <1>; #size-cells = <0>; /* funnel output port */ port@0 { reg = <0>; funnel0_out_port: endpoint { remote-endpoint = <&etf0_in_port>; }; }; /* funnel input ports */ port@1 { reg = <0>; funnel0_in_port0: endpoint { slave-mode; remote-endpoint = <&etm0_out_port>; }; }; port@2 { reg = <1>; funnel0_in_port1: endpoint { slave-mode; remote-endpoint = <&etm1_out_port>; }; }; port@3 { reg = <2>; funnel0_in_port2: endpoint { slave-mode; remote-endpoint = <&etm2_out_port>; }; }; port@4 { reg = <3>; funnel0_in_port3: endpoint { slave-mode; remote-endpoint = <&etm3_out_port>; }; }; }; }; etf0:etf@0,ec802000 { compatible = "arm,coresight-tmc","arm,primecell"; reg = <0 0xec802000 0 0x1000>; clocks = <&pclk>; clock-names = "apb_pclk"; ports { #address-cells = <1>; #size-cells = <0>; /* input port */ port@0 { reg = <0>; etf0_in_port: endpoint { slave-mode; remote-endpoint = <&funnel0_out_port>; }; }; /* output port */ port@1 { reg = <0>; etf0_out_port: endpoint { remote-endpoint = <&funnel2_in_port0>; }; }; }; }; /* A73 cluster internal coresight */ etm@4,ed440000 { compatible = "arm,coresight-etm4x","arm,primecell"; reg = <0 0xed440000 0 0x1000>; clocks = <&pclk>; clock-names = "apb_pclk"; cpu = <&cpu4>; port { etm4_out_port: endpoint { remote-endpoint = <&funnel1_in_port0>; }; }; }; etm@5,ed540000 { compatible = "arm,coresight-etm4x","arm,primecell"; reg = <0 0xed540000 0 0x1000>; clocks = <&pclk>; clock-names = "apb_pclk"; cpu = <&cpu5>; port { etm5_out_port: endpoint { remote-endpoint = <&funnel1_in_port1>; }; }; }; etm@6,ed640000 { compatible = "arm,coresight-etm4x","arm,primecell"; reg = <0 0xed640000 0 0x1000>; clocks = <&pclk>; clock-names = "apb_pclk"; cpu = <&cpu6>; port { etm6_out_port: endpoint { remote-endpoint = <&funnel1_in_port2>; }; }; }; etm@7,ed740000 { compatible = "arm,coresight-etm4x","arm,primecell"; reg = <0 0xed740000 0 0x1000>; clocks = <&pclk>; clock-names = "apb_pclk"; cpu = <&cpu7>; port { etm7_out_port: endpoint { remote-endpoint = <&funnel1_in_port3>; }; }; }; funnel1:funnel@1,ed001000 { compatible = "arm,coresight-funnel","arm,primecell"; reg = <0 0xed001000 0 0x1000>; clocks = <&pclk>; clock-names = "apb_pclk"; ports { #address-cells = <1>; #size-cells = <0>; /* funnel output port */ port@0 { reg = <0>; funnel1_out_port: endpoint { remote-endpoint = <&etf1_in_port>; }; }; /* funnel input ports */ port@1 { reg = <0>; funnel1_in_port0: endpoint { slave-mode; remote-endpoint = <&etm4_out_port>; }; }; port@2 { reg = <1>; funnel1_in_port1: endpoint { slave-mode; remote-endpoint = <&etm5_out_port>; }; }; port@3 { reg = <2>; funnel1_in_port2: endpoint { slave-mode; remote-endpoint = <&etm6_out_port>; }; }; port@4 { reg = <3>; funnel1_in_port3: endpoint { slave-mode; remote-endpoint = <&etm7_out_port>; }; }; }; }; etf1:etf@1,ed002000 { compatible = "arm,coresight-tmc","arm,primecell"; reg = <0 0xed002000 0 0x1000>; clocks = <&pclk>; clock-names = "apb_pclk"; ports { #address-cells = <1>; #size-cells = <0>; /* input port */ port@0 { reg = <0>; etf1_in_port: endpoint { slave-mode; remote-endpoint = <&funnel1_out_port>; }; }; /* output port */ port@1 { reg = <0>; etf1_out_port: endpoint { remote-endpoint = <&funnel2_in_port0>; }; }; }; }; /* Top coresight config */ funnel@2,ec031000 { compatible = "arm,coresight-funnel","arm,primecell"; reg = <0 0xec031000 0 0x1000>; clocks = <&pclk>; clock-names = "apb_pclk"; ports { #address-cells = <1>; #size-cells = <0>; /* funnel output port */ port@0 { reg = <0>; funnel2_out_port: endpoint { remote-endpoint = <&etf2_in_port>; }; }; /* funnel input ports */ /* - both cluster ETFs go to port 0 - there may be another (hidden) funnel between these */ port@1 { reg = <0>; funnel2_in_port0: endpoint { slave-mode; remote-endpoint = <&etf0_out_port>; }; }; }; }; etf@2,ec036000 { compatible = "arm,coresight-tmc","arm,primecell"; reg = <0 0xec036000 0 0x1000>; clocks = <&pclk>; clock-names = "apb_pclk"; ports { #address-cells = <1>; #size-cells = <0>; /* input port */ port@0 { reg = <0>; etf2_in_port: endpoint { slave-mode; remote-endpoint = <&funnel2_out_port>; }; }; /* output port */ port@1 { reg = <0>; etf2_out_port: endpoint { remote-endpoint = <&replicator0_in_port>; }; }; }; }; replicator { compatible = "arm,coresight-replicator"; ports { #address-cells = <1>; #size-cells = <0>; /* etr out port */ port@0 { reg = <0>; replicator0_out_port0: endpoint { remote-endpoint = <&etr_in_port>; }; }; /* TPIU out port */ port@1 { reg = <1>; replicator0_out_port1: endpoint { remote-endpoint = <&tpiu_in_port>; }; }; /* input port */ port@2 { reg = <0>; replicator0_in_port: endpoint { slave-mode; remote-endpoint = <&etf2_out_port>; }; }; }; }; etr@0,ec033000 { compatible = "arm,coresight-tmc","arm,primecell"; reg = <0 0xec033000 0 0x1000>; clocks = <&pclk>; clock-names = "apb_pclk"; ports { #address-cells = <1>; #size-cells = <0>; /* etr input port */ port@0 { reg = <0>; etr_in_port: endpoint { slave-mode; remote-endpoint = <&replicator0_out_port0>; }; }; }; }; tpiu@ec032000 { compatible = "arm,coresight-tpiu", "arm,primecell"; reg = <0 0xec032000 0 0x1000>; clocks = <&pclk>; clock-names = "apb_pclk"; port { tpiu_in_port: endpoint { slave-mode; remote-endpoint = <&replicator0_out_port1>; }; }; }; }; }; ``` On occasions when the trace succeeded (which i could not fathom when it will work and when it will crash), I used ptm2human to acquire the following decoded log: ```bash Reading cs.bin Decode trace stream of ID 0 Syncing the trace stream... Decode trace stream of ID 1 Syncing the trace stream... Decode trace stream of ID 2 Syncing the trace stream... Decode trace stream of ID 3 Syncing the trace stream... Decode trace stream of ID 4 Syncing the trace stream... Decode trace stream of ID 5 Syncing the trace stream... Decode trace stream of ID 6 Syncing the trace stream... Decode trace stream of ID 7 Syncing the trace stream... Decode trace stream of ID 8 Syncing the trace stream... Decode trace stream of ID 9 Syncing the trace stream... Decode trace stream of ID 10 Syncing the trace stream... Decode trace stream of ID 11 Syncing the trace stream... Decode trace stream of ID 12 Syncing the trace stream... Decode trace stream of ID 13 Syncing the trace stream... Decode trace stream of ID 14 Syncing the trace stream... Decode trace stream of ID 15 Syncing the trace stream... Decode trace stream of ID 16 Syncing the trace stream... Decode trace stream of ID 17 Syncing the trace stream... Decode trace stream of ID 18 Syncing the trace stream... Decode trace stream of ID 19 Syncing the trace stream... Decode trace stream of ID 20 Syncing the trace stream... Decode trace stream of ID 21 Syncing the trace stream... Decode trace stream of ID 22 Syncing the trace stream... Decode trace stream of ID 23 Syncing the trace stream... Decoding the trace stream... TraceInfo - Cycle count disabled, Tracing of conditional non-branch instruction disabled, No explicit tracing of load instructions, No explicit tracing of store instructions, p0_key = 0x0, curr_spec_depth = 0, cc_threshold = 0x0 Context - Context ID = 0x0, VMID = 0x0, Exception level = EL0, Security = NS, 32-bit instruction Address - Instruction address 0x0000005edb77ed10, Instruction set Aarch32 (Thumb) Mispredict ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - N Commit - 1 Address - Instruction address 0x0000007c010c9388, Instruction set Aarch32 (Thumb) ATOM - N Commit - 1 ATOM - E Commit - 1 ATOM - N Commit - 1 ATOM - E Commit - 1 ATOM - N Commit - 1 ATOM - E Commit - 1 Address - Instruction address 0x0000007c01167c70, Instruction set Aarch32 (Thumb) ATOM - N Commit - 1 ATOM - N Commit - 1 ATOM - N Commit - 1 ATOM - E Commit - 1 Address - Instruction address 0x0000007c010ccf68, Instruction set Aarch32 (Thumb) ATOM - N Commit - 1 ATOM - N Commit - 1 ATOM - N Commit - 1 ATOM - N Commit - 1 ATOM - N Commit - 1 ATOM - N Commit - 1 ATOM - N Commit - 1 ATOM - N Commit - 1 ATOM - N Commit - 1 ATOM - N Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 Address - Instruction address 0x0000007c010c93b0, Instruction set Aarch32 (Thumb) ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 ATOM - E Commit - 1 Address - Instruction address 0x0000005edb77ed44, Instruction set Aarch32 (Thumb) ATOM - E Commit - 1 Address - Instruction address 0x0000005edb77ec98, Instruction set Aarch32 (Thumb) Complete decode of the trace stream ``` Once again, I am awfully sorry for the abnormally long post. However, I have been trying to work on this for the past 6 weeks and I have not been able to understand the entire framework and hence I am getting desperate to seek professional help. I have attached the source code and the log file to this email to hopefully help you understand my problems better. Yours Sincerely, Jeremy This email may contain confidential and/or proprietary information that is exempt from disclosure under applicable law and is intended for receipt and use solely by the addressee(s) named above. If you are not the intended recipient, you are notified that any use, dissemination, distribution, or copying of this email, or any attachment, is strictly prohibited. Please delete the email immediately and inform the sender. Thank You The above message may contain confidential and/or proprietary information that is exempt from disclosure under applicable law and is intended for receipt and use solely by the addressee(s) named above. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this message, or any attachment, is strictly prohibited. If you have received this email in error, please inform the sender immediately by reply e-mail or telephone, reversing the charge if necessary. Please delete the message thereafter. Thank you.

5 years, 11 months

GDB process record and replay with ARM CoreSight

by Zied Guermazi

Good afternoon, GDB has a valuable feature consisting of process record and replay. In fact, GDB can record a log of process execution and save it. This record can be loaded later on, and used for debugging. This is called offline debugging. it offers the advantage that you can catch the issue once, and replay it as much as needed to find the root cause and fix it. this is extremely valuable for non reproduce-able or hard to reproduce bugs. you can replay the record either forwards or backwards, which is very convivial for observing and analyzing the software. To realize this functionality, GDB is in fact executing the software, one assembly instruction after another and recording relevant registers and memory locations. This is a slow operation that can drastically change the timing of process execution, and thus change the conditions that raise the bug. To overcome this limitation, GDB can use available SoC IPs to accelerate the operation. As per today, GDB has support for "Intel Processor Trace" and " Branch Trace Store" IP on Intel processors. ARM based SoCs have also IPs that can be used to assist process record, namely CoreSight trace sources (ETM, PTM ..), trace links ( Funnels ...) and trace sinks (ETB, ETR, TPIU...). They are now supported in Linux kernel, through corresponding drivers and the extension of perf. A library for decoding ETM traces (OpenCSD) is also available. The way is now paved to bring acceleration of process record for ARM based SoC to GDB. I am re-sending RFC and making it available as basis for discussions for implementing this feature. it is also attached as text file B.R. Zied Guermazi Non intrusive execution recording for GDB using ARM CoreSight Status of this Memo This memo provides information for Linaro coresight and toolchain communities. Distribution of this memo is unlimited. Abstract A method of realizing execution recording in GDB in a non-intrusive way. This method is based on the use of CoreSight hardware tracing, available on ARM Cortex devices. Table of Contents 1 Introduction 2 State of the art 3 Use cases 3.1 Self hosted debug monitor 3.2 Remote debug monitor 3.3 External debugger 4 Implementation needs 4.1 Self hosted debug monitor 4.2 Remote Debug monitor 4.3 External debugger 5 Remote protocol execution sequence 6 Remote protocol extensions 7 Solutions and alternatives . 7.1 Scope definition 7.2 CoreSight infrastructure exposure to the user 7.3 Parameters needed for parsing traces 1. Introduction CoreSight technology offers a toolset for tracing the execution of a program on a CPU, as well as routing the traces to an external trace port analyzer or storing it in a dedicated internal memory. Those traces do not affects system performance, and can be used as a record for program execution. GDB offers reverse debugging by recording program execution and storing it. GDB offers either full record or program flow (branch) record. Records can be replayed later-on for forwards or backwards debugging. This request for comments is about realizing GDB record and replay functionality using CoreSight technology. it presents typical use cases and discuss different alternatives for realizing above mentioned feature. 2. State of the art GDB currently supports two execution recording variants: - full record: where registers as well as memory are recorded for each instruction. in this case GDB collects the registers as well as involved memory area after each instruction. currently this has no support for hardware accelerators - branch record: where only program flow is recorded. In this case GDB collects program execution flow. currently branch record is implemented either with or without hardware acceleration by using Intel branch trace store "bts" and Intel processor trace "pt" hardware accelerator on supported cpus. 3. Use cases Programs running on ARM processors can be be debugged in many configurations. three of them are selected in this RFC as base for discussion : 3.1. Self hosted debug monitor Those are systems where the debugger program runs on the same cpu as the debugged program and monitors it. user interacts with the debugging session on the target host itself. Linux GDB is an example of such systems. in such a system following setup is considered - Target: a process running on an ARM cortex A - Debugger: gnu GDB via ptrace API (arm-linux-gnueabihf-gdb) +-----------------------------------+ | Target | | +------------+ | | +------+ | Coresight | | | | | | components:| | | | GDB |<--------->| | | | | | ^ | DWT, ETM, | | | +------+ | | ITM, TPIU | | | ^ | | TMC, ETB | | | | | +------------+ | +----|---------|--------------------+ | | | | arm-linux- | gnueabihf- | gdb | debug: ptrace trace: perf/CoreSight drivers 3.2. Remote debug monitor Those are systems where the debugger program runs on the same cpu as the debugged program and monitors it. user interacts with the debugging session remotely from a PC Linux GDB is an example of such systems. in such a system following setup is considered - Target: a process running on an ARM cortex A - GDB server: gnu gdbserver (arm-linux-gnueabihf-gdbserver) - GDB client: gnu GDB (arm-linux-gnueabihf-gdb) - UI: eclipse with needed plugins, MI interface is used. +--------------------------+ +---------------------------------------+ | Host | | Target | | | | +------------+ | | +-----+ +------+ | | +------+ | Coresight | | | | | | GDB | | | | GDB | | components:| | | | UI |<--->| |<--->|<--->|<--->| |<--------->| | | | | | ^ |Client| ^ | ^ | |Server| ^ | DWT, ETM, | | | +-----+ | +------+ | | | | +------+ | | ITM, TPIU | | | ^ | ^ | | | | ^ | | TMC, ETB | | | | | | | | | | | | +------------+ | +----|-----|-----|------|--+ | +--------|---------|--------------------+ | | | | | | | | | | | | | | Eclipse | arm-linux- | | arm-linux- | | gnueabihf- | TCP/IP gnueabihf- | | gdb | UART gdbserver | GDB MI GDB remote debug: ptrace protocol trace: perf/CoreSight drivers 3.3. External debugger Those are systems where an external debugger is used. It accesses the target using JTAG or SWD. Target is usually a bare metal embedded systems or systems with an rtos. as an example, following setup is considered: - Target: firmware running on ARM cortex M. - Debugger: external debug and trace device. - GDB server: OpenOcd. - GDB Client: arm-none-eabi-gdb. - UI: eclipse with needed plugins, MI interface is used. +--------------------------------------+ +-------+ +-------------+ | Host | | dbggr | | Target | | | | | | | | +-----+ +------+ +------+ | | | | Coresight | | | | | GDB | | GDB | | | Debug | | components: | | | UI |<--->| |<--->| |<-->|<--->| + |<--->| | | | | ^ |Client| ^ |Server| | ^ | Trace | ^ | DWT, ETM, | | +-----+ | +------+ | +------+ | | | | | | ITM, TPIU | | ^ | ^ | ^ | | | | | | | | | | | | | | | | | | | | +----|-----|-----|------|-----|--------+ | +-------+ | +-------------+ | | | | | | | | | | | | | | Eclipse | arm-none- | OpenOcd | | | eabi-gdb | PyOcd | | | | | | GDB MI GDB remote Ethernet debug: JTAG/SWD protocol USB trace: Serial/Parallel 4. Implementation needs 4.1 Self hosted debug monitor GDB : arm-linux-gnueabihf-gdb the interface defined in btrace.h for capturing and processing traces has to be implemented for arm CoreSight. needed actions: - in btrace-common.h: add needed structures for capturing and handling etm traces - in linux-btrace.h: - add btrace_tinfo_etm - amend btrace_target_info - in linux-btrace.c: change following functions to support etm traces - linux_enable_btrace - linux_disable_btrace - linux_read_btrace - linux_btrace_conf - in arm-linux-nat.c:add an api to - configure btrace - enable btrace - disable btrace - read btrace - in btrace.c - btrace_add_pc btrace_fetch has to be implemented for Coresight this means using opencsd library to parse etms and then reconstruct executed instructions accordingly (btrace_compute_ftrace_1) - in record-btrace.c - add command for showing record btrace etm options - add command for starting tracing with CoreSight and its handler (cmd_record_btrace_etm_start) - adapt cmd_show_record_btrace_cpu ... perf: needed actions: - make sure that perf can start/stop tracing a process with its threads, collect etm traces and deliver them to the user 4.2 Remote Debug monitor changes described in 7.1 are needed. in addition, and to support remote protocol following changes are needed GDB server: arm-linux-gnueabihf-gdbserver needed actions: - in linux-low - linux_low_read_btrace: add support for etm traces formatting. - linux_low_btrace_conf: :add support for etm configuration formatting. GDB client: arm-linux-gnueabihf-gdb needed actions: - in remote.c - adapt enable_btrace - adapt disable_btrace - in btrace.c - parse_xml_btrace: update btrace.dtd [2] and related data structures btrace_xxx - parse_xml_btrace_conf: update btrace-conf.dtd [3] and related data structures btrace_conf_xxx - extend Remote protocol handling to support coresight etm traces UI: eclipse needed actions make sure that the plugin for recoding execution and replaying it is coping well in case of arm-linux Remote protocol needs to be extended by -1- Adding Qbtrace:CoreSight (or etm) to start collecting etm traces -2- Amending 'Branch Trace Format' xml specification to consider etm traces transfer -3- Amending 'Branch Trace Configuration Format' xml specification to consider parameters needed for etm 4.3 External debugger changes described in 7.2 are needed. in addition, and to support tracing a remote dealing with an external debugger (bare metal embedded system) following changes are needed GDB server: OpenOcd needed actions: - rework etm driver to make it up to date. - add a driver for configuring trace interconnect IPs - rework the driver for TPIU. - integrate support for a Trace port analyzer. - extend remote protocol implementation to support recording Coresight infrastructure of the SoC is to be set in OpenOcd through configuration files. Parameters that are not relevant for GDB are also specified in configuration files (trace sink, trace protocol, port size, trace synch frequency, cycle accurate tracing etc ...) GDB client: arm-none-eabi-gdb needed actions: - extend Remote protocol to support coresight etm traces - integrate etm trace parsing library - interface the parser to record_btrace_target Remote protocol needs -in addition to 7.2- to be extended by - Adding Qbtrace-conf:CoreSight:core=value to support multicore SoC - Adding btrace-conf:CoreSight:id=value to support demultiplexing multiple trace sources - Adding Qbtrace-conf:CoreSight:filter:context=value to support filtering traces belonging to a given process/thread - Adding Qbtrace-conf:CoreSight:filter:start-address=value and Qbtrace-conf:CoreSight:filter:end-address=value to support filtering traces for given functions/blocks/lib - Adding Qbtrace-conf:CoreSight:trigger:on-address=value and Qbtrace-conf:CoreSight:trigger:off-address=value to support triggering tracing or stop tracing if a certain function/block/lib is executed alternatively some of configurations related to filtering and triggering can be delegated to the GDB server. UI: eclipse test and verify that existing plugins cope well with GDB extensions 5. Remote protocol execution sequence GDB and gdbserver are communicating using the GDB remote protocol. on a semantic level a tracing session runs though following sequence (1) GDB client queries gdb server support for branch trace (2) GDB server answers with - qXfer:btrace:read - qXfer:btrace-conf:read - Qbtrace:off - Qbtrace:CoreSight - Qbtrace-conf:CoreSight:xxx where xxx is the parameter name (3) GDB client sends command to let start emitting and collecting traces (Qbtrace:CoreSight) (4) GDB server executes the commands (5) GDB client sends command to stop emitting and collecting traces (Qbtrace:off) (6) GDB server exectues the command (7) GDB client sends command to get collected traces from trace sink (qXfer:btrace:read:annex:offset,length) (8) GDB server executes the command and sends back collected traces (9) GDB client parses the traces and reconstructs target states 6. Remote protocol extensions the remote protocol needs be extended with following primitives to support CoreSight tracing - start tracing and traces capture using CoreSight (Qbtrace:CoreSight) the remote protocol can be extended with following primitives to take advantages of etm functionalities. - select the core to trace on in the case of a multicore system GDB client sends command to select the core to trace (Qbtrace-conf:CoreSight:core=value) - set the trace id for the traces GDB client sends command to set trace id (Qbtrace-conf:CoreSight:id=value) - select the context to trace GDB client sends command to select the context to trace (Qbtrace-conf:CoreSight:filter:context=value) - select the address range to trace GDB client sends command to select the address range to trace (Qbtrace-conf:CoreSight:filter:start-address=value) (Qbtrace-conf:CoreSight:filter:end-address=value) - set triggers for starting and stopping tracing GDB client sends command to select the address to trigger tracing (Qbtrace-conf:CoreSight:trigger:on-address=value) (Qbtrace-conf:CoreSight:trigger:off-address=value) 7. alternatives and discussions 7.1. Scope definition Coresight ETM IP comes in many versions and many implementations. According to its capabilities, it can trace instructions only or instructions and involved data/data address. All ETMs variants support instructions tracing and can therefore be used for for branch tracing. 7.2. CoreSight infrastructure exposure to the user it is here about assigning the responsibility of configuring Coresight infrastructure to generate and route traces. two alternatives are possible: - coresight infrastructure exposed to GDB client (and UI): in this alternative the user or the UI is responsible for configuring coresight IPs in the SoC, by accessing their registers directly or via coresigh driver. Remote protocol is used to configure trace sink (ETB or TPA) to start/stop collecting traces - coresight infrastructure is not exposed outside of gdbserver. in this case high level commands can be provided by gdbserver remote protocol to setup and configure coresight IPs in the SoC. My recommendation is to extend remote protocol to provide high level commands to setup and configure coresight IPs in the SoC, or to use a different channel to pass configuration parameters to GDB server 7.3. Parameters needed for parsing traces Some configuration parameters like etm version, trace id ... (content of registers ETMCR, ETMIDR, ETMCCER, ETMTRACEIDR) are needed for extracting and parsing etm trace, those parameters needs to be exchanged between GDB server and client. following alternatives are possible: - extend the remote protocol to get those params with explicit queries - add them to the content of the response to qXfer:btrace-conf:read - add them to the content of the response to qXfer:btrace:read

5 years, 11 months

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

CoreSight