DEN0154 states that hardware will be allowed to ignore writes to TRB*
registers while the trace buffer is enabled. Add an ISB to ensure that
it's disabled before clearing the other registers.
This is purely defensive because it's expected that arm_trbe_disable()
would be called before teardown which has the required ISB.
Fixes: a2b579c41fe9 ("coresight: trbe: Remove redundant disable call")
Signed-off-by: James Clark <james.clark(a)linaro.org>
---
drivers/hwtracing/coresight/coresight-trbe.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/hwtracing/coresight/coresight-trbe.c b/drivers/hwtracing/coresight/coresight-trbe.c
index 8267dd1a2130..10f3fb401edf 100644
--- a/drivers/hwtracing/coresight/coresight-trbe.c
+++ b/drivers/hwtracing/coresight/coresight-trbe.c
@@ -257,6 +257,7 @@ static void trbe_drain_and_disable_local(struct trbe_cpudata *cpudata)
static void trbe_reset_local(struct trbe_cpudata *cpudata)
{
write_sysreg_s(0, SYS_TRBLIMITR_EL1);
+ isb();
trbe_drain_buffer();
write_sysreg_s(0, SYS_TRBPTR_EL1);
write_sysreg_s(0, SYS_TRBBASER_EL1);
---
base-commit: 19272b37aa4f83ca52bdf9c16d5d81bdd1354494
change-id: 20250609-james-cs-trblimitr-isb-523f20d874d6
Best regards,
--
James Clark <james.clark(a)linaro.org>
This series refactors the way CPU IDs are retrieved from the device
tree.
Usually, there is a for loop that goes over every single CPU that can be
avoided. This also reduces the amount of NULL pointer checks in drivers.
I have abstracted away that loop and introduced a new function
(of_cpu_node_to_id) for this.
This patchset is a subset of [1], where I removed content and patches
relevant to hyper-threaded cores for DT. Based on the discussion, the
code refactor is still useful, hence this patchset.
[1] https://lore.kernel.org/all/20250512080715.82-1-alireza.sanaee@huawei.com/
Alireza Sanaee (5):
of: add infra for finding CPU id from phandle
arch_topology: update CPU map to use the new API
coresight: cti: Use of_cpu_phandle_to_id for grabbing CPU id
coresight: Use of_cpu_phandle_to_id for grabbing CPU id
perf/arm-dsu: refactor cpu id retrieval via new API
of_cpu_phandle_to_id
drivers/base/arch_topology.c | 12 ++++----
.../coresight/coresight-cti-platform.c | 15 ++--------
.../hwtracing/coresight/coresight-platform.c | 14 ++-------
drivers/of/cpu.c | 29 +++++++++++++++++++
drivers/perf/arm_dsu_pmu.c | 6 ++--
include/linux/of.h | 9 ++++++
6 files changed, 51 insertions(+), 34 deletions(-)
--
2.43.0
On Wed, Jul 02, 2025 at 12:05:10PM +0100, Yeoreum Yun wrote:
[...]
> > @@ -445,13 +445,37 @@ static int etm4_enable_trace_unit(struct etmv4_drvdata *drvdata)
> > etm4x_relaxed_write32(csa, TRCRSR_TA, TRCRSR);
> >
> > etm4x_allow_trace(drvdata);
> > +
> > + /*
> > + * According to software usage PKLXF in Arm ARM (ARM DDI 0487 L.a),
> > + * execute a Context synchronization event to guarantee the trace unit
> > + * will observe the new values of the System registers.
> > + */
> > + if (!csa->io_mem)
> > + isb();
> > +
>
> But, when write to SYS_TRFCR_EL1 in etm4x_allow_trace(), it already does
> isb(). Is it redundant?
Good point. It is not sufficient. As a system register writing in
kvm_tracing_set_el1_configuration(), this is why adds a ISB here.
Thanks,
Leo
Hi Levi,
On Wed, Jul 02, 2025 at 12:10:17PM +0100, Yeoreum Yun wrote:
[...]
> > @@ -579,6 +572,13 @@ static int etm4_enable_hw(struct etmv4_drvdata *drvdata)
> >
> > if (!drvdata->paused)
> > rc = etm4_enable_trace_unit(drvdata);
> > +
> > + /*
> > + * As recommended by section 4.3.7 (Synchronization of register updates)
> > + * of ARM IHI 0064H.b, the self-hosted trace analyzer always executes an
> > + * ISB instruction after programming the trace unit registers.
> > + */
> > + isb();
>
> But according to 4.3.7 ("Synchronization when using memory-mapped
> interface"), doesn't it need to dsb like:
>
> if (csa->iomem)
> dsb(sy);
> isb();
>
> Or am I missing something?
Section 4.3.7 suggests using a DSB barrier to ensure that writes have
completed in MMIO mode. It also mentions an alternative:
"If the memory is marked as Device-nGnRE or stronger, read back the
value of any register in the trace unit. This relies on the peripheral
coherence order defined in the Arm architecture."
In the etm4_{enable|disable}_trace_unit() functions, each time the
TRCPRGCTLR register is written, the driver polls bits in TRCSTATR.
This acts as synchronization using read-after-write (RAW), which is
exactly the approach suggested above.
This is why we don't need DSB() anymore.
Thanks,
Leo
Hi Levi,
On Wed, Jul 02, 2025 at 10:49:01AM +0100, Yeoreum Yun wrote:
> Hi Leo,
>
> > {
> > - return local_cmpxchg(&csdev->mode, CS_MODE_DISABLED, new_mode) ==
> > - CS_MODE_DISABLED;
> > + int curr = CS_MODE_DISABLED;
> > +
> > + return atomic_try_cmpxchg_acquire(&csdev->mode, &curr, new_mode);
> > }
>
> Just question. why is acquire symentic enough in here?
My understanding is that acquire semantics ensure ordering between
cmpxchg_acquire() and all memory accesses that follow it. However, it
does not guarantee that memory accesses appearing before the acquire
are ordered as well.
This is exactly what we want in the driver. We must ensure to first grab
an active device mode, then it is safe to proceed later operations (e.g.
set configurations in driver data and access registers).
> before this change, local_cmpxchg seems to use full_fenced.
Not really. Arm64 has atomic instruction for cmpxchg, it does not use
full_fenced. It should run into the path of arch_cmpxchg().
Thanks,
Leo
On 2025/6/20 19:53, Jonathan Cameron wrote:
> On Fri, 20 Jun 2025 15:54:10 +0800
> Junhao He <hejunhao3(a)huawei.com> wrote:
>
>> From: Yicong Yang <yangyicong(a)hisilicon.com>
>>
>> tmc_drvdata::reading is used to indicate whether a reading process
>> is performed through /dev/xyz.tmc. Document it.
>>
>> Reviewed-by: James Clark <james.clark(a)linaro.org>
>> Signed-off-by: Yicong Yang <yangyicong(a)hisilicon.com>
>> Signed-off-by: Junhao He <hejunhao3(a)huawei.com>
>> ---
>> drivers/hwtracing/coresight/coresight-tmc.h | 1 +
>> 1 file changed, 1 insertion(+)
>>
>> diff --git a/drivers/hwtracing/coresight/coresight-tmc.h b/drivers/hwtracing/coresight/coresight-tmc.h
>> index 6541a27a018e..3ca0d40c580d 100644
>> --- a/drivers/hwtracing/coresight/coresight-tmc.h
>> +++ b/drivers/hwtracing/coresight/coresight-tmc.h
>> @@ -220,6 +220,7 @@ struct tmc_resrv_buf {
>> * @pid: Process ID of the process that owns the session that is using
>> * this component. For example this would be the pid of the Perf
>> * process.
>> + * @reading: buffer's in the reading through "/dev/xyz.tmc" entry
> Hi,
>
> Perhaps reword:
>
> "buffer is being read through "/dev/xyz.tmc" entry" or
> "buffer read in progress through "/dev/xyz.tmc" entry"
>
> I've not checked what this actually means - just looking at what you have here.
Will update this. Thanks
Best regards,
Junhao.
>> * @stop_on_flush: Stop on flush trigger user configuration.
>> * @buf: Snapshot of the trace data for ETF/ETB.
>> * @etr_buf: details of buffer used in TMC-ETR
> .
>
On 2025/6/20 20:45, Jonathan Cameron wrote:
> On Fri, 20 Jun 2025 15:54:11 +0800
> Junhao He <hejunhao3(a)huawei.com> wrote:
>
>> When trying to run perf and sysfs mode simultaneously, the WARN_ON()
>> in tmc_etr_enable_hw() is triggered sometimes:
>>
>> WARNING: CPU: 42 PID: 3911571 at drivers/hwtracing/coresight/coresight-tmc-etr.c:1060 tmc_etr_enable_hw+0xc0/0xd8 [coresight_tmc]
>> [..snip..]
>> Call trace:
>> tmc_etr_enable_hw+0xc0/0xd8 [coresight_tmc] (P)
>> tmc_enable_etr_sink+0x11c/0x250 [coresight_tmc] (L)
>> tmc_enable_etr_sink+0x11c/0x250 [coresight_tmc]
>> coresight_enable_path+0x1c8/0x218 [coresight]
>> coresight_enable_sysfs+0xa4/0x228 [coresight]
>> enable_source_store+0x58/0xa8 [coresight]
>> dev_attr_store+0x20/0x40
>> sysfs_kf_write+0x4c/0x68
>> kernfs_fop_write_iter+0x120/0x1b8
>> vfs_write+0x2c8/0x388
>> ksys_write+0x74/0x108
>> __arm64_sys_write+0x24/0x38
>> el0_svc_common.constprop.0+0x64/0x148
>> do_el0_svc+0x24/0x38
>> el0_svc+0x3c/0x130
>> el0t_64_sync_handler+0xc8/0xd0
>> el0t_64_sync+0x1ac/0x1b0
>> ---[ end trace 0000000000000000 ]---
>>
>> Since the sysfs buffer allocation and the hardware enablement is not
>> in the same critical region, it's possible to race with the perf
>>
>> mode:
>> [sysfs mode] [perf mode]
>> tmc_etr_get_sysfs_buffer()
>> spin_lock(&drvdata->spinlock)
>> [sysfs buffer allocation]
>> spin_unlock(&drvdata->spinlock)
>> spin_lock(&drvdata->spinlock)
>> tmc_etr_enable_hw()
>> drvdata->etr_buf = etr_perf->etr_buf
>> spin_unlock(&drvdata->spinlock)
>> spin_lock(&drvdata->spinlock)
>> tmc_etr_enable_hw()
>> WARN_ON(drvdata->etr_buf) // WARN sicne etr_buf initialized at
>> the perf side
>> spin_unlock(&drvdata->spinlock)
>>
>> A race condition is introduced here, perf always prioritizes execution, and
>> warnings can lead to concerns about potential hidden bugs, such as getting
>> out of sync.
>>
>> To fix this, configure the tmc-etr mode before invoking enable_etr_perf()
>> or enable_etr_sysfs(), explicitly check if the tmc-etr sink is already
>> enabled in a different mode, and simplily the setup and checks for "mode".
>> To prevent race conditions between mode transitions.
>>
>> Fixes: 296b01fd106e ("coresight: Refactor out buffer allocation function for ETR")
>> Reported-by: Yicong Yang <yangyicong(a)hisilicon.com>
>> Closes: https://lore.kernel.org/linux-arm-kernel/20241202092419.11777-2-yangyicong@…
>> Signed-off-by: Junhao He <hejunhao3(a)huawei.com>
>> ---
>> .../hwtracing/coresight/coresight-tmc-etr.c | 73 +++++++++++--------
>> 1 file changed, 43 insertions(+), 30 deletions(-)
>>
>> diff --git a/drivers/hwtracing/coresight/coresight-tmc-etr.c b/drivers/hwtracing/coresight/coresight-tmc-etr.c
>> index b07fcdb3fe1a..252a57a8e94e 100644
>> --- a/drivers/hwtracing/coresight/coresight-tmc-etr.c
>> +++ b/drivers/hwtracing/coresight/coresight-tmc-etr.c
>> @@ -1263,11 +1263,6 @@ static struct etr_buf *tmc_etr_get_sysfs_buffer(struct coresight_device *csdev)
>> raw_spin_lock_irqsave(&drvdata->spinlock, flags);
>> }
>>
>> - if (drvdata->reading || coresight_get_mode(csdev) == CS_MODE_PERF) {
>> - ret = -EBUSY;
>> - goto out;
>> - }
>> -
>> /*
>> * If we don't have a buffer or it doesn't match the requested size,
>> * use the buffer allocated above. Otherwise reuse the existing buffer.
>> @@ -1278,7 +1273,6 @@ static struct etr_buf *tmc_etr_get_sysfs_buffer(struct coresight_device *csdev)
>> drvdata->sysfs_buf = new_buf;
>> }
>>
>> -out:
>> raw_spin_unlock_irqrestore(&drvdata->spinlock, flags);
>>
>> /* Free memory outside the spinlock if need be */
>> @@ -1289,7 +1283,7 @@ static struct etr_buf *tmc_etr_get_sysfs_buffer(struct coresight_device *csdev)
>>
>> static int tmc_enable_etr_sink_sysfs(struct coresight_device *csdev)
>> {
>> - int ret = 0;
>> + int ret;
>> unsigned long flags;
>> struct tmc_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
>> struct etr_buf *sysfs_buf = tmc_etr_get_sysfs_buffer(csdev);
>> @@ -1299,23 +1293,10 @@ static int tmc_enable_etr_sink_sysfs(struct coresight_device *csdev)
>>
>> raw_spin_lock_irqsave(&drvdata->spinlock, flags);
>>
>> - /*
>> - * In sysFS mode we can have multiple writers per sink. Since this
>> - * sink is already enabled no memory is needed and the HW need not be
>> - * touched, even if the buffer size has changed.
>> - */
>> - if (coresight_get_mode(csdev) == CS_MODE_SYSFS) {
>> - csdev->refcnt++;
>> - goto out;
>> - }
>> -
>> ret = tmc_etr_enable_hw(drvdata, sysfs_buf);
>> - if (!ret) {
>> - coresight_set_mode(csdev, CS_MODE_SYSFS);
>> + if (!ret)
>> csdev->refcnt++;
>> - }
>>
>> -out:
>> raw_spin_unlock_irqrestore(&drvdata->spinlock, flags);
>>
>> if (!ret)
>> @@ -1735,11 +1716,6 @@ static int tmc_enable_etr_sink_perf(struct coresight_device *csdev, void *data)
>> struct etr_perf_buffer *etr_perf = etm_perf_sink_config(handle);
>>
>> raw_spin_lock_irqsave(&drvdata->spinlock, flags);
>> - /* Don't use this sink if it is already claimed by sysFS */
>> - if (coresight_get_mode(csdev) == CS_MODE_SYSFS) {
>> - rc = -EBUSY;
>> - goto unlock_out;
>> - }
>>
>> if (WARN_ON(!etr_perf || !etr_perf->etr_buf)) {
>> rc = -EINVAL;
>> @@ -1768,7 +1744,6 @@ static int tmc_enable_etr_sink_perf(struct coresight_device *csdev, void *data)
>> if (!rc) {
>> /* Associate with monitored process. */
>> drvdata->pid = pid;
>> - coresight_set_mode(csdev, CS_MODE_PERF);
>> drvdata->perf_buf = etr_perf->etr_buf;
>> csdev->refcnt++;
>> }
>> @@ -1781,14 +1756,52 @@ static int tmc_enable_etr_sink_perf(struct coresight_device *csdev, void *data)
>> static int tmc_enable_etr_sink(struct coresight_device *csdev,
>> enum cs_mode mode, void *data)
>> {
>> + struct tmc_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
>> + enum cs_mode old_mode;
>> + int rc = -EINVAL;
>> +
>> + scoped_guard(spinlock_irqsave, &drvdata->spinlock) {
>> + old_mode = coresight_get_mode(csdev);
>> + if (old_mode != CS_MODE_DISABLED && old_mode != mode)
>> + return -EBUSY;
>> +
>> + if (drvdata->reading)
>> + return -EBUSY;
>> +
>> + /* In sysFS mode we can have multiple writers per sink. */
>> + if (old_mode == CS_MODE_SYSFS) {
> This seems odd. You are incrementing the reference count when potentially changing
> away from CS_MODE_SYSFS?
> Is this meant to only occur if old_mode == mode && old_mode == CS_MODE_SYSFS?
Yes, old_mode == CS_MODE_SYSFS means tmc_etr is occupied by sysfs, and
old_mode == mode, so can add reference counting directly.
When tmc_etr is idle its mode is CS_MODE_DISABLED.
> In the code prior to this patch this bit only ran in tmc_enable_etr_sink_sysfs()
> which was only called based on the mode being configured (mode here I think) being
> sysfs. That no longer looks to be the case.
before the patch, first check (mode == CS_MODE_SYSFS), then check
(coresight_get_mode(csdev) == CS_MODE_SYSFS),
and finally csdev->refcnt++. This is the same as the current functionality.
> Maybe I'm missing something as the flows around this are complex.
>
>
>> + csdev->refcnt++;
>> + return 0;
>> + }
>> +
>> + /*
>> + * minor note: In sysFS mode, the task1 get locked first, it setup
>> + * etr mode to SYSFS. Then task2 get locked,it will directly return
>> + * success even when the tmc-etr is not enabled at this moment.
>> + * Ultimately, task1 will still successfully enable tmc-etr.
>> + * This is a transient state and does not cause an anomaly.
>> + */
>> + coresight_set_mode(csdev, mode);
>> + }
>> +
>> switch (mode) {
>> case CS_MODE_SYSFS:
>> - return tmc_enable_etr_sink_sysfs(csdev);
>> + rc = tmc_enable_etr_sink_sysfs(csdev);
>> + break;
>> case CS_MODE_PERF:
>> - return tmc_enable_etr_sink_perf(csdev, data);
>> + rc = tmc_enable_etr_sink_perf(csdev, data);
>> + break;
>> default:
>> - return -EINVAL;
>> + rc = -EINVAL;
>> }
>> +
>> + if (rc && old_mode != mode) {
>> + scoped_guard(spinlock_irqsave, &drvdata->spinlock) {
> Might be a local style matching thing but if not the scope is tight anyway
> so could use the unscoped version.
>
> guard(spinlock_irqsave)(&drvdata->spinlock);
> coresight_set_mode(csdev, old_mode);
Sure, Will fix in next version.
>> + coresight_set_mode(csdev, old_mode);
>> + }
>> + }
>> +
>> + return rc;
>> }
>>
>> static int tmc_disable_etr_sink(struct coresight_device *csdev)
> .
>