This series focuses on CoreSight path power management. The changes can be divided into four parts for review:
Patches 01~06: Refactor the CPU idle flow with moving common code into the CoreSight core layer. Patches 07~14: Add link control during CPU idle. Patches 15~16: Support the sink (TRBE) control during CPU idle. Patches 17~19: Move the CPU hotplug flow into the coresight core layer and simplify the code.
This series is rebased on the coresight-next branch and has been verified on Juno-r2 and FVP RevC.
--- Changes in v5: - Set the per-CPU source pointer on target CPU (Suzuki). - Reused existed enable/disable buffer functions in TRBE callbacks (James). - Refactored refcount for source devices in SysFS mode. - Released path in cpu-hotplug off flow to avoid memory leak. - Updated ETMv3 driver when move common code into core layer. - Rebased on the latest coresight-next branch. - Link to v4: https://lore.kernel.org/r/20251104-arm_coresight_path_power_management_impro...
Changes in v4: - Changed to store path pointer in coresight_device, this is easier for fetching path pointer based on source device (Mike). - Dropped changes in CTI driver. - Only disabled path for CPU hot-plugged off but not enable path for hot-plugged in. - Removed James' test tags for modified patches. - Link to v3: https://lore.kernel.org/r/20250915-arm_coresight_power_management_fix-v3-0-e...
Signed-off-by: Leo Yan leo.yan@arm.com
--- Leo Yan (18): coresight: sysfs: Validate CPU online status for per-CPU sources coresight: Set per-CPU source pointer coresight: Register CPU PM notifier in core layer coresight: etm4x: Hook CPU PM callbacks coresight: Add callback to determine if PM is needed coresight: etm4x: Remove redundant condition checks in save and restore coresight: syscfg: Use spinlock to protect active variables coresight: Introduce coresight_enable_source() helper coresight: Save active path for system tracers coresight: etm4x: Set active path on target CPU coresight: etm3x: Set active path on target CPU coresight: sysfs: Use source's path pointer for path control coresight: Add 'in_idle' argument to path coresight: Control path during CPU idle coresight: Add PM callbacks for sink device coresight: sysfs: Increment refcount only for system tracers coresight: Take hotplug lock in enable_source_store() for Sysfs mode coresight: Move CPU hotplug callbacks to core layer
Yabin Cui (1): coresight: trbe: Save and restore state across CPU low power state
drivers/hwtracing/coresight/coresight-catu.c | 1 + drivers/hwtracing/coresight/coresight-core.c | 242 ++++++++++++++++++++- drivers/hwtracing/coresight/coresight-ctcu-core.c | 1 + drivers/hwtracing/coresight/coresight-cti-core.c | 1 + drivers/hwtracing/coresight/coresight-dummy.c | 1 + drivers/hwtracing/coresight/coresight-etb10.c | 1 + drivers/hwtracing/coresight/coresight-etm-perf.c | 2 +- drivers/hwtracing/coresight/coresight-etm3x-core.c | 65 ++---- drivers/hwtracing/coresight/coresight-etm4x-core.c | 153 +++---------- drivers/hwtracing/coresight/coresight-funnel.c | 1 + drivers/hwtracing/coresight/coresight-priv.h | 3 + drivers/hwtracing/coresight/coresight-replicator.c | 1 + drivers/hwtracing/coresight/coresight-stm.c | 1 + drivers/hwtracing/coresight/coresight-syscfg.c | 22 +- drivers/hwtracing/coresight/coresight-syscfg.h | 2 + drivers/hwtracing/coresight/coresight-sysfs.c | 126 ++++------- drivers/hwtracing/coresight/coresight-tmc-core.c | 1 + drivers/hwtracing/coresight/coresight-tnoc.c | 2 + drivers/hwtracing/coresight/coresight-tpda.c | 1 + drivers/hwtracing/coresight/coresight-tpdm.c | 1 + drivers/hwtracing/coresight/coresight-tpiu.c | 1 + drivers/hwtracing/coresight/coresight-trbe.c | 62 +++++- drivers/hwtracing/coresight/ultrasoc-smb.c | 1 + include/linux/coresight.h | 11 + 24 files changed, 429 insertions(+), 274 deletions(-) --- base-commit: 9e9182cab5ebc3ee7544e60ef08ba19fdf216920 change-id: 20251104-arm_coresight_path_power_management_improvement-dab4966f8280
Best regards,
The current SysFS flow first enables the links and sink, then rolls back to disable them if the source fails to enable. This failure can occur if the associated CPU is offline, which causes the SMP call to fail.
Populate CPU ID into the coresight_device structure, for components that are not CPU bound, set this field to -1.
Validate whether the associated CPU is online for a per-CPU tracer. If the CPU is offline, return -ENODEV and bail out.
Reviewed-by: Yeoreum Yun yeoreum.yun@arm.com Reviewed-by: Mike Leach mike.leach@linaro.org Tested-by: James Clark james.clark@linaro.org Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-catu.c | 1 + drivers/hwtracing/coresight/coresight-core.c | 1 + drivers/hwtracing/coresight/coresight-ctcu-core.c | 1 + drivers/hwtracing/coresight/coresight-cti-core.c | 1 + drivers/hwtracing/coresight/coresight-dummy.c | 1 + drivers/hwtracing/coresight/coresight-etb10.c | 1 + drivers/hwtracing/coresight/coresight-etm3x-core.c | 1 + drivers/hwtracing/coresight/coresight-etm4x-core.c | 1 + drivers/hwtracing/coresight/coresight-funnel.c | 1 + drivers/hwtracing/coresight/coresight-replicator.c | 1 + drivers/hwtracing/coresight/coresight-stm.c | 1 + drivers/hwtracing/coresight/coresight-sysfs.c | 3 +++ drivers/hwtracing/coresight/coresight-tmc-core.c | 1 + drivers/hwtracing/coresight/coresight-tnoc.c | 2 ++ drivers/hwtracing/coresight/coresight-tpda.c | 1 + drivers/hwtracing/coresight/coresight-tpdm.c | 1 + drivers/hwtracing/coresight/coresight-tpiu.c | 1 + drivers/hwtracing/coresight/coresight-trbe.c | 1 + drivers/hwtracing/coresight/ultrasoc-smb.c | 1 + include/linux/coresight.h | 4 ++++ 20 files changed, 26 insertions(+)
diff --git a/drivers/hwtracing/coresight/coresight-catu.c b/drivers/hwtracing/coresight/coresight-catu.c index 69b36bae97ab488197f23e6eafe3729ca22445d4..3373ddbfcc9fde7d87034b9f98d66572edb4f31a 100644 --- a/drivers/hwtracing/coresight/coresight-catu.c +++ b/drivers/hwtracing/coresight/coresight-catu.c @@ -575,6 +575,7 @@ static int __catu_probe(struct device *dev, struct resource *res) catu_desc.type = CORESIGHT_DEV_TYPE_HELPER; catu_desc.subtype.helper_subtype = CORESIGHT_DEV_SUBTYPE_HELPER_CATU; catu_desc.ops = &catu_ops; + catu_desc.cpu = -1;
coresight_clear_self_claim_tag(&catu_desc.access); drvdata->csdev = coresight_register(&catu_desc); diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index c660cf8adb1c7cafff8f85e501f056e4e151e372..0c48f5c85db6a7ec0377335eaa734527959d57c4 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -1340,6 +1340,7 @@ struct coresight_device *coresight_register(struct coresight_desc *desc) csdev->ops = desc->ops; csdev->access = desc->access; csdev->orphan = true; + csdev->cpu = desc->cpu;
csdev->dev.type = &coresight_dev_type[desc->type]; csdev->dev.groups = desc->groups; diff --git a/drivers/hwtracing/coresight/coresight-ctcu-core.c b/drivers/hwtracing/coresight/coresight-ctcu-core.c index abed15eb72b4acb0d6eb743616afe1b414f1f639..9677f43e77830681b9487d47c06c30b43aeff460 100644 --- a/drivers/hwtracing/coresight/coresight-ctcu-core.c +++ b/drivers/hwtracing/coresight/coresight-ctcu-core.c @@ -228,6 +228,7 @@ static int ctcu_probe(struct platform_device *pdev) desc.dev = dev; desc.ops = &ctcu_ops; desc.access = CSDEV_ACCESS_IOMEM(base); + desc.cpu = -1;
drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) diff --git a/drivers/hwtracing/coresight/coresight-cti-core.c b/drivers/hwtracing/coresight/coresight-cti-core.c index bfbc365bb2ef2744efab11c056b8450472957005..7b04e982342c64562a5d42654279d998b4f4c45c 100644 --- a/drivers/hwtracing/coresight/coresight-cti-core.c +++ b/drivers/hwtracing/coresight/coresight-cti-core.c @@ -932,6 +932,7 @@ static int cti_probe(struct amba_device *adev, const struct amba_id *id) cti_desc.ops = &cti_ops; cti_desc.groups = drvdata->ctidev.con_groups; cti_desc.dev = dev; + cti_desc.cpu = drvdata->ctidev.cpu;
coresight_clear_self_claim_tag(&cti_desc.access); drvdata->csdev = coresight_register(&cti_desc); diff --git a/drivers/hwtracing/coresight/coresight-dummy.c b/drivers/hwtracing/coresight/coresight-dummy.c index 14322c99e29d4696bc27ac2326280026412fe05f..b9a553561437f5bc880a4b1602e34293d083c952 100644 --- a/drivers/hwtracing/coresight/coresight-dummy.c +++ b/drivers/hwtracing/coresight/coresight-dummy.c @@ -179,6 +179,7 @@ static int dummy_probe(struct platform_device *pdev)
desc.pdata = pdev->dev.platform_data; desc.dev = &pdev->dev; + desc.cpu = -1; drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) { ret = PTR_ERR(drvdata->csdev); diff --git a/drivers/hwtracing/coresight/coresight-etb10.c b/drivers/hwtracing/coresight/coresight-etb10.c index 6657602d8f2e1b75ea6a8440e35c489d21f4245e..84951468db8bfbcbcf05d21cb1e91bf2603372a6 100644 --- a/drivers/hwtracing/coresight/coresight-etb10.c +++ b/drivers/hwtracing/coresight/coresight-etb10.c @@ -770,6 +770,7 @@ static int etb_probe(struct amba_device *adev, const struct amba_id *id) desc.pdata = pdata; desc.dev = dev; desc.groups = coresight_etb_groups; + desc.cpu = -1;
coresight_clear_self_claim_tag(&desc.access); drvdata->csdev = coresight_register(&desc); diff --git a/drivers/hwtracing/coresight/coresight-etm3x-core.c b/drivers/hwtracing/coresight/coresight-etm3x-core.c index a5e809589d3e382acde24ee457e94e5dcb18ea35..42115c4697d77d5dd86dc01a4da6b7ea076d4558 100644 --- a/drivers/hwtracing/coresight/coresight-etm3x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm3x-core.c @@ -886,6 +886,7 @@ static int etm_probe(struct amba_device *adev, const struct amba_id *id) desc.pdata = pdata; desc.dev = dev; desc.groups = coresight_etm_groups; + desc.cpu = drvdata->cpu; drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) return PTR_ERR(drvdata->csdev); diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 560975b70474c3449aa6799fc1a8e05d5ea93495..a3158223de78866f9a7f8180922cf8e7995d048b 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -2231,6 +2231,7 @@ static int etm4_add_coresight_dev(struct etm4_init_arg *init_arg) desc.pdata = pdata; desc.dev = dev; desc.groups = coresight_etmv4_groups; + desc.cpu = drvdata->cpu; drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) return PTR_ERR(drvdata->csdev); diff --git a/drivers/hwtracing/coresight/coresight-funnel.c b/drivers/hwtracing/coresight/coresight-funnel.c index 3b248e54471a38f501777fe162fea850d1c851b3..457291bb24212c502de9aadecb41da218c733c2f 100644 --- a/drivers/hwtracing/coresight/coresight-funnel.c +++ b/drivers/hwtracing/coresight/coresight-funnel.c @@ -263,6 +263,7 @@ static int funnel_probe(struct device *dev, struct resource *res) desc.ops = &funnel_cs_ops; desc.pdata = pdata; desc.dev = dev; + desc.cpu = -1; drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) return PTR_ERR(drvdata->csdev); diff --git a/drivers/hwtracing/coresight/coresight-replicator.c b/drivers/hwtracing/coresight/coresight-replicator.c index e6472658235dc479cec91ac18f3737f76f8c74f0..38518ba862928ea904c953c8303559e349e45200 100644 --- a/drivers/hwtracing/coresight/coresight-replicator.c +++ b/drivers/hwtracing/coresight/coresight-replicator.c @@ -273,6 +273,7 @@ static int replicator_probe(struct device *dev, struct resource *res) desc.ops = &replicator_cs_ops; desc.pdata = dev->platform_data; desc.dev = dev; + desc.cpu = -1;
drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) diff --git a/drivers/hwtracing/coresight/coresight-stm.c b/drivers/hwtracing/coresight/coresight-stm.c index e68529bf89c9815a8118955bf3114ad1ed4fb346..e2197ae0a7c8412785ce8b1161311048cbaa3f58 100644 --- a/drivers/hwtracing/coresight/coresight-stm.c +++ b/drivers/hwtracing/coresight/coresight-stm.c @@ -901,6 +901,7 @@ static int __stm_probe(struct device *dev, struct resource *res) desc.pdata = pdata; desc.dev = dev; desc.groups = coresight_stm_groups; + desc.cpu = -1; drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) { ret = PTR_ERR(drvdata->csdev); diff --git a/drivers/hwtracing/coresight/coresight-sysfs.c b/drivers/hwtracing/coresight/coresight-sysfs.c index d2a6ed8bcc74d64dccc735463f14790b4e80d101..7c7d3a3cb9f18a9dfa78eb5c19dda43fa6656271 100644 --- a/drivers/hwtracing/coresight/coresight-sysfs.c +++ b/drivers/hwtracing/coresight/coresight-sysfs.c @@ -162,6 +162,9 @@ static int coresight_validate_source_sysfs(struct coresight_device *csdev, return -EINVAL; }
+ if (coresight_is_percpu_source(csdev) && !cpu_online(csdev->cpu)) + return -ENODEV; + return 0; }
diff --git a/drivers/hwtracing/coresight/coresight-tmc-core.c b/drivers/hwtracing/coresight/coresight-tmc-core.c index 36599c431be6203e871fdcb8de569cc6701c52bb..bd783c9dcb56afa0adfa9724e45fcd0c4cfe367b 100644 --- a/drivers/hwtracing/coresight/coresight-tmc-core.c +++ b/drivers/hwtracing/coresight/coresight-tmc-core.c @@ -868,6 +868,7 @@ static int __tmc_probe(struct device *dev, struct resource *res) } dev->platform_data = pdata; desc.pdata = pdata; + desc.cpu = -1;
coresight_clear_self_claim_tag(&desc.access); drvdata->csdev = coresight_register(&desc); diff --git a/drivers/hwtracing/coresight/coresight-tnoc.c b/drivers/hwtracing/coresight/coresight-tnoc.c index ff9a0a9cfe96e5f5e3077c750ea2f890cdd50d94..b310f832506db5cd9f2134ff66909f64d4c785ce 100644 --- a/drivers/hwtracing/coresight/coresight-tnoc.c +++ b/drivers/hwtracing/coresight/coresight-tnoc.c @@ -199,6 +199,8 @@ static int trace_noc_probe(struct amba_device *adev, const struct amba_id *id) desc.dev = &adev->dev; desc.access = CSDEV_ACCESS_IOMEM(drvdata->base); desc.groups = coresight_tnoc_groups; + desc.cpu = -1; + drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) { coresight_trace_id_put_system_id(drvdata->atid); diff --git a/drivers/hwtracing/coresight/coresight-tpda.c b/drivers/hwtracing/coresight/coresight-tpda.c index 3a3825d27f861585ca1d847929747f8096004089..430e4c451479443a6cff4f6940e6b0c66ce8806d 100644 --- a/drivers/hwtracing/coresight/coresight-tpda.c +++ b/drivers/hwtracing/coresight/coresight-tpda.c @@ -317,6 +317,7 @@ static int tpda_probe(struct amba_device *adev, const struct amba_id *id) desc.pdata = adev->dev.platform_data; desc.dev = &adev->dev; desc.access = CSDEV_ACCESS_IOMEM(base); + desc.cpu = -1; drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) return PTR_ERR(drvdata->csdev); diff --git a/drivers/hwtracing/coresight/coresight-tpdm.c b/drivers/hwtracing/coresight/coresight-tpdm.c index 06e0a905a67d7aa36625960711c28717b1d27fda..bb813970c3aaa2e9220e45fd37af0a286f74a74a 100644 --- a/drivers/hwtracing/coresight/coresight-tpdm.c +++ b/drivers/hwtracing/coresight/coresight-tpdm.c @@ -1425,6 +1425,7 @@ static int tpdm_probe(struct device *dev, struct resource *res) desc.pdata = dev->platform_data; desc.dev = dev; desc.access = CSDEV_ACCESS_IOMEM(base); + desc.cpu = -1; if (res) desc.groups = tpdm_attr_grps; drvdata->csdev = coresight_register(&desc); diff --git a/drivers/hwtracing/coresight/coresight-tpiu.c b/drivers/hwtracing/coresight/coresight-tpiu.c index aaa44bc521c330f5b2e385279e0cbda6ce72bccb..d83d5bf4cf0f30ccee150b2635dacee3dcf0d24c 100644 --- a/drivers/hwtracing/coresight/coresight-tpiu.c +++ b/drivers/hwtracing/coresight/coresight-tpiu.c @@ -171,6 +171,7 @@ static int __tpiu_probe(struct device *dev, struct resource *res) desc.ops = &tpiu_cs_ops; desc.pdata = pdata; desc.dev = dev; + desc.cpu = -1; drvdata->csdev = coresight_register(&desc);
if (!IS_ERR(drvdata->csdev)) diff --git a/drivers/hwtracing/coresight/coresight-trbe.c b/drivers/hwtracing/coresight/coresight-trbe.c index 293715b4ff0eb0abe30f9b477700ca94f81cd4a2..f3e3a39a792f51876f230774658935f031a36169 100644 --- a/drivers/hwtracing/coresight/coresight-trbe.c +++ b/drivers/hwtracing/coresight/coresight-trbe.c @@ -1289,6 +1289,7 @@ static void arm_trbe_register_coresight_cpu(struct trbe_drvdata *drvdata, int cp desc.ops = &arm_trbe_cs_ops; desc.groups = arm_trbe_groups; desc.dev = dev; + desc.cpu = cpu; trbe_csdev = coresight_register(&desc); if (IS_ERR(trbe_csdev)) goto cpu_clear; diff --git a/drivers/hwtracing/coresight/ultrasoc-smb.c b/drivers/hwtracing/coresight/ultrasoc-smb.c index 8f7922a5e534fde57ae4c84f2e72b33749809857..61044bdb3bf785d8938eb3b124072c856bdc2c59 100644 --- a/drivers/hwtracing/coresight/ultrasoc-smb.c +++ b/drivers/hwtracing/coresight/ultrasoc-smb.c @@ -484,6 +484,7 @@ static int smb_register_sink(struct platform_device *pdev, return -ENOMEM; } desc.access = CSDEV_ACCESS_IOMEM(drvdata->base); + desc.cpu = -1;
drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) diff --git a/include/linux/coresight.h b/include/linux/coresight.h index 2b48be97fcd0d7ea2692206692bd33f35ba4ec79..fcea4f3253671e371570c29a65ef8ec96034f249 100644 --- a/include/linux/coresight.h +++ b/include/linux/coresight.h @@ -153,6 +153,7 @@ struct csdev_access { * in the component's sysfs sub-directory. * @name: name for the coresight device, also shown under sysfs. * @access: Describe access to the device + * @cpu: The CPU this component is affined to (-1 for not CPU bound). */ struct coresight_desc { enum coresight_dev_type type; @@ -163,6 +164,7 @@ struct coresight_desc { const struct attribute_group **groups; const char *name; struct csdev_access access; + int cpu; };
/** @@ -261,6 +263,7 @@ struct coresight_trace_id_map { * CS_MODE_SYSFS. Otherwise it must be accessed from inside the * spinlock. * @orphan: true if the component has connections that haven't been linked. + * @cpu: The CPU this component is affined to (-1 for not CPU bound). * @sysfs_sink_activated: 'true' when a sink has been selected for use via sysfs * by writing a 1 to the 'enable_sink' file. A sink can be * activated but not yet enabled. Enabling for a _sink_ happens @@ -287,6 +290,7 @@ struct coresight_device { atomic_t mode; int refcnt; bool orphan; + int cpu; /* sink specific fields */ bool sysfs_sink_activated; struct dev_ext_attribute *ea;
Add coresight_set_percpu_source() to set the CPU source device. Source devices are stored in the per-CPU structure csdev_source.
When the CPU is online, set the per-CPU pointer on the local CPU to avoid race conditions between setting and fetching the same pointer.
Reviewed-by: Mike Leach mike.leach@linaro.org Tested-by: James Clark james.clark@linaro.org Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index 0c48f5c85db6a7ec0377335eaa734527959d57c4..f22fd89f29b97ffe7004a2c988b09d5fd412af8d 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -34,6 +34,7 @@ */ DEFINE_MUTEX(coresight_mutex); static DEFINE_PER_CPU(struct coresight_device *, csdev_sink); +static DEFINE_PER_CPU(struct coresight_device *, csdev_source);
/** * struct coresight_node - elements of a path, from source to sink @@ -79,6 +80,22 @@ struct coresight_device *coresight_get_percpu_sink(int cpu) } EXPORT_SYMBOL_GPL(coresight_get_percpu_sink);
+static void coresight_set_percpu_source_local(void *csdev) +{ + this_cpu_write(csdev_source, csdev); +} + +static void coresight_set_percpu_source(int cpu, struct coresight_device *csdev) +{ + cpus_read_lock(); + if (cpu_online(cpu)) + smp_call_function_single(cpu, coresight_set_percpu_source_local, + csdev, 1); + else + per_cpu(csdev_source, cpu) = csdev; + cpus_read_unlock(); +} + static struct coresight_device *coresight_get_source(struct coresight_path *path) { struct coresight_device *csdev; @@ -1397,6 +1414,10 @@ struct coresight_device *coresight_register(struct coresight_desc *desc) goto out_unlock; } } + + if (coresight_is_percpu_source(csdev)) + coresight_set_percpu_source(csdev->cpu, csdev); + /* Device is now registered */ registered = true;
@@ -1428,6 +1449,8 @@ EXPORT_SYMBOL_GPL(coresight_register);
void coresight_unregister(struct coresight_device *csdev) { + if (coresight_is_percpu_source(csdev)) + coresight_set_percpu_source(csdev->cpu, NULL); etm_perf_del_symlink_sink(csdev); /* Remove references of that device in the topology */ if (cti_assoc_ops && cti_assoc_ops->remove)
The current implementation only saves and restores the context for ETM sources while ignoring the context of links. However, if funnels or replicators on a linked path resides in a CPU or cluster power domain, the hardware context for the link will be lost after resuming from low power states.
To support context management for links during CPU low power modes, a better way is to implement CPU PM callbacks in the Arm CoreSight core layer. As the core layer has sufficient information for linked paths, from tracers to links, which can be used for power management.
As a first step, this patch registers CPU PM notifier in the core layer. If a source device provides callbacks for saving and restoring context, these callbacks will be invoked in CPU suspend and resume.
Further changes will extend for controlling path.
Reviewed-by: James Clark james.clark@linaro.org Tested-by: James Clark james.clark@linaro.org Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 70 ++++++++++++++++++++++++++++ include/linux/coresight.h | 2 + 2 files changed, 72 insertions(+)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index f22fd89f29b97ffe7004a2c988b09d5fd412af8d..2aa1bbc53731637cccbf82ff83702ddc5ff80069 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -6,6 +6,7 @@ #include <linux/acpi.h> #include <linux/bitfield.h> #include <linux/build_bug.h> +#include <linux/cpu_pm.h> #include <linux/kernel.h> #include <linux/init.h> #include <linux/types.h> @@ -1590,6 +1591,68 @@ char *coresight_alloc_device_name(struct coresight_dev_list *dict, } EXPORT_SYMBOL_GPL(coresight_alloc_device_name);
+static bool coresight_pm_is_needed(struct coresight_device *csdev) +{ + if (!csdev) + return false; + + /* pm_save_disable() and pm_restore_enable() must be paired */ + if (!coresight_ops(csdev)->pm_save_disable || + !coresight_ops(csdev)->pm_restore_enable) + return false; + + return true; +} + +static int coresight_pm_save(struct coresight_device *csdev) +{ + return coresight_ops(csdev)->pm_save_disable(csdev); +} + +static void coresight_pm_restore(struct coresight_device *csdev) +{ + coresight_ops(csdev)->pm_restore_enable(csdev); +} + +static int coresight_cpu_pm_notify(struct notifier_block *nb, unsigned long cmd, + void *v) +{ + unsigned int cpu = smp_processor_id(); + struct coresight_device *source = per_cpu(csdev_source, cpu); + + if (!coresight_pm_is_needed(source)) + return NOTIFY_OK; + + switch (cmd) { + case CPU_PM_ENTER: + if (coresight_pm_save(source)) + return NOTIFY_BAD; + break; + case CPU_PM_EXIT: + case CPU_PM_ENTER_FAILED: + coresight_pm_restore(source); + break; + default: + return NOTIFY_DONE; + } + + return NOTIFY_OK; +} + +static struct notifier_block coresight_cpu_pm_nb = { + .notifier_call = coresight_cpu_pm_notify, +}; + +static int __init coresight_pm_setup(void) +{ + return cpu_pm_register_notifier(&coresight_cpu_pm_nb); +} + +static void coresight_pm_cleanup(void) +{ + cpu_pm_unregister_notifier(&coresight_cpu_pm_nb); +} + const struct bus_type coresight_bustype = { .name = "coresight", }; @@ -1644,9 +1707,15 @@ static int __init coresight_init(void)
/* initialise the coresight syscfg API */ ret = cscfg_init(); + if (ret) + goto exit_notifier; + + ret = coresight_pm_setup(); if (!ret) return 0;
+ cscfg_exit(); +exit_notifier: atomic_notifier_chain_unregister(&panic_notifier_list, &coresight_notifier); exit_perf: @@ -1658,6 +1727,7 @@ static int __init coresight_init(void)
static void __exit coresight_exit(void) { + coresight_pm_cleanup(); cscfg_exit(); atomic_notifier_chain_unregister(&panic_notifier_list, &coresight_notifier); diff --git a/include/linux/coresight.h b/include/linux/coresight.h index fcea4f3253671e371570c29a65ef8ec96034f249..25a31a4dd1486cefe6b442ad127954f8411ce949 100644 --- a/include/linux/coresight.h +++ b/include/linux/coresight.h @@ -442,6 +442,8 @@ struct coresight_ops_panic { struct coresight_ops { int (*trace_id)(struct coresight_device *csdev, enum cs_mode mode, struct coresight_device *sink); + int (*pm_save_disable)(struct coresight_device *csdev); + void (*pm_restore_enable)(struct coresight_device *csdev); const struct coresight_ops_sink *sink_ops; const struct coresight_ops_link *link_ops; const struct coresight_ops_source *source_ops;
Since the CoreSight core layer has registered CPU PM notifiers, this patch hooks CPU save and restore callbacks to be invoked from the core layer.
The CPU PM notifier in the ETMv4 driver is no longer needed, remove it along with its registration and unregistration code.
Reviewed-by: James Clark james.clark@linaro.org Tested-by: James Clark james.clark@linaro.org Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm4x-core.c | 59 ++++------------------ 1 file changed, 11 insertions(+), 48 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index a3158223de78866f9a7f8180922cf8e7995d048b..5d308afe737e9061ab75ee1c59c2b84a4a66e349 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -1179,11 +1179,6 @@ static const struct coresight_ops_source etm4_source_ops = { .pause_perf = etm4_pause_perf, };
-static const struct coresight_ops etm4_cs_ops = { - .trace_id = coresight_etm_get_trace_id, - .source_ops = &etm4_source_ops, -}; - static bool cpu_supports_sysreg_trace(void) { u64 dfr0 = read_sysreg_s(SYS_ID_AA64DFR0_EL1); @@ -1979,8 +1974,9 @@ static int __etm4_cpu_save(struct etmv4_drvdata *drvdata) return ret; }
-static int etm4_cpu_save(struct etmv4_drvdata *drvdata) +static int etm4_cpu_save(struct coresight_device *csdev) { + struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent); int ret = 0;
if (pm_save_enable != PARAM_PM_SAVE_SELF_HOSTED) @@ -2096,8 +2092,10 @@ static void __etm4_cpu_restore(struct etmv4_drvdata *drvdata) etm4_cs_lock(drvdata, csa); }
-static void etm4_cpu_restore(struct etmv4_drvdata *drvdata) +static void etm4_cpu_restore(struct coresight_device *csdev) { + struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent); + if (pm_save_enable != PARAM_PM_SAVE_SELF_HOSTED) return;
@@ -2105,38 +2103,11 @@ static void etm4_cpu_restore(struct etmv4_drvdata *drvdata) __etm4_cpu_restore(drvdata); }
-static int etm4_cpu_pm_notify(struct notifier_block *nb, unsigned long cmd, - void *v) -{ - struct etmv4_drvdata *drvdata; - unsigned int cpu = smp_processor_id(); - - if (!etmdrvdata[cpu]) - return NOTIFY_OK; - - drvdata = etmdrvdata[cpu]; - - if (WARN_ON_ONCE(drvdata->cpu != cpu)) - return NOTIFY_BAD; - - switch (cmd) { - case CPU_PM_ENTER: - if (etm4_cpu_save(drvdata)) - return NOTIFY_BAD; - break; - case CPU_PM_EXIT: - case CPU_PM_ENTER_FAILED: - etm4_cpu_restore(drvdata); - break; - default: - return NOTIFY_DONE; - } - - return NOTIFY_OK; -} - -static struct notifier_block etm4_cpu_pm_nb = { - .notifier_call = etm4_cpu_pm_notify, +static const struct coresight_ops etm4_cs_ops = { + .trace_id = coresight_etm_get_trace_id, + .pm_save_disable = etm4_cpu_save, + .pm_restore_enable = etm4_cpu_restore, + .source_ops = &etm4_source_ops, };
/* Setup PM. Deals with error conditions and counts */ @@ -2144,16 +2115,12 @@ static int __init etm4_pm_setup(void) { int ret;
- ret = cpu_pm_register_notifier(&etm4_cpu_pm_nb); - if (ret) - return ret; - ret = cpuhp_setup_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING, "arm/coresight4:starting", etm4_starting_cpu, etm4_dying_cpu);
if (ret) - goto unregister_notifier; + return ret;
ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, "arm/coresight4:online", @@ -2167,15 +2134,11 @@ static int __init etm4_pm_setup(void)
/* failed dyn state - remove others */ cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); - -unregister_notifier: - cpu_pm_unregister_notifier(&etm4_cpu_pm_nb); return ret; }
static void etm4_pm_clear(void) { - cpu_pm_unregister_notifier(&etm4_cpu_pm_nb); cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); if (hp_online) { cpuhp_remove_state_nocalls(hp_online);
Add a callback in the source device that returns a boolean indicating whether power management operations are required. The save and restore flow is skipped if the callback returns false.
The ETMv4 driver implements its own version's callback.
Reviewed-by: James Clark james.clark@linaro.org Tested-by: James Clark james.clark@linaro.org Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 9 ++++++++- drivers/hwtracing/coresight/coresight-etm4x-core.c | 16 ++++++++++++++++ include/linux/coresight.h | 1 + 3 files changed, 25 insertions(+), 1 deletion(-)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index 2aa1bbc53731637cccbf82ff83702ddc5ff80069..b2118a49ee688e194930de888b5283c4a6c33e2f 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -1601,7 +1601,14 @@ static bool coresight_pm_is_needed(struct coresight_device *csdev) !coresight_ops(csdev)->pm_restore_enable) return false;
- return true; + /* + * PM callbacks are provided but pm_is_neended() is absent, it means + * no extra check is needed. + */ + if (!coresight_ops(csdev)->pm_is_needed) + return true; + + return coresight_ops(csdev)->pm_is_needed(csdev); }
static int coresight_pm_save(struct coresight_device *csdev) diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 5d308afe737e9061ab75ee1c59c2b84a4a66e349..1c95484d7e2de2d0980e897f7f615b86ddd89f55 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -1832,6 +1832,21 @@ static int etm4_dying_cpu(unsigned int cpu) return 0; }
+static bool etm4_need_save_restore_context(struct coresight_device *csdev) +{ + if (pm_save_enable != PARAM_PM_SAVE_SELF_HOSTED) + return false; + + /* + * Save and restore the ETM Trace registers only if + * the ETM is active. + */ + if (coresight_get_mode(csdev)) + return true; + + return false; +} + static int __etm4_cpu_save(struct etmv4_drvdata *drvdata) { int i, ret = 0; @@ -2107,6 +2122,7 @@ static const struct coresight_ops etm4_cs_ops = { .trace_id = coresight_etm_get_trace_id, .pm_save_disable = etm4_cpu_save, .pm_restore_enable = etm4_cpu_restore, + .pm_is_needed = etm4_need_save_restore_context, .source_ops = &etm4_source_ops, };
diff --git a/include/linux/coresight.h b/include/linux/coresight.h index 25a31a4dd1486cefe6b442ad127954f8411ce949..e5ce9b956deeac1a303a7250b48a4083d580f55a 100644 --- a/include/linux/coresight.h +++ b/include/linux/coresight.h @@ -444,6 +444,7 @@ struct coresight_ops { struct coresight_device *sink); int (*pm_save_disable)(struct coresight_device *csdev); void (*pm_restore_enable)(struct coresight_device *csdev); + bool (*pm_is_needed)(struct coresight_device *csdev); const struct coresight_ops_sink *sink_ops; const struct coresight_ops_link *link_ops; const struct coresight_ops_source *source_ops;
Since a dedicated callback is now used to determine whether context save and restore operations are needed, performing the same check within the save and restore callbacks is redundant.
The save and restore flows currently use two-level functions: the first level handles the condition check, while the second level performs the low level operations. As the checks are no longer necessary, simplify the logic into single-level functions.
Reviewed-by: Mike Leach mike.leach@linaro.org Tested-by: James Clark james.clark@linaro.org Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm4x-core.c | 35 +++------------------- 1 file changed, 4 insertions(+), 31 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 1c95484d7e2de2d0980e897f7f615b86ddd89f55..a2803ad543d7f0995e1c0a0d0b8c9d11e92bbbe1 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -1847,11 +1847,11 @@ static bool etm4_need_save_restore_context(struct coresight_device *csdev) return false; }
-static int __etm4_cpu_save(struct etmv4_drvdata *drvdata) +static int etm4_cpu_save(struct coresight_device *csdev) { int i, ret = 0; struct etmv4_save_state *state; - struct coresight_device *csdev = drvdata->csdev; + struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent); struct csdev_access *csa; struct device *etm_dev;
@@ -1989,26 +1989,10 @@ static int __etm4_cpu_save(struct etmv4_drvdata *drvdata) return ret; }
-static int etm4_cpu_save(struct coresight_device *csdev) -{ - struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent); - int ret = 0; - - if (pm_save_enable != PARAM_PM_SAVE_SELF_HOSTED) - return 0; - - /* - * Save and restore the ETM Trace registers only if - * the ETM is active. - */ - if (coresight_get_mode(drvdata->csdev)) - ret = __etm4_cpu_save(drvdata); - return ret; -} - -static void __etm4_cpu_restore(struct etmv4_drvdata *drvdata) +static void etm4_cpu_restore(struct coresight_device *csdev) { int i; + struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent); struct etmv4_save_state *state = drvdata->save_state; struct csdev_access *csa = &drvdata->csdev->access;
@@ -2107,17 +2091,6 @@ static void __etm4_cpu_restore(struct etmv4_drvdata *drvdata) etm4_cs_lock(drvdata, csa); }
-static void etm4_cpu_restore(struct coresight_device *csdev) -{ - struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent); - - if (pm_save_enable != PARAM_PM_SAVE_SELF_HOSTED) - return; - - if (coresight_get_mode(drvdata->csdev)) - __etm4_cpu_restore(drvdata); -} - static const struct coresight_ops etm4_cs_ops = { .trace_id = coresight_etm_get_trace_id, .pm_save_disable = etm4_cpu_save,
To support dynamically enabling and disabling the CoreSight path during CPU idle, cscfg_config_sysfs_get_active_cfg() may be called by the ETM driver in an atomic context. However, the function currently acquires a mutex, which is a sleepable lock, and this is not allowed in atomic context.
To make cscfg_config_sysfs_get_active_cfg() called in the idle flow, replace the mutex with a raw spinlock to protect the active variables.
Reviewed-by: James Clark james.clark@linaro.org Tested-by: James Clark james.clark@linaro.org Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-syscfg.c | 22 +++++++++++++--------- drivers/hwtracing/coresight/coresight-syscfg.h | 2 ++ 2 files changed, 15 insertions(+), 9 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-syscfg.c b/drivers/hwtracing/coresight/coresight-syscfg.c index 6836b05986e8091b4f8ad7d082d6a016f8cf7d74..53aa1a44f9ac8f65b4e3cce06a33255abfc17351 100644 --- a/drivers/hwtracing/coresight/coresight-syscfg.c +++ b/drivers/hwtracing/coresight/coresight-syscfg.c @@ -957,15 +957,19 @@ int cscfg_config_sysfs_activate(struct cscfg_config_desc *config_desc, bool acti
cfg_hash = (unsigned long)config_desc->event_ea->var;
+ raw_spin_lock(&cscfg_mgr->spinlock); + if (activate) { + /* cannot be a current active value to activate this */ if (cscfg_mgr->sysfs_active_config) { err = -EBUSY; - goto exit_unlock; + } else { + err = _cscfg_activate_config(cfg_hash); + if (!err) + cscfg_mgr->sysfs_active_config = cfg_hash; } - err = _cscfg_activate_config(cfg_hash); - if (!err) - cscfg_mgr->sysfs_active_config = cfg_hash; + } else { /* disable if matching current value */ if (cscfg_mgr->sysfs_active_config == cfg_hash) { @@ -975,7 +979,8 @@ int cscfg_config_sysfs_activate(struct cscfg_config_desc *config_desc, bool acti err = -EINVAL; }
-exit_unlock: + raw_spin_unlock(&cscfg_mgr->spinlock); + mutex_unlock(&cscfg_mutex); return err; } @@ -983,9 +988,8 @@ int cscfg_config_sysfs_activate(struct cscfg_config_desc *config_desc, bool acti /* set the sysfs preset value */ void cscfg_config_sysfs_set_preset(int preset) { - mutex_lock(&cscfg_mutex); + guard(raw_spinlock)(&cscfg_mgr->spinlock); cscfg_mgr->sysfs_active_preset = preset; - mutex_unlock(&cscfg_mutex); }
/* @@ -994,10 +998,9 @@ void cscfg_config_sysfs_set_preset(int preset) */ void cscfg_config_sysfs_get_active_cfg(unsigned long *cfg_hash, int *preset) { - mutex_lock(&cscfg_mutex); + guard(raw_spinlock)(&cscfg_mgr->spinlock); *preset = cscfg_mgr->sysfs_active_preset; *cfg_hash = cscfg_mgr->sysfs_active_config; - mutex_unlock(&cscfg_mutex); } EXPORT_SYMBOL_GPL(cscfg_config_sysfs_get_active_cfg);
@@ -1201,6 +1204,7 @@ static int cscfg_create_device(void) INIT_LIST_HEAD(&cscfg_mgr->load_order_list); atomic_set(&cscfg_mgr->sys_active_cnt, 0); cscfg_mgr->load_state = CSCFG_NONE; + raw_spin_lock_init(&cscfg_mgr->spinlock);
/* setup the device */ dev = cscfg_device(); diff --git a/drivers/hwtracing/coresight/coresight-syscfg.h b/drivers/hwtracing/coresight/coresight-syscfg.h index 66e2db890d8203853a0c3c907b48aa66dd8014e6..f52aba4608dadc1c7666e0e28c9b43dc1c33d1eb 100644 --- a/drivers/hwtracing/coresight/coresight-syscfg.h +++ b/drivers/hwtracing/coresight/coresight-syscfg.h @@ -42,6 +42,7 @@ enum cscfg_load_ops { * @sysfs_active_config:Active config hash used if CoreSight controlled from sysfs. * @sysfs_active_preset:Active preset index used if CoreSight controlled from sysfs. * @load_state: A multi-stage load/unload operation is in progress. + * @spinlock: Exclusive access sysfs_active_* variables. */ struct cscfg_manager { struct device dev; @@ -54,6 +55,7 @@ struct cscfg_manager { u32 sysfs_active_config; int sysfs_active_preset; enum cscfg_load_ops load_state; + raw_spinlock_t spinlock; };
/* get reference to dev in cscfg_manager */
Introduce the coresight_enable_source() helper for enabling source device, refine the comment for the imbalance between enable and disable pair functions.
Reviewed-by: James Clark james.clark@linaro.org Tested-by: James Clark james.clark@linaro.org Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 15 ++++++++++++--- drivers/hwtracing/coresight/coresight-etm-perf.c | 2 +- drivers/hwtracing/coresight/coresight-priv.h | 3 +++ drivers/hwtracing/coresight/coresight-sysfs.c | 2 +- 4 files changed, 17 insertions(+), 5 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index b2118a49ee688e194930de888b5283c4a6c33e2f..2c287ceaa2aaf0dcb7a5243c8743f401f8500100 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -400,15 +400,24 @@ static void coresight_disable_helpers(struct coresight_device *csdev, }
/* - * Helper function to call source_ops(csdev)->disable and also disable the - * helpers. + * coresight_enable_source() only enables the source but does nothing for the + * associated helpers. In contrast, coresight_disable_source() calls + * source_ops(csdev)->disable() and also disables the helpers. * * There is an imbalance between coresight_enable_path() and * coresight_disable_path(). Enabling also enables the source's helpers as part * of the path, but disabling always skips the first item in the path (which is * the source), so sources and their helpers don't get disabled as part of that - * function and we need the extra step here. + * function and we need the extra step in coresight_disable_source(). */ +int coresight_enable_source(struct coresight_device *csdev, + struct perf_event *event, enum cs_mode mode, + struct coresight_path *path) +{ + return source_ops(csdev)->enable(csdev, event, mode, path); +} +EXPORT_SYMBOL_GPL(coresight_enable_source); + void coresight_disable_source(struct coresight_device *csdev, void *data) { source_ops(csdev)->disable(csdev, data); diff --git a/drivers/hwtracing/coresight/coresight-etm-perf.c b/drivers/hwtracing/coresight/coresight-etm-perf.c index 17afa0f4cdeea94ddf428df4a502cefe02e382d5..4e64655cb7ebcf6377e43cf41faf3d6c8de6c2ef 100644 --- a/drivers/hwtracing/coresight/coresight-etm-perf.c +++ b/drivers/hwtracing/coresight/coresight-etm-perf.c @@ -531,7 +531,7 @@ static void etm_event_start(struct perf_event *event, int flags) goto fail_end_stop;
/* Finally enable the tracer */ - if (source_ops(csdev)->enable(csdev, event, CS_MODE_PERF, path)) + if (coresight_enable_source(csdev, event, CS_MODE_PERF, path)) goto fail_disable_path;
/* diff --git a/drivers/hwtracing/coresight/coresight-priv.h b/drivers/hwtracing/coresight/coresight-priv.h index fd896ac07942ec0a4c7acff9f32421352c1efef2..3137555518b5258f68246140877e08bc32b1e5aa 100644 --- a/drivers/hwtracing/coresight/coresight-priv.h +++ b/drivers/hwtracing/coresight/coresight-priv.h @@ -249,6 +249,9 @@ void coresight_add_helper(struct coresight_device *csdev,
void coresight_set_percpu_sink(int cpu, struct coresight_device *csdev); struct coresight_device *coresight_get_percpu_sink(int cpu); +int coresight_enable_source(struct coresight_device *csdev, + struct perf_event *event, enum cs_mode mode, + struct coresight_path *path); void coresight_disable_source(struct coresight_device *csdev, void *data); void coresight_pause_source(struct coresight_device *csdev); int coresight_resume_source(struct coresight_device *csdev); diff --git a/drivers/hwtracing/coresight/coresight-sysfs.c b/drivers/hwtracing/coresight/coresight-sysfs.c index 7c7d3a3cb9f18a9dfa78eb5c19dda43fa6656271..cb1d4f90a9c245b2c2f6e818c0f4d905b86477db 100644 --- a/drivers/hwtracing/coresight/coresight-sysfs.c +++ b/drivers/hwtracing/coresight/coresight-sysfs.c @@ -66,7 +66,7 @@ static int coresight_enable_source_sysfs(struct coresight_device *csdev, */ lockdep_assert_held(&coresight_mutex); if (coresight_get_mode(csdev) != CS_MODE_SYSFS) { - ret = source_ops(csdev)->enable(csdev, NULL, mode, path); + ret = coresight_enable_source(csdev, NULL, mode, path); if (ret) return ret; }
Store the active path in the source device structure for system tracers (e.g. STM).
Tested-by: James Clark james.clark@linaro.org Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 22 ++++++++++++++++++++-- include/linux/coresight.h | 2 ++ 2 files changed, 22 insertions(+), 2 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index 2c287ceaa2aaf0dcb7a5243c8743f401f8500100..e8efb09dd8fc5d94d7a07ddd86f3270bab424f54 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -414,14 +414,32 @@ int coresight_enable_source(struct coresight_device *csdev, struct perf_event *event, enum cs_mode mode, struct coresight_path *path) { - return source_ops(csdev)->enable(csdev, event, mode, path); + int ret; + + + ret = source_ops(csdev)->enable(csdev, event, mode, path); + if (ret) + return ret; + + /* + * The per-CPU source has updated its path pointer in the enable() + * callback, ensuring synchronization on the target CPU. Set the + * path pointer here for non per-CPU sources. + */ + if (!coresight_is_percpu_source(csdev)) + csdev->path = path; + + return 0; } EXPORT_SYMBOL_GPL(coresight_enable_source);
void coresight_disable_source(struct coresight_device *csdev, void *data) { source_ops(csdev)->disable(csdev, data); - coresight_disable_helpers(csdev, NULL); + coresight_disable_helpers(csdev, csdev->path); + + if (!coresight_is_percpu_source(csdev)) + csdev->path = NULL; } EXPORT_SYMBOL_GPL(coresight_disable_source);
diff --git a/include/linux/coresight.h b/include/linux/coresight.h index e5ce9b956deeac1a303a7250b48a4083d580f55a..9e1edff3ace951146f1801a732105d8560fa2356 100644 --- a/include/linux/coresight.h +++ b/include/linux/coresight.h @@ -264,6 +264,7 @@ struct coresight_trace_id_map { * spinlock. * @orphan: true if the component has connections that haven't been linked. * @cpu: The CPU this component is affined to (-1 for not CPU bound). + * @path: Activated path pointer (only used for source device). * @sysfs_sink_activated: 'true' when a sink has been selected for use via sysfs * by writing a 1 to the 'enable_sink' file. A sink can be * activated but not yet enabled. Enabling for a _sink_ happens @@ -291,6 +292,7 @@ struct coresight_device { int refcnt; bool orphan; int cpu; + struct coresight_path *path; /* sink specific fields */ bool sysfs_sink_activated; struct dev_ext_attribute *ea;
Set and clear the path pointer on the target CPU during ETM enable and disable operations to avoid race conditions when retrieving the pointer in CPU PM callbacks.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm4x-core.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index a2803ad543d7f0995e1c0a0d0b8c9d11e92bbbe1..44673446ba1f4745f0601b646e10546cce58a3e4 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -240,6 +240,7 @@ void etm4_release_trace_id(struct etmv4_drvdata *drvdata)
struct etm4_enable_arg { struct etmv4_drvdata *drvdata; + struct coresight_path *path; int rc; };
@@ -627,8 +628,12 @@ static void etm4_enable_sysfs_smp_call(void *info) arg->rc = etm4_enable_hw(arg->drvdata);
/* The tracer didn't start */ - if (arg->rc) + if (arg->rc) { coresight_set_mode(csdev, CS_MODE_DISABLED); + return; + } + + csdev->path = arg->path; }
/* @@ -865,9 +870,13 @@ static int etm4_enable_perf(struct coresight_device *csdev,
out: /* Failed to start tracer; roll back to DISABLED mode */ - if (ret) + if (ret) { coresight_set_mode(csdev, CS_MODE_DISABLED); - return ret; + return ret; + } + + csdev->path = path; + return 0; }
static int etm4_enable_sysfs(struct coresight_device *csdev, struct coresight_path *path) @@ -897,6 +906,7 @@ static int etm4_enable_sysfs(struct coresight_device *csdev, struct coresight_pa * ensures that register writes occur when cpu is powered. */ arg.drvdata = drvdata; + arg.path = path; ret = smp_call_function_single(drvdata->cpu, etm4_enable_sysfs_smp_call, &arg, 1); if (!ret) @@ -1038,6 +1048,7 @@ static void etm4_disable_sysfs_smp_call(void *info)
etm4_disable_hw(drvdata);
+ drvdata->csdev->path = NULL; coresight_set_mode(drvdata->csdev, CS_MODE_DISABLED); }
@@ -1070,6 +1081,7 @@ static int etm4_disable_perf(struct coresight_device *csdev, /* TRCVICTLR::SSSTATUS, bit[9] */ filters->ssstatus = (control & BIT(9));
+ drvdata->csdev->path = NULL; coresight_set_mode(drvdata->csdev, CS_MODE_DISABLED);
/*
Set and clear the path pointer on the target CPU during ETM enable and disable operations.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm3x-core.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm3x-core.c b/drivers/hwtracing/coresight/coresight-etm3x-core.c index 42115c4697d77d5dd86dc01a4da6b7ea076d4558..60d93ba2da0f0810720f6c01fde082475eeefa7e 100644 --- a/drivers/hwtracing/coresight/coresight-etm3x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm3x-core.c @@ -436,6 +436,7 @@ static int etm_enable_hw(struct etm_drvdata *drvdata)
struct etm_enable_arg { struct etm_drvdata *drvdata; + struct coresight_path *path; int rc; };
@@ -457,8 +458,12 @@ static void etm_enable_sysfs_smp_call(void *info) arg->rc = etm_enable_hw(arg->drvdata);
/* The tracer didn't start */ - if (arg->rc) + if (arg->rc) { coresight_set_mode(csdev, CS_MODE_DISABLED); + return; + } + + csdev->path = arg->path; }
static int etm_cpu_id(struct coresight_device *csdev) @@ -494,10 +499,13 @@ static int etm_enable_perf(struct coresight_device *csdev, ret = etm_enable_hw(drvdata);
/* Failed to start tracer; roll back to DISABLED mode */ - if (ret) + if (ret) { coresight_set_mode(csdev, CS_MODE_DISABLED); + return ret; + }
- return ret; + csdev->path = path; + return 0; }
static int etm_enable_sysfs(struct coresight_device *csdev, struct coresight_path *path) @@ -585,6 +593,7 @@ static void etm_disable_sysfs_smp_call(void *info)
etm_disable_hw(drvdata);
+ drvdata->csdev->path = NULL; coresight_set_mode(drvdata->csdev, CS_MODE_DISABLED); }
@@ -609,6 +618,7 @@ static void etm_disable_perf(struct coresight_device *csdev)
CS_LOCK(drvdata->csa.base);
+ drvdata->csdev->path = NULL; coresight_set_mode(drvdata->csdev, CS_MODE_DISABLED);
/*
Since the path pointer is stored in the source's structure, retrieve it directly when disabling the path.
As a result, the global variables used for caching path pointers are no longer needed. Remove them to simplify the code.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-sysfs.c | 79 +++------------------------ 1 file changed, 9 insertions(+), 70 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-sysfs.c b/drivers/hwtracing/coresight/coresight-sysfs.c index cb1d4f90a9c245b2c2f6e818c0f4d905b86477db..9e94505bccf4ac91207f2f5393ae57ae077d61d6 100644 --- a/drivers/hwtracing/coresight/coresight-sysfs.c +++ b/drivers/hwtracing/coresight/coresight-sysfs.c @@ -12,19 +12,6 @@ #include "coresight-priv.h" #include "coresight-trace-id.h"
-/* - * Use IDR to map the hash of the source's device name - * to the pointer of path for the source. The idr is for - * the sources which aren't associated with CPU. - */ -static DEFINE_IDR(path_idr); - -/* - * When operating Coresight drivers from the sysFS interface, only a single - * path can exist from a tracer (associated to a CPU) to a sink. - */ -static DEFINE_PER_CPU(struct coresight_path *, tracer_path); - ssize_t coresight_simple_show_pair(struct device *_dev, struct device_attribute *attr, char *buf) { @@ -170,11 +157,10 @@ static int coresight_validate_source_sysfs(struct coresight_device *csdev,
int coresight_enable_sysfs(struct coresight_device *csdev) { - int cpu, ret = 0; + int ret = 0; struct coresight_device *sink; struct coresight_path *path; enum coresight_dev_subtype_source subtype; - u32 hash;
subtype = csdev->subtype.source_subtype;
@@ -226,35 +212,6 @@ int coresight_enable_sysfs(struct coresight_device *csdev) if (ret) goto err_source;
- switch (subtype) { - case CORESIGHT_DEV_SUBTYPE_SOURCE_PROC: - /* - * When working from sysFS it is important to keep track - * of the paths that were created so that they can be - * undone in 'coresight_disable()'. Since there can only - * be a single session per tracer (when working from sysFS) - * a per-cpu variable will do just fine. - */ - cpu = source_ops(csdev)->cpu_id(csdev); - per_cpu(tracer_path, cpu) = path; - break; - case CORESIGHT_DEV_SUBTYPE_SOURCE_SOFTWARE: - case CORESIGHT_DEV_SUBTYPE_SOURCE_TPDM: - case CORESIGHT_DEV_SUBTYPE_SOURCE_OTHERS: - /* - * Use the hash of source's device name as ID - * and map the ID to the pointer of the path. - */ - hash = hashlen_hash(hashlen_string(NULL, dev_name(&csdev->dev))); - ret = idr_alloc_u32(&path_idr, path, &hash, hash, GFP_KERNEL); - if (ret) - goto err_source; - break; - default: - /* We can't be here */ - break; - } - out: mutex_unlock(&coresight_mutex); return ret; @@ -270,9 +227,8 @@ EXPORT_SYMBOL_GPL(coresight_enable_sysfs);
void coresight_disable_sysfs(struct coresight_device *csdev) { - int cpu, ret; - struct coresight_path *path = NULL; - u32 hash; + int ret; + struct coresight_path *path;
mutex_lock(&coresight_mutex);
@@ -280,32 +236,15 @@ void coresight_disable_sysfs(struct coresight_device *csdev) if (ret) goto out;
+ /* + * coresight_disable_source_sysfs() clears the 'csdev->path' pointer + * when disabling the source. Retrieve the path pointer here. + */ + path = csdev->path; + if (!coresight_disable_source_sysfs(csdev, NULL)) goto out;
- switch (csdev->subtype.source_subtype) { - case CORESIGHT_DEV_SUBTYPE_SOURCE_PROC: - cpu = source_ops(csdev)->cpu_id(csdev); - path = per_cpu(tracer_path, cpu); - per_cpu(tracer_path, cpu) = NULL; - break; - case CORESIGHT_DEV_SUBTYPE_SOURCE_SOFTWARE: - case CORESIGHT_DEV_SUBTYPE_SOURCE_TPDM: - case CORESIGHT_DEV_SUBTYPE_SOURCE_OTHERS: - hash = hashlen_hash(hashlen_string(NULL, dev_name(&csdev->dev))); - /* Find the path by the hash. */ - path = idr_find(&path_idr, hash); - if (path == NULL) { - pr_err("Path is not found for %s\n", dev_name(&csdev->dev)); - goto out; - } - idr_remove(&path_idr, hash); - break; - default: - /* We can't be here */ - break; - } - coresight_disable_path(path); coresight_release_path(path);
Introduce an 'in_idle' argument to the path. When set to true for idle flow, it skips to touch the sink device to avoid long latency caused by the sink operations.
This is a preparation for managing the path during CPU idle.
Tested-by: James Clark james.clark@linaro.org Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 17 +++++++++++++---- include/linux/coresight.h | 2 ++ 2 files changed, 15 insertions(+), 4 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index e8efb09dd8fc5d94d7a07ddd86f3270bab424f54..e3efa7c9ef9fa1bf7b2f727205abe577bfc493e3 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -494,6 +494,10 @@ static void coresight_disable_path_from(struct coresight_path *path, CORESIGHT_DEV_TYPE_SINK : CORESIGHT_DEV_TYPE_LINK;
+ /* To reduce latency, CPU idle does not touch the sink */ + if (path->in_idle && type == CORESIGHT_DEV_TYPE_SINK) + continue; + switch (type) { case CORESIGHT_DEV_TYPE_SINK: coresight_disable_sink(csdev); @@ -560,10 +564,6 @@ int coresight_enable_path(struct coresight_path *path, enum cs_mode mode) csdev = nd->csdev; type = csdev->type;
- /* Enable all helpers adjacent to the path first */ - ret = coresight_enable_helpers(csdev, mode, path); - if (ret) - goto err_disable_path; /* * ETF devices are tricky... They can be a link or a sink, * depending on how they are configured. If an ETF has been @@ -575,6 +575,15 @@ int coresight_enable_path(struct coresight_path *path, enum cs_mode mode) CORESIGHT_DEV_TYPE_SINK : CORESIGHT_DEV_TYPE_LINK;
+ /* To reduce latency, CPU idle does not touch the sink */ + if (path->in_idle && type == CORESIGHT_DEV_TYPE_SINK) + continue; + + /* Enable all helpers adjacent to the path first */ + ret = coresight_enable_helpers(csdev, mode, path); + if (ret) + goto err_disable_path; + switch (type) { case CORESIGHT_DEV_TYPE_SINK: ret = coresight_enable_sink(csdev, mode, path); diff --git a/include/linux/coresight.h b/include/linux/coresight.h index 9e1edff3ace951146f1801a732105d8560fa2356..ba1bd6479230d660eaaf3cf76ab531f6dc2a6f4a 100644 --- a/include/linux/coresight.h +++ b/include/linux/coresight.h @@ -337,11 +337,13 @@ static struct coresight_dev_list (var) = { \ * @path_list: path from source to sink. * @trace_id: trace_id of the whole path. * @handle: handle of the aux_event. + * @in_idle: A flag to indicate if it is in CPU idle. */ struct coresight_path { struct list_head path_list; u8 trace_id; struct perf_output_handle *handle; + bool in_idle; };
enum cs_mode {
Control links and helpers on an activated path during CPU idle. Set the "path->in_idle" flag to true during idle to guide path control.
Since coresight_disable_path() doesn't handle a source device's helpers, explicitly disable them alongside the source device.
Reviewed-by: James Clark james.clark@linaro.org Tested-by: James Clark james.clark@linaro.org Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index e3efa7c9ef9fa1bf7b2f727205abe577bfc493e3..7a9f2c1b105d097cee846ba7ff54475483a0c8e0 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -1649,12 +1649,33 @@ static bool coresight_pm_is_needed(struct coresight_device *csdev)
static int coresight_pm_save(struct coresight_device *csdev) { - return coresight_ops(csdev)->pm_save_disable(csdev); + int ret; + + if (WARN_ON(!csdev->path)) + return -EINVAL; + + /* in_idle flag will be used for path controlling */ + csdev->path->in_idle = true; + + ret = coresight_ops(csdev)->pm_save_disable(csdev); + if (ret) { + csdev->path->in_idle = false; + return ret; + } + + coresight_disable_helpers(csdev, csdev->path); + coresight_disable_path(csdev->path); + return 0; }
static void coresight_pm_restore(struct coresight_device *csdev) { + if (WARN_ON(!csdev->path)) + return; + + coresight_enable_path(csdev->path, coresight_get_mode(csdev)); coresight_ops(csdev)->pm_restore_enable(csdev); + csdev->path->in_idle = false; }
static int coresight_cpu_pm_notify(struct notifier_block *nb, unsigned long cmd,
Unlike a system level's sink, the per-CPU sink may lose power during CPU idle states. Currently, this refers specifically to TRBE as the sink.
This commit registers save and restore callbacks for the sink via the PM notifier.
Tested-by: James Clark james.clark@linaro.org Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index 7a9f2c1b105d097cee846ba7ff54475483a0c8e0..ee97e4adbc395d4a4aa88846df91b0b53ae0bd8d 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -1647,6 +1647,22 @@ static bool coresight_pm_is_needed(struct coresight_device *csdev) return coresight_ops(csdev)->pm_is_needed(csdev); }
+static int coresight_pm_sink_save(struct coresight_device *sink) +{ + if (!sink || !coresight_ops(sink)->pm_save_disable) + return 0; + + return coresight_ops(sink)->pm_save_disable(sink); +} + +static void coresight_pm_sink_restore(struct coresight_device *sink) +{ + if (!sink || !coresight_ops(sink)->pm_restore_enable) + return; + + coresight_ops(sink)->pm_restore_enable(sink); +} + static int coresight_pm_save(struct coresight_device *csdev) { int ret; @@ -1665,7 +1681,18 @@ static int coresight_pm_save(struct coresight_device *csdev)
coresight_disable_helpers(csdev, csdev->path); coresight_disable_path(csdev->path); + + ret = coresight_pm_sink_save(coresight_get_sink(csdev->path)); + if (ret) + goto failed_out; + return 0; + +failed_out: + coresight_enable_path(csdev->path, coresight_get_mode(csdev)); + coresight_ops(csdev)->pm_restore_enable(csdev); + csdev->path->in_idle = false; + return ret; }
static void coresight_pm_restore(struct coresight_device *csdev) @@ -1673,6 +1700,7 @@ static void coresight_pm_restore(struct coresight_device *csdev) if (WARN_ON(!csdev->path)) return;
+ coresight_pm_sink_restore(coresight_get_sink(csdev->path)); coresight_enable_path(csdev->path, coresight_get_mode(csdev)); coresight_ops(csdev)->pm_restore_enable(csdev); csdev->path->in_idle = false;
From: Yabin Cui yabinc@google.com
Similar to ETE, TRBE may lose its context when a CPU enters low power state. To make things worse, if ETE is restored without TRBE being restored, an enabled source device with no enabled sink devices can cause CPU hang on some devices (e.g., Pixel 9).
The save and restore flows are described in the section K5.5 "Context switching" of Arm ARM (ARM DDI 0487 L.a). This commit adds save and restore callbacks with following the software usages defined in the architecture manual.
Signed-off-by: Yabin Cui yabinc@google.com Co-developed-by: Leo Yan leo.yan@arm.com Tested-by: James Clark james.clark@linaro.org Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-trbe.c | 61 +++++++++++++++++++++++++++- 1 file changed, 60 insertions(+), 1 deletion(-)
diff --git a/drivers/hwtracing/coresight/coresight-trbe.c b/drivers/hwtracing/coresight/coresight-trbe.c index f3e3a39a792f51876f230774658935f031a36169..91832ad2b2e9809c96f3c8994eaf2534c5bf88dc 100644 --- a/drivers/hwtracing/coresight/coresight-trbe.c +++ b/drivers/hwtracing/coresight/coresight-trbe.c @@ -116,6 +116,20 @@ static int trbe_errata_cpucaps[] = { */ #define TRBE_WORKAROUND_OVERWRITE_FILL_MODE_SKIP_BYTES 256
+/* + * struct trbe_save_state: Register values representing TRBE state + * @trblimitr - Trace Buffer Limit Address Register value + * @trbbaser - Trace Buffer Base Register value + * @trbptr - Trace Buffer Write Pointer Register value + * @trbsr - Trace Buffer Status Register value + */ +struct trbe_save_state { + u64 trblimitr; + u64 trbbaser; + u64 trbptr; + u64 trbsr; +}; + /* * struct trbe_cpudata: TRBE instance specific data * @trbe_flag - TRBE dirty/access flag support @@ -134,6 +148,7 @@ struct trbe_cpudata { enum cs_mode mode; struct trbe_buf *buf; struct trbe_drvdata *drvdata; + struct trbe_save_state save_state; DECLARE_BITMAP(errata, TRBE_ERRATA_MAX); };
@@ -1189,6 +1204,48 @@ static irqreturn_t arm_trbe_irq_handler(int irq, void *dev) return IRQ_HANDLED; }
+static int arm_trbe_save(struct coresight_device *csdev) +{ + struct trbe_cpudata *cpudata = dev_get_drvdata(&csdev->dev); + struct trbe_save_state *state = &cpudata->save_state; + + state->trblimitr = read_sysreg_s(SYS_TRBLIMITR_EL1); + + /* No need to save or restore context when the unit is disabled */ + if (!(state->trblimitr & TRBLIMITR_EL1_E)) + return 0; + + /* Disable the unit, ensure the writes to memory are complete */ + trbe_drain_and_disable_local(cpudata); + + state->trbbaser = read_sysreg_s(SYS_TRBBASER_EL1); + state->trbptr = read_sysreg_s(SYS_TRBPTR_EL1); + state->trbsr = read_sysreg_s(SYS_TRBSR_EL1); + return 0; +} + +static void arm_trbe_restore(struct coresight_device *csdev) +{ + struct trbe_cpudata *cpudata = dev_get_drvdata(&csdev->dev); + struct trbe_save_state *state = &cpudata->save_state; + + if (!(state->trblimitr & TRBLIMITR_EL1_E)) + return; + + write_sysreg_s(state->trbbaser, SYS_TRBBASER_EL1); + write_sysreg_s(state->trbptr, SYS_TRBPTR_EL1); + write_sysreg_s(state->trbsr, SYS_TRBSR_EL1); + + /* + * The section K5.5 Context switching, Arm ARM (ARM DDI 0487 L.a), the + * software usage PKLXF requires a Context synchronization event to + * guarantee the Trace Buffer Unit will observe the new values of the + * System registers. + */ + isb(); + set_trbe_enabled(cpudata, state->trblimitr); +} + static const struct coresight_ops_sink arm_trbe_sink_ops = { .enable = arm_trbe_enable, .disable = arm_trbe_disable, @@ -1198,7 +1255,9 @@ static const struct coresight_ops_sink arm_trbe_sink_ops = { };
static const struct coresight_ops arm_trbe_cs_ops = { - .sink_ops = &arm_trbe_sink_ops, + .pm_save_disable = arm_trbe_save, + .pm_restore_enable = arm_trbe_restore, + .sink_ops = &arm_trbe_sink_ops, };
static ssize_t align_show(struct device *dev, struct device_attribute *attr, char *buf)
Except for system tracers (e.g. STM), other sources treat multiple enables as equivalent to a single enable. The device mode already tracks the binary state, so it is redundant to operate refcount.
Refactor to maintain the refcount only for system sources. This simplifies future CPU PM handling without refcount logic.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-sysfs.c | 35 +++++++++++++++------------ 1 file changed, 19 insertions(+), 16 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-sysfs.c b/drivers/hwtracing/coresight/coresight-sysfs.c index 9e94505bccf4ac91207f2f5393ae57ae077d61d6..04c1425eb37d69c6feb8e49446fc58ee57a5e82e 100644 --- a/drivers/hwtracing/coresight/coresight-sysfs.c +++ b/drivers/hwtracing/coresight/coresight-sysfs.c @@ -58,7 +58,17 @@ static int coresight_enable_source_sysfs(struct coresight_device *csdev, return ret; }
- csdev->refcnt++; + /* + * There could be multiple applications driving the software + * source. So keep the refcount for each such user when the + * source is already enabled. + * + * No need to increment the reference counter for other source + * types, as multiple enables are the same as a single enable. + */ + if (csdev->subtype.source_subtype == + CORESIGHT_DEV_SUBTYPE_SOURCE_SOFTWARE) + csdev->refcnt++;
return 0; } @@ -80,7 +90,10 @@ static bool coresight_disable_source_sysfs(struct coresight_device *csdev, if (coresight_get_mode(csdev) != CS_MODE_SYSFS) return false;
- csdev->refcnt--; + if (csdev->subtype.source_subtype == + CORESIGHT_DEV_SUBTYPE_SOURCE_SOFTWARE) + csdev->refcnt--; + if (csdev->refcnt == 0) { coresight_disable_source(csdev, data); return true; @@ -159,10 +172,7 @@ int coresight_enable_sysfs(struct coresight_device *csdev) { int ret = 0; struct coresight_device *sink; - struct coresight_path *path; - enum coresight_dev_subtype_source subtype; - - subtype = csdev->subtype.source_subtype; + struct coresight_path *path = NULL;
mutex_lock(&coresight_mutex);
@@ -176,16 +186,8 @@ int coresight_enable_sysfs(struct coresight_device *csdev) * coresight_enable_source() so can still race with Perf mode which * doesn't hold coresight_mutex. */ - if (coresight_get_mode(csdev) == CS_MODE_SYSFS) { - /* - * There could be multiple applications driving the software - * source. So keep the refcount for each such user when the - * source is already enabled. - */ - if (subtype == CORESIGHT_DEV_SUBTYPE_SOURCE_SOFTWARE) - csdev->refcnt++; - goto out; - } + if (coresight_get_mode(csdev) == CS_MODE_SYSFS) + goto enable_source;
sink = coresight_find_activated_sysfs_sink(csdev); if (!sink) { @@ -208,6 +210,7 @@ int coresight_enable_sysfs(struct coresight_device *csdev) if (ret) goto err_path;
+enable_source: ret = coresight_enable_source_sysfs(csdev, CS_MODE_SYSFS, path); if (ret) goto err_source;
The hotplug lock is acquired and released in etm{3|4}_disable_sysfs(), which are low-level functions. This prevents us from a new solution for hotplug.
Firstly, hotplug callbacks cannot invoke etm{3|4}_disable_sysfs() to disable the source; otherwise, a deadlock issue occurs. The reason is that, in the hotplug flow, the kernel acquires the hotplug lock before calling callbacks. Subsequently, if coresight_disable_source() is invoked and it calls etm{3|4}_disable_sysfs(), the hotplug lock will be acquired twice, leading to a double lock issue.
Secondly, when hotplugging a CPU on or off, if we want to manipulate all components on a path attached to the CPU, we need to maintain atomicity for the entire path. Otherwise, a race condition may occur with users setting the same path via the Sysfs knobs, ultimately causing mess states in CoreSight components.
This patch moves the hotplug locking from etm{3|4}_disable_sysfs() into enable_source_store(). As a result, when users control the Sysfs knobs, the whole flow is protected by hotplug locking, ensuring it is mutual exclusive with hotplug callbacks.
Note, the paired function etm{3|4}_enable_sysfs() does not use hotplug locking, which is why this patch does not modify it.
Reviewed-by: Mike Leach mike.leach@linaro.org Tested-by: James Clark james.clark@linaro.org Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm3x-core.c | 8 -------- drivers/hwtracing/coresight/coresight-etm4x-core.c | 9 --------- drivers/hwtracing/coresight/coresight-sysfs.c | 7 +++++++ 3 files changed, 7 insertions(+), 17 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm3x-core.c b/drivers/hwtracing/coresight/coresight-etm3x-core.c index 60d93ba2da0f0810720f6c01fde082475eeefa7e..730d2584d625f11db87729352bcb80b9df1caa04 100644 --- a/drivers/hwtracing/coresight/coresight-etm3x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm3x-core.c @@ -632,13 +632,6 @@ static void etm_disable_sysfs(struct coresight_device *csdev) { struct etm_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
- /* - * Taking hotplug lock here protects from clocks getting disabled - * with tracing being left on (crash scenario) if user disable occurs - * after cpu online mask indicates the cpu is offline but before the - * DYING hotplug callback is serviced by the ETM driver. - */ - cpus_read_lock(); spin_lock(&drvdata->spinlock);
/* @@ -649,7 +642,6 @@ static void etm_disable_sysfs(struct coresight_device *csdev) drvdata, 1);
spin_unlock(&drvdata->spinlock); - cpus_read_unlock();
/* * we only release trace IDs when resetting sysfs. diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 44673446ba1f4745f0601b646e10546cce58a3e4..85f974f8635d0e0b548c7cb506c734a0c3d55f24 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -1096,13 +1096,6 @@ static void etm4_disable_sysfs(struct coresight_device *csdev) { struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
- /* - * Taking hotplug lock here protects from clocks getting disabled - * with tracing being left on (crash scenario) if user disable occurs - * after cpu online mask indicates the cpu is offline but before the - * DYING hotplug callback is serviced by the ETM driver. - */ - cpus_read_lock(); raw_spin_lock(&drvdata->spinlock);
/* @@ -1116,8 +1109,6 @@ static void etm4_disable_sysfs(struct coresight_device *csdev)
cscfg_csdev_disable_active_config(csdev);
- cpus_read_unlock(); - /* * we only release trace IDs when resetting sysfs. * This permits sysfs users to read the trace ID after the trace diff --git a/drivers/hwtracing/coresight/coresight-sysfs.c b/drivers/hwtracing/coresight/coresight-sysfs.c index 04c1425eb37d69c6feb8e49446fc58ee57a5e82e..c8d45f69072546bc5ae61e22646000cd47530dba 100644 --- a/drivers/hwtracing/coresight/coresight-sysfs.c +++ b/drivers/hwtracing/coresight/coresight-sysfs.c @@ -305,6 +305,13 @@ static ssize_t enable_source_store(struct device *dev, if (ret) return ret;
+ /* + * CoreSight hotplug callbacks in core layer control a activated path + * from its source to sink. Taking hotplug lock here protects a race + * condition with hotplug callbacks. + */ + guard(cpus_read_lock)(); + if (val) { ret = coresight_enable_sysfs(csdev); if (ret)
This commit moves CPU hotplug callbacks from ETMv4 driver to core layer. The motivation is the core layer can control all components on an activated path rather but not only managing tracer in ETMv4 driver.
The perf event layer will disable CoreSight PMU event 'cs_etm' when hotplug off a CPU. That means a perf mode will be always converted to disabled mode in CPU hotplug. Arm CoreSight CPU hotplug callbacks only need to handle the Sysfs mode and ignore the perf mode.
The core layer disables and releases the active path when hotplug-off a CPU. When hotplug-in a CPU, it does not re-enable the path, in this case, users need to re-enable the trace via Sysfs interface.
Tested-by: James Clark james.clark@linaro.org Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 42 +++++++++++++++++++++- drivers/hwtracing/coresight/coresight-etm3x-core.c | 40 --------------------- drivers/hwtracing/coresight/coresight-etm4x-core.c | 37 ------------------- 3 files changed, 41 insertions(+), 78 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index ee97e4adbc395d4a4aa88846df91b0b53ae0bd8d..86beb9adefaa22007572b79c2e5d70cc4b731840 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -1706,6 +1706,33 @@ static void coresight_pm_restore(struct coresight_device *csdev) csdev->path->in_idle = false; }
+static int coresight_dying_cpu(unsigned int cpu) +{ + struct coresight_device *source = per_cpu(csdev_source, cpu); + struct coresight_path *path; + + if (!source || !source->path) + return 0; + + /* + * The perf event layer will disable PMU events in the CPU hotplug. + * CoreSight driver should never handle the CS_MODE_PERF case. + */ + if (coresight_get_mode(source) != CS_MODE_SYSFS) + return 0; + + /* + * Save 'source->path' here, as it will be cleared in + * coresight_disable_source(). + */ + path = source->path; + + coresight_disable_source(source, NULL); + coresight_disable_path(path); + coresight_release_path(path); + return 0; +} + static int coresight_cpu_pm_notify(struct notifier_block *nb, unsigned long cmd, void *v) { @@ -1737,11 +1764,24 @@ static struct notifier_block coresight_cpu_pm_nb = {
static int __init coresight_pm_setup(void) { - return cpu_pm_register_notifier(&coresight_cpu_pm_nb); + int ret; + + ret = cpu_pm_register_notifier(&coresight_cpu_pm_nb); + if (ret) + return ret; + + ret = cpuhp_setup_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING, + "arm/coresight-core:starting", + NULL, coresight_dying_cpu); + if (ret) + cpu_pm_unregister_notifier(&coresight_cpu_pm_nb); + + return ret; }
static void coresight_pm_cleanup(void) { + cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); cpu_pm_unregister_notifier(&coresight_cpu_pm_nb); }
diff --git a/drivers/hwtracing/coresight/coresight-etm3x-core.c b/drivers/hwtracing/coresight/coresight-etm3x-core.c index 730d2584d625f11db87729352bcb80b9df1caa04..e05c752d57d9772e771b2df1bbf76e85623ea9ab 100644 --- a/drivers/hwtracing/coresight/coresight-etm3x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm3x-core.c @@ -701,35 +701,6 @@ static int etm_online_cpu(unsigned int cpu) return 0; }
-static int etm_starting_cpu(unsigned int cpu) -{ - if (!etmdrvdata[cpu]) - return 0; - - spin_lock(&etmdrvdata[cpu]->spinlock); - if (!etmdrvdata[cpu]->os_unlock) { - etm_os_unlock(etmdrvdata[cpu]); - etmdrvdata[cpu]->os_unlock = true; - } - - if (coresight_get_mode(etmdrvdata[cpu]->csdev)) - etm_enable_hw(etmdrvdata[cpu]); - spin_unlock(&etmdrvdata[cpu]->spinlock); - return 0; -} - -static int etm_dying_cpu(unsigned int cpu) -{ - if (!etmdrvdata[cpu]) - return 0; - - spin_lock(&etmdrvdata[cpu]->spinlock); - if (coresight_get_mode(etmdrvdata[cpu]->csdev)) - etm_disable_hw(etmdrvdata[cpu]); - spin_unlock(&etmdrvdata[cpu]->spinlock); - return 0; -} - static bool etm_arch_supported(u8 arch) { switch (arch) { @@ -797,13 +768,6 @@ static int __init etm_hp_setup(void) { int ret;
- ret = cpuhp_setup_state_nocalls_cpuslocked(CPUHP_AP_ARM_CORESIGHT_STARTING, - "arm/coresight:starting", - etm_starting_cpu, etm_dying_cpu); - - if (ret) - return ret; - ret = cpuhp_setup_state_nocalls_cpuslocked(CPUHP_AP_ONLINE_DYN, "arm/coresight:online", etm_online_cpu, NULL); @@ -814,15 +778,11 @@ static int __init etm_hp_setup(void) return 0; }
- /* failed dyn state - remove others */ - cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); - return ret; }
static void etm_hp_clear(void) { - cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); if (hp_online) { cpuhp_remove_state_nocalls(hp_online); hp_online = 0; diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 85f974f8635d0e0b548c7cb506c734a0c3d55f24..5bb4f8a40b78368cb61844cd800a6a42eeb48f87 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -1808,33 +1808,6 @@ static int etm4_online_cpu(unsigned int cpu) return 0; }
-static int etm4_starting_cpu(unsigned int cpu) -{ - if (!etmdrvdata[cpu]) - return 0; - - raw_spin_lock(&etmdrvdata[cpu]->spinlock); - if (!etmdrvdata[cpu]->os_unlock) - etm4_os_unlock(etmdrvdata[cpu]); - - if (coresight_get_mode(etmdrvdata[cpu]->csdev)) - etm4_enable_hw(etmdrvdata[cpu]); - raw_spin_unlock(&etmdrvdata[cpu]->spinlock); - return 0; -} - -static int etm4_dying_cpu(unsigned int cpu) -{ - if (!etmdrvdata[cpu]) - return 0; - - raw_spin_lock(&etmdrvdata[cpu]->spinlock); - if (coresight_get_mode(etmdrvdata[cpu]->csdev)) - etm4_disable_hw(etmdrvdata[cpu]); - raw_spin_unlock(&etmdrvdata[cpu]->spinlock); - return 0; -} - static bool etm4_need_save_restore_context(struct coresight_device *csdev) { if (pm_save_enable != PARAM_PM_SAVE_SELF_HOSTED) @@ -2107,13 +2080,6 @@ static int __init etm4_pm_setup(void) { int ret;
- ret = cpuhp_setup_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING, - "arm/coresight4:starting", - etm4_starting_cpu, etm4_dying_cpu); - - if (ret) - return ret; - ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, "arm/coresight4:online", etm4_online_cpu, NULL); @@ -2124,14 +2090,11 @@ static int __init etm4_pm_setup(void) return 0; }
- /* failed dyn state - remove others */ - cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); return ret; }
static void etm4_pm_clear(void) { - cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); if (hp_online) { cpuhp_remove_state_nocalls(hp_online); hp_online = 0;
-
Leo Yan