Hi
On 05/11/2026 12:19, Songwei.Chai wrote:
> Hi Suzuki,
>
> I wanted to follow up and see if you might have some time to take a look
> at this patch.
Apologies for the delay in responding on this one. But I have now
reviewed the series, please address the comments and repost them
based on 7.4-rcX
Cheers
Suzuki
>
> We’d really appreciate any feedback you may have, as it would help us
> determine the next steps and move this work forward.
> Thanks for your time.
>
> Best regards,
> Songwei
>
> On 7/30/2026 4:06 PM, Songwei Chai wrote:
>> From: Songwei Chai <songchai(a)qti.qualcomm.com>
>>
>> We propose creating a new qcom directory under drivers/hwtracing
>> to host this TGU driver, as well as additional Qualcomm-specific
>> hwtracing drivers that we plan to submit in the coming months.
>> This structure will help organize vendor-specific implementations
>> and facilitate future development and maintenance.
>>
>> Feedback from the community on this proposal is highly appreciated.
>>
>> - Why we are proposing this:
>>
>> TGU has the ability to monitor signal conditions and trigger debug-
>> related
>> actions, serving as a programmable hardware component that enhances
>> system
>> trace and debug capabilities. Placing it under drivers/hwtracing aligns
>> with its function as a trace generation utility.
>>
>> We previously attempted to push this driver to drivers/hwtracing/
>> coresight,
>> but did not receive support from the maintainers of the CoreSight
>> subsystem. The reason provided was: “This component is primarily a part
>> of the Qualcomm proprietary QPMDA subsystem, and is capable of operating
>> independently from the CoreSight hardware trace generation system.”
>>
>> Chat history : https://lore.kernel.org/all/
>> CAJ9a7ViKxHThyZfFFDV_FkNRimk4uo1NrMtQ-kcaj1qO4ZcGnA(a)mail.gmail.com/
>>
>> Given this, we have been considering whether it would be appropriate
>> to create a dedicated drivers/hwtracing/qcom directory for
>> Qualcomm-related hwtracing drivers. This would follow the precedent set
>> by Intel, which maintains its own directory at drivers/hwtracing/
>> intel_th.
>> We believe this structure would significantly facilitate
>> future submissions of related Qualcomm drivers.
>>
>> - Maintenance of drivers/hwtracing/qcom:
>>
>> Bjorn, who maintains linux-arm-msm, will be the maintainer of this
>> directory — we’ve discussed this with him and he’s aware that his task
>> list may grow accordingly. Additionally, Qualcomm engineers familiar with
>> the debug hardware — such as [Tingwei Zhang, Jinlong Mao, Songwei Chai],
>> will be available to review incoming patches and support ongoing
>> development.
>>
>> - Detail for TGU:
>>
>> This component can be utilized to sense a plurality of signals and
>> create a trigger into the CTI or generate interrupts to processors
>> once the input signal meets the conditions. We can treat the TGU’s
>> workflow as a flowsheet, it has some “steps” regions for customization.
>> In each step region, we can set the signals that we want with priority
>> in priority_group, set the conditions in each step via condition_decode,
>> and set the resultant action by condition_select. Meanwhile,
>> some TGUs (not all) also provide timer/counter functionality.
>> Based on the characteristics described above, we consider the TGU as a
>> helper in the CoreSight subsystem. Its master device is the TPDM, which
>> can transmit signals from other subsystems, and we reuse the existing
>> ports mechanism to link the TPDM to the connected TGU.
>>
>> Here is a detailed example to explain how to use the TGU:
>>
>> In this example, the TGU is configured to use 2 conditions, 2 steps, and
>> the timer. The goal is to look for one of two patterns which are
>> generated
>> from TPDM, giving priority to one, and then generate a trigger once the
>> timer reaches a certain value. In other words, two conditions are used
>> for the first step to look for the two patterns, where the one with the
>> highest priority is used in the first condition. Then, in the second
>> step,
>> the timer is enabled and set to be compared to the given value at each
>> clock cycle. These steps are better shown below.
>> |-----------------|
>> | |
>> | TPDM |
>> | |
>> |-----------------|
>> |
>> |
>> --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- ---
>> ------
>> | |
>> |
>> | |
>> |--------------------| |
>> | |---- ---> | | Go to next
>> steps | |
>> | | | |--- ---> | Enable
>> timer | |
>> | | v | |
>> | |
>> | | |-----------------| |
>> |--------------------| |
>> | | | | Yes | |
>> |
>> | | | inputs==0xB | ----->| |
>> <-------- |
>> | | | | | |
>> No | |
>> | No | |-----------------| |
>> v | |
>> | | | |
>> |-----------------| | |
>> | | | | |
>> | | |
>> | | | | | timer>=3
>> |-- |
>> | | v | |
>> | |
>> | | |-----------------| |
>> |-----------------| |
>> | | | | Yes | |
>> |
>> | |--- | inputs==0xA | ----->| |
>> Yes |
>> | | | |
>> |
>> | |-----------------|
>> v |
>> |
>> |-----------------| |
>> | |
>> | |
>> | | Trigger
>> | |
>> | |
>> | |
>> |
>> |-----------------| |
>> | TGU |
>> |
>> |--- --- --- --- --- --- --- --- --- --- --- --- --- --- |--- ---
>> -- |
>> |
>> v
>> |-----------------|
>> |The controllers |
>> |which will use |
>> |triggers further |
>> |-----------------|
>>
>> steps:
>> 1. Reset TGU /*it will disable tgu and reset dataset*/
>> - echo 1 > /sys/bus/amba/devices/<tgu-name>/reset_tgu
>>
>> 2. Set the pattern match for priority0 to 0xA = 0b1010 and for
>> priority 1 to 0xB = 0b1011.
>> - echo 0x11113232 > /sys/bus/amba/devices/<tgu-name>/
>> step0_priority0/reg0
>> - echo 0x11113233 > /sys/bus/amba/devices/<tgu-name>/
>> step0_priority1/reg0
>>
>> Note:
>> Bit distribution diagram for each priority register
>>
>> |-------------------------------------------------------------------|
>> | Bits | Field Nam |
>> Description |
>>
>> |-------------------------------------------------------------------|
>> | | | 00 = bypass for OR
>> output |
>> | 29:28 | SEL_BIT7_TYPE2 | 01 = bypass for AND
>> output |
>> | | | 10 = sense input '0' is
>> true|
>> | | | 11 = sense input '1' is
>> true|
>>
>> |-------------------------------------------------------------------|
>> | | | 00 = bypass for OR
>> output |
>> | 25:24 | SEL_BIT6_TYPE2 | 01 = bypass for AND
>> output |
>> | | | 10 = sense input '0' is
>> true|
>> | | | 11 = sense input '1' is
>> true|
>>
>> |-------------------------------------------------------------------|
>> | | | 00 = bypass for OR
>> output |
>> | 21:20 | SEL_BIT5_TYPE2 | 01 = bypass for AND
>> output |
>> | | | 10 = sense input '0' is
>> true|
>> | | | 11 = sense input '1' is
>> true|
>>
>> |-------------------------------------------------------------------|
>> | | | 00 = bypass for OR
>> output |
>> | 17:16 | SEL_BIT4_TYPE2 | 01 = bypass for AND
>> output |
>> | | | 10 = sense input '0' is
>> true|
>> | | | 11 = sense input '1' is
>> true|
>>
>> |-------------------------------------------------------------------|
>> | | | 00 = bypass for OR
>> output |
>> | 13:12 | SEL_BIT3_TYPE2 | 01 = bypass for AND
>> output |
>> | | | 10 = sense input '0' is
>> true|
>> | | | 11 = sense input '1' is
>> true|
>>
>> |-------------------------------------------------------------------|
>> | | | 00 = bypass for OR
>> output |
>> | 9:8 | SEL_BIT2_TYPE2 | 01 = bypass for AND
>> output |
>> | | | 10 = sense input '0' is
>> true|
>> | | | 11 = sense input '1' is
>> true|
>>
>> |-------------------------------------------------------------------|
>> | | | 00 = bypass for OR
>> output |
>> | 5:4 | SEL_BIT1_TYPE2 | 01 = bypass for AND
>> output |
>> | | | 10 = sense input '0' is
>> true|
>> | | | 11 = sense input '1' is
>> true|
>>
>> |-------------------------------------------------------------------|
>> | | | 00 = bypass for OR
>> output |
>> | 1:0 | SEL_BIT0_TYPE2 | 01 = bypass for AND
>> output |
>> | | | 10 = sense input '0' is
>> true|
>> | | | 11 = sense input '1' is
>> true|
>>
>> |-------------------------------------------------------------------|
>> These bits are used to identify the signals we want to sense, with
>> a maximum signal number of 140. For example, to sense the signal
>> 0xA (binary 1010), we set the value of bits 0 to 13 to 3232, which
>> represents 1010. The remaining bits are set to 1, as we want to use
>> AND gate to summarize all the signals we want to sense here. For
>> rising or falling edge detection of any input to the priority, set
>> the remaining bits to 0 to use an OR gate.
>>
>> 3. look for the pattern for priority_i i=0,1.
>> - echo 0x3 > /sys/bus/amba/devices/<tgu-name>/
>> step0_condition_decode/reg0
>> - echo 0x30 > /sys/bus/amba/devices/<tgu-name>/
>> step0_condition_decode/reg1
>>
>>
>> |-------------------------------------------------------------------------------|
>> | Bits | Field Nam |
>> Description |
>>
>> |-------------------------------------------------------------------------------|
>> | | |For each decoded
>> condition, this |
>> | 24 | NOT |inverts the output. If
>> the condition |
>> | | |decodes to true, and the
>> NOT field |
>> | | |is '1', then the output
>> is NOT true. |
>>
>> |-------------------------------------------------------------------------------|
>> | | |When '1' the output from
>> the associated|
>> | 21 | BC0_COMP_ACTIVE |comparator will be
>> actively included in|
>> | | |the decoding of this
>> particular |
>> | | |
>> condition. |
>>
>> |-------------------------------------------------------------------------------|
>> | | |When '1' the output from
>> the associated|
>> | | |comparator will need to
>> be 1 to affect |
>> | 20 | BC0_COMP_HIGH |the decoding of this
>> condition. |
>> | | |Conversely, a '0' here
>> requires a '0' |
>> | | |from the
>> comparator |
>>
>> |-------------------------------------------------------------------------------|
>> | | |When '1' the output from
>> the associated|
>> | 17 | |comparator will be
>> actively included in|
>> | | TC0_COMP_ACTIVE |the decoding of this
>> particular |
>> | | |
>> condition. |
>>
>> |-------------------------------------------------------------------------------|
>> | | |When '1' the output from
>> the associated|
>> | | |comparator will need to
>> be 1 to affect |
>> | 16 | TC0_COMP_HIGH |the decoding of this
>> particular |
>> | | |condition.Conversely, a
>> 0 here |
>> | | |requires a '0' from the
>> comparator |
>>
>> |-------------------------------------------------------------------------------|
>> | | |When '1' the output from
>> Priority_n |
>> | | |OR logic will be
>> actively |
>> | 4n+3 | Priority_n_OR_ACTIVE|included in the decoding
>> of |
>> | | (n=0,1,2,3) |this particular
>> condition. |
>> | | |
>> |
>>
>> |-------------------------------------------------------------------------------|
>> | | |When '1' the output from
>> Priority_n |
>> | | |will need to be '1' to
>> affect the |
>> | 4n+2 | Priority_n_OR_HIGH |decoding of this
>> particular |
>> | | (n=0,1,2,3) |condition. Conversely, a
>> '0' here |
>> | | |requires a '0' from
>> Priority_n OR logic|
>>
>> |-------------------------------------------------------------------------------|
>> | | |When '1' the output from
>> Priority_n |
>> | | |AND logic will be
>> actively |
>> | 4n+1 |Priority_n_AND_ACTIVE|included in the decoding
>> of this |
>> | | (n=0,1,2,3) |particular
>> condition. |
>> | | |
>> |
>>
>> |-------------------------------------------------------------------------------|
>> | | |When '1' the output from
>> Priority_n |
>> | | |AND logic will need to
>> be '1' to |
>> | 4n | Priority_n_AND_HIGH |affect the decoding of
>> this |
>> | | (n=0,1,2,3) |particular condition.
>> Conversely, |
>> | | |a '0' here requires a
>> '0' from |
>> | | |Priority_n AND
>> logic. |
>>
>> |-------------------------------------------------------------------------------|
>> Since we use `priority_0` and `priority_1` with an AND output in
>> step 2, we set `0x3`
>> and `0x30` here to activate them.
>>
>> 4. Set NEXT_STEP = 1 and TC0_ENABLE = 1 so that when the conditions
>> are met then the next step will be step 1 and the timer will
>> be enabled.
>> - echo 0x20008 > /sys/bus/amba/devices/<tgu-name>/
>> step0_condition_select/reg0
>> - echo 0x20008 > /sys/bus/amba/devices/<tgu-name>/
>> step0_condition_select/reg1
>>
>>
>> |-----------------------------------------------------------------------------|
>> | Bits | Field Nam |
>> Description |
>>
>> |-----------------------------------------------------------------------------|
>> | | |This field defines the
>> next step the |
>> | 18:17 | NEXT_STEP |TGU will 'goto' for the
>> associated |
>> | | |Condition and
>> Step. |
>>
>> |-----------------------------------------------------------------------------|
>> | | |For each possible output
>> trigger |
>> | 13 | TRIGGER |available, set a '1' if
>> you want |
>> | | |the trigger to go active
>> for the |
>> | | |associated condition and
>> Step. |
>>
>> |-----------------------------------------------------------------------------|
>> | | |This will cause BC0 to
>> increment if the|
>> | 9 | BC0_INC |associated Condition is
>> decoded for |
>> | | |this
>> step. |
>>
>> |-----------------------------------------------------------------------------|
>> | | |This will cause BC0 to
>> decrement if the|
>> | 8 | BC0_DEC |associated Condition is
>> decoded for |
>> | | |this
>> step. |
>>
>> |-----------------------------------------------------------------------------|
>> | | |This will clear BC0 count
>> value to 0 if|
>> | 7 | BC0_CLEAR |the associated Condition
>> is decoded |
>> | | |for this
>> step. |
>>
>> |-----------------------------------------------------------------------------|
>> | | |This will cause TC0 to
>> increment until |
>> | 3 | TC0_ENABLE |paused or cleared if the
>> associated |
>> | | |Condition is decoded for
>> this step. |
>>
>> |-----------------------------------------------------------------------------|
>> | | |This will cause TC0 to
>> pause until |
>> | 2 | TC0_PAUSE |enabled if the associated
>> Condition |
>> | | |is decoded for this
>> step. |
>>
>> |-----------------------------------------------------------------------------|
>> | | |This will clear TC0 count
>> value to 0 |
>> | 1 | TC0_CLEAR |if the associated
>> Condition is |
>> | | |decoded for this
>> step. |
>>
>> |-----------------------------------------------------------------------------|
>> | | |This will set the done
>> signal to the |
>> | 0 | DONE |TGU FSM if the associated
>> Condition |
>> | | |is decoded for this
>> step. |
>>
>> |-----------------------------------------------------------------------------|
>> Based on the distribution diagram, we set `0x20008` for
>> `priority0` and `priority1` to
>> achieve "jump to step 1 and enable TC0" once the signal is sensed.
>>
>> 5. activate the timer comparison for this step.
>> - echo 0x30000 > /sys/bus/amba/devices/<tgu-name>/
>> step1_condition_decode/reg0
>>
>>
>> |-------------------------------------------------------------------------------|
>> | | |When '1' the output from
>> the associated|
>> | 17 | |comparator will be
>> actively included in|
>> | | TC0_COMP_ACTIVE |the decoding of this
>> particular |
>> | | |
>> condition. |
>>
>> |-------------------------------------------------------------------------------|
>> | | |When '1' the output from
>> the associated|
>> | | |comparator will need to
>> be 1 to affect |
>> | 16 | TC0_COMP_HIGH |the decoding of this
>> particular |
>> | | |condition.Conversely, a
>> 0 here |
>> | | |requires a '0' from the
>> comparator |
>>
>> |-------------------------------------------------------------------------------|
>> Accroding to the decode distribution diagram , we give 0x30000
>> here to set 16th&17th bit
>> to enable timer comparison.
>>
>> 6. Set the NEXT_STEP = 0 and TC0_PAUSE = 1 and TC0_CLEAR = 1 once
>> the timer
>> has reached the given value.
>> - echo 0x6 > /sys/bus/amba/devices/<tgu-name>/
>> step1_condition_select/reg0
>>
>> 7. Enable Trigger 0 for TGU when the condition 0 is met in step1,
>> i.e. when the timer reaches 3.
>> - echo 0x2000 > /sys/bus/amba/devices/<tgu-name>/
>> step1_condition_select/default
>>
>> Note:
>> 1. 'default' register allows for establishing the resultant
>> action for
>> the default condition
>>
>> 2. Trigger:For each possible output trigger available from
>> the Design document, there are three triggers: interrupts, CTI,
>> and Cross-TGU mapping.All three triggers can occur, but
>> the choice of which trigger to use depends on the user's
>> needs.
>>
>> 8. Compare the timer to 3 in step 1.
>> - echo 0x3 > /sys/bus/amba/devices/<tgu-name>/step1_timer/reg0
>>
>> 9. enale tgu
>> - echo 1 > /sys/bus/amba/devices/<tgu-name>/enable_tgu
>> ---
>> Link to V15: https://lore.kernel.org/all/20260701055529.445819-1-
>> songwei.chai(a)oss.qualcomm.com/
>>
>> Changes in V16:
>> - Fixed enable/disable race and PM refcount handling.
>> - Fixed sysfs exposure before driver initialization.
>> - Fixed priority allocation and access races.
>> - Fixed uninitialized return value in condition decode.
>> - Fixed sysfs attribute naming collision issues.
>> - Added ABI documentation for the default node.
>> ---
>> Link to V14: https://lore.kernel.org/all/20260417073336.2712426-1-
>> songwei.chai(a)oss.qualcomm.com/
>>
>> Changes in V15:
>> - Add a maintainer for "drivers/hwtracing/qcom" in MAINTAINERS
>> ---
>> Link to V13: https://lore.kernel.org/all/20260402092838.341295-1-
>> songwei.chai(a)oss.qualcomm.com/
>>
>> Changes in V14:
>> - Fix some typos and formatting.
>> ---
>> Link to V12: https://lore.kernel.org/all/20260317032639.2393221-1-
>> songwei.chai(a)oss.qualcomm.com/
>>
>> Changes in V13:
>> - add ":" after "KernelVersion"
>> - add an enablement check in the enable function to avoid increasing
>> the counter each time
>> ---
>> Link to V11: https://lore.kernel.org/all/ee1ca8e6-8e5f-47d8-8a24-
>> f904ee2fc6d0(a)oss.qualcomm.com/
>>
>> Changes in V12:
>> - Remove the in-ports property from the bindings, as this device is
>> decoupled from CoreSight.
>> - Update kernel version and date.
>> ---
>> Link to V10: https://lore.kernel.org/
>> all/20c5406d-3e9f-4fdb-84ba-4cbe629c79b5(a)oss.qualcomm.com/
>>
>> Changes in V11:
>> - Change the names of members in drvdata: max_xxx -> num_xxx, enable -
>> > enabled
>> - Use "FIELD_GET" to replace "BMVAL"
>> - Use devm_kcalloc to replace devm_kzalloc once create members of
>> value_table
>> - Keep a consistent \n above return
>> - Keep reverse-Christmas-tree style
>> - Add checks so that the enable and reset nodes only accept 0 or 1
>> ---
>> Link to V9: https://lore.kernel.org/all/20251219065902.2296896-1-
>> songwei.chai(a)oss.qualcomm.com/
>>
>> Changes in V10:
>> - Modified code formatting based on Jie's feedback to improve
>> readability.
>> - Applied inverse Christmas tree order to the variables.
>> ---
>> Link to V8: https://lore.kernel.org/all/20251203090055.2432719-1-
>> songwei.chai(a)oss.qualcomm.com/
>>
>> Changes in V9:
>> - Decoupled the tgu driver from coresight header file and registered
>> it as an amba device.
>> - Retained Rob's reviewed-by tag on patch1/7 since the file remains
>> unchanged.
>> - Updated the sysfs node path in the Documentation directory.
>> ---
>> Link to V7: https://lore.kernel.org/all/20251104064043.88972-1-
>> songwei.chai(a)oss.qualcomm.com/
>>
>> Changes in V8:
>> - Add "select" section in bindings.
>> - Update publish date in "sysfs-bus-coresight-devices-tgu".
>> ---
>> Link to V6: https://lore.kernel.org/all/20250709104114.22240-1-
>> songchai(a)qti.qualcomm.com/
>>
>> Changes in V7:
>> - Move the TGU code location from 'drivers/hwtracing/coresight/' to
>> 'drivers/hwtracing/qcom/'.
>> - Rename the spinlock used in the code from 'spinlock' to 'lock'.
>> - Perform the 'calculate_array_location' separately, instead of doing
>> it within the function.
>> - Update the sender email address.
>> ---
>> Link to V5: https://lore.kernel.org/all/20250529081949.26493-1-
>> quic_songchai(a)quicinc.com/
>>
>> Changes in V6:
>> - Replace spinlock with guard(spinlock) in tgu_enable.
>> - Remove redundant blank line.
>> - Update publish date and contact member's name in "sysfs-bus-
>> coresight-devices-tgu".
>> ---
>> Link to V4: https://patchwork.kernel.org/project/linux-arm-msm/
>> cover/20250423101054.954066-1-quic_songchai(a)quicinc.com/
>>
>> Changes in V5:
>> - Update publish date and kernel_version in "sysfs-bus-coresight-
>> devices-tgu"
>> ---
>> Link to V3: https://lore.kernel.org/all/20250227092640.2666894-1-
>> quic_songchai(a)quicinc.com/
>>
>> Changes in V4:
>> - Add changlog in coverletter.
>> - Correct 'year' in Copyright in patch1.
>> - Correct port mechansim description in patch1.
>> - Remove 'tgu-steps','tgu-regs','tgu-conditions','tgu-timer-counters'
>> from dt-binding
>> and set them through reading DEVID register as per Mike's suggestion.
>> - Modify tgu_disable func to make it have single return point in
>> patch2 as per
>> Mike's suggestion.
>> - Use sysfs_emit in enable_tgu_show func in ptach2.
>> - Remove redundant judgement in enable_tgu_store in patch2.
>> - Correct typo in description in patch3.
>> - Set default ret as SYSFS_GROUP_INVISIBLE, and returnret at end in
>> pacth3 as
>> per Mike's suggestion.
>> - Remove tgu_dataset_ro definition in patch3
>> - Use #define constants with explanations of what they are rather than
>> arbitrary magic numbers in patch3 and patch4.
>> - Check -EINVAL before using 'calculate_array_location()' in array in
>> patch4.
>> - Add 'default' in 'tgu_dataset_show''s switch part in patch4.
>> - Document the value needed to initiate the reset in pacth7.
>> - Check "value" in 'reset_tgu_store' and bail out with an error code
>> if 0 in patch7.
>> - Remove dev_dbg in 'reset_tgu_store' in patch7.
>> ---
>> Link to V2: https://lore.kernel.org/all/20241010073917.16023-1-
>> quic_songchai(a)quicinc.com/
>>
>> Changes in V3:
>> - Correct typo and format in dt-binding in patch1
>> - Rebase to the latest kernel version
>> ---
>> Link to V1: https://lore.kernel.org/all/20240830092311.14400-1-
>> quic_songchai(a)quicinc.com/
>>
>> Changes in V2:
>> - Use real name instead of login name,
>> - Correct typo and format in dt-binding and code.
>> - Bring order in tgu_prob(declarations with and without assignments)
>> as per
>> Krzysztof's suggestion.
>> - Add module device table in patch2.
>> - Set const for tgu_common_grp and tgu_ids in patch2.
>> - Initialize 'data' in tgu_ids to fix the warning in pacth2.
>> ---
>>
>> Songwei Chai (7):
>> dt-bindings: arm: Add support for Qualcomm TGU trace
>> qcom-tgu: Add TGU driver
>> qcom-tgu: Add signal priority support
>> qcom-tgu: Add TGU decode support
>> qcom-tgu: Add support to configure next action
>> qcom-tgu: Add timer/counter functionality for TGU
>> qcom-tgu: Add reset node to initialize
>>
>> .../ABI/testing/sysfs-bus-amba-devices-tgu | 60 ++
>> .../devicetree/bindings/arm/qcom,tgu.yaml | 71 ++
>> MAINTAINERS | 11 +
>> drivers/Makefile | 1 +
>> drivers/hwtracing/Kconfig | 2 +
>> drivers/hwtracing/qcom/Kconfig | 20 +
>> drivers/hwtracing/qcom/Makefile | 3 +
>> drivers/hwtracing/qcom/tgu.c | 782 ++++++++++++++++++
>> drivers/hwtracing/qcom/tgu.h | 318 +++++++
>> 9 files changed, 1268 insertions(+)
>> create mode 100644 Documentation/ABI/testing/sysfs-bus-amba-devices-tgu
>> create mode 100644 Documentation/devicetree/bindings/arm/qcom,tgu.yaml
>> create mode 100644 drivers/hwtracing/qcom/Kconfig
>> create mode 100644 drivers/hwtracing/qcom/Makefile
>> create mode 100644 drivers/hwtracing/qcom/tgu.c
>> create mode 100644 drivers/hwtracing/qcom/tgu.h
>>
On Wed, Nov 04, 2026 at 12:59:13PM +0000, Will Deacon wrote:
[...]
> > Sorry for the late reply, and sorry that the commit message in
> > 18049c8cff9c ("perf/aux: Allocate non-contiguous AUX pages by default")
> > was imprecise about "exacerbating memory fragmentation". You and Dev are
> > right that grouping same-lifetime pages into higher-order allocations
> > does not itself worsen fragmentation, and the TLB reduction numbers for
> > TRBE and SPE with large vmap() mappings make total sense given TRBE's
> > tiny TLB.
> >
> > However, a straight revert will reintroduce a severe production issue we
> > hit on Android devices, so I'd like to share what actually happened and
> > suggest a way to get large mappings for SPE/TRBE without bringing back
> > the regression.
>
> [...]
>
> Thanks, Yabin. This just hardens my stance [1] that the allocation decision
> should be driven by userspace and not second-guessed by the kernel. So the
> revert is _not_ the right approach, in my view.
If a userspace interface is needed, what remaining reclaim or allocation
policy should userspace control? I am not clear how userspace control
would fix the PSI reclaim stalls described in Yabin's reply.
Alternatively, we could optimize the reported reclaim cost in the kernel
by clearing __GFP_RECLAIM for optional higher-order attempts, while
preserving the existing policy at order 0 and for
PERF_PMU_CAP_AUX_NO_SG PMUs.
If testing confirms this works, combining this mitigation with a revert
of 18049c8cff9c would be a reasonable way to restore higher-order
allocations for SPE/TRBE without adding a userspace interface.
Thanks,
Leo
On Tue, Nov 03, 2026 at 12:46:24PM -0800, Yabin Cui wrote:
...
> In field telemetry and local ftrace on Pixel devices (prior to
> 18049c8cff9c, so __GFP_NORETRY was already in place), we observed heavy
> rb_alloc_aux() -> __alloc_pages_direct_compact() stalls alongside
> kcompactd0 pinned on big cores scanning order-9, and this accounted for
> >20% of all perceptible app kills (via lmkd) across dogfood devices
> until we stopped high-order AUX allocations.
Thanks for the detailed explanation. My understanding is that repeated
reclaim/compaction during AUX allocation raises memory PSI and can lead
to lmkd killing applications. The problem to address is therefore the
cost of obtaining higher-order pages under memory fragmentation.
> Initially we worked around this in userspace (simpleperf) by setting
> perf_event_attr::aux_watermark = 4096 so get_order(watermark) == 0. That
> was fine for CoreSight ETR (which uses its own separate scatter-gather
> buffer and only copies into AUX on flush/stop), but when enabling TRBE
> (and SPE) it caused hardware maintenance interrupts every 4 KiB instead
> of every 2 MiB. Also, in overwrite/snapshot mode rb_alloc_aux() ignores
> aux_watermark and uses ilog2(nr_pages). That is why I sent 18049c8cff9c.
Agreed. Reducing watermark to 4096 also increases maintenance interrupt
frequency for SPE/TRBE. We should refine buffer allocation rather than
using watermark as a workaround.
> ### Possible solutions
>
> For PMUs without PERF_PMU_CAP_AUX_NO_SG (such as SPE, TRBE, and modern
> Intel PT), higher-order AUX pages are an opportunistic optimization
> rather than a functional requirement -- very similar to THP allocations
> on fault (GFP_TRANSHUGE_LIGHT) or opportunistic high-order slabs in SLUB
> (allocate_slab()).
>
> Instead of reverting to GFP_KERNEL reclaim/compaction for all high-order
> attempts, could we do the following?
>
> 1. Make opportunistic high-order AUX allocations no-reclaim:
> When !(event->pmu->capabilities & PERF_PMU_CAP_AUX_NO_SG) and
> order > 0, clear __GFP_RECLAIM (and add __GFP_NOMEMALLOC, matching
> SLUB's allocate_slab() and GFP_TRANSHUGE_LIGHT):
>
> gfp_t gfp = PERF_AUX_GFP;
>
> if (order && !has_no_sg)
> gfp = (PERF_AUX_GFP | __GFP_NOMEMALLOC) & ~__GFP_RECLAIM;
>
> When order falls back to 0 (or when PERF_PMU_CAP_AUX_NO_SG is set),
> keep PERF_AUX_GFP so single-page allocations still reclaim normally
> under memory pressure.
>
> That way, when free order-9 / order-4 blocks exist, SPE and TRBE get
> them immediately for PMD / contiguous-PTE vmap() mappings (and pages
> stay grouped together); when memory is fragmented on a long-running
> device, high-order probes fail fast on the freelist without waking
> kcompactd/kswapd or triggering direct compaction and reclaim.
This looks reasonable for me: try to obtain larger blocks cheaply, then
then retain the PERF_AUX_GFP policy for order-0 allocations. I also
agree with preserving the allocation policy for PERF_PMU_CAP_AUX_NO_SG
case.
Could you compare the memory PSI stall deltas before and after applying
this option? I think measurements from your production devices would be
convinced evidence for benefits.
> 2. Clamp max_order in rb_alloc_aux() after fallback:
> If rb_alloc_aux_page() requested `order` and had to fall back to a
> smaller order, update `max_order = page_private(page)` in
> rb_alloc_aux() so subsequent iterations for the rest of the buffer do
> not keep re-probing higher orders that just failed.
This also can avoid reclaim repeatedly, but the side effect is subsequent
iterations would no longer try high order, which have no chance to
benefit from large mapping.
I would prefer to evaluate option 1 first.
> (Optionally, if server/benchmark workloads ever want synchronous
> compaction to form huge AUX mappings even on fragmented hosts, that could
> be gated behind an opt-in attr/sysfs knob similar to THP defrag modes,
> with light/no-reclaim as the default.)
I will follow up with Will separately about the userspace interface.
> Would this approach work for your SPE/TRBE use cases? Happy to help test
> on our side.
If option 1 works, we can confidently revert 18049c8cff9c, as it would
restore higher-order allocations for TRBE/SPE without introducing the
regression you mentioned.
Thanks,
Leo
On 26/10/2026 05:48, Amir Ayupov wrote:
> Drivers outside the PMU driver may need to know whether the local CPU's
> PMU implements a common event before relying on it, for example a trace
> unit driver checking that the PMU counts its external outputs as
> TRCEXTOUTn.
>
> Add pmuv3_event_implemented() to the arm64 asm/arm_pmuv3.h, next to
> read_pmceid0() and read_pmceid1(). It reads PMCEID0/1 on the calling CPU:
> events 0x0000-0x003f are reported in bits [31:0], and the common events
> from ARMV8_PMUV3_EXT_COMMON_EVENT_BASE in bits [63:32] from PMUv3p1. Move
> ARMV8_PMUV3_EXT_COMMON_EVENT_BASE next to the event numbers in
> linux/perf/arm_pmuv3.h so it is available without CONFIG_ARM_PMU.
>
> Suggested-by: James Clark <james.clark(a)linaro.org>
> Assisted-by: Devmate:Opus-5.5
> Signed-off-by: Amir Ayupov <aaupov(a)fb.com>
> ---
> arch/arm64/include/asm/arm_pmuv3.h | 19 +++++++++++++++++++
> include/linux/perf/arm_pmu.h | 1 -
> include/linux/perf/arm_pmuv3.h | 3 +++
> 3 files changed, 22 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/arm_pmuv3.h b/arch/arm64/include/asm/arm_pmuv3.h
> index 8a777dec8d88a..acd62c4f16743 100644
> --- a/arch/arm64/include/asm/arm_pmuv3.h
> +++ b/arch/arm64/include/asm/arm_pmuv3.h
> @@ -173,6 +173,25 @@ static inline bool pmuv3_implemented(int pmuver)
> pmuver == ID_AA64DFR0_EL1_PMUVer_NI);
> }
>
> +/*
> + * Whether this CPU's PMU implements a common event, as reported by
> + * PMCEID{0,1}: events 0x0000-0x003f in bits [31:0], and from PMUv3p1 events
> + * 0x4000-0x403f in bits [63:32], which read as zero before PMUv3p1.
> + * Must run on the CPU whose PMU is queried.
> + */
> +static inline bool pmuv3_event_implemented(u16 event)
> +{
> + bool ext = event >= ARMV8_PMUV3_EXT_COMMON_EVENT_BASE;
> + u16 id = ext ? event - ARMV8_PMUV3_EXT_COMMON_EVENT_BASE : event;
> + u64 pmceid;
> +
> + if (id >= 64 || !pmuv3_implemented(read_pmuver()))
> + return false;
> +
> + pmceid = id < 32 ? :() : read_pmceid1();
> + return pmceid & BIT_ULL(id % 32 + (ext ? 32 : 0));
> +}
> +
> static inline bool is_pmuv3p4(int pmuver)
> {
> return pmuver >= ID_AA64DFR0_EL1_PMUVer_V3P4;
> diff --git a/include/linux/perf/arm_pmu.h b/include/linux/perf/arm_pmu.h
> index 02d2c7f45b527..792283d29d7b5 100644
> --- a/include/linux/perf/arm_pmu.h
> +++ b/include/linux/perf/arm_pmu.h
> @@ -124,7 +124,6 @@ struct arm_pmu {
> u64 reg_brbidr;
> #define ARMV8_PMUV3_MAX_COMMON_EVENTS 0x40
> DECLARE_BITMAP(pmceid_bitmap, ARMV8_PMUV3_MAX_COMMON_EVENTS);
> -#define ARMV8_PMUV3_EXT_COMMON_EVENT_BASE 0x4000
> DECLARE_BITMAP(pmceid_ext_bitmap, ARMV8_PMUV3_MAX_COMMON_EVENTS);
Could we not avoid this change ? Or why not move both of the definitions ?
Also, I would need an ACK from the arm_pmu maintainers to accept this
change via coresight tree.
Cheers
Suzuki
>
> /* Only to be used by ACPI probing code */
> diff --git a/include/linux/perf/arm_pmuv3.h b/include/linux/perf/arm_pmuv3.h
> index d698efba28a27..c21f185a283d6 100644
> --- a/include/linux/perf/arm_pmuv3.h
> +++ b/include/linux/perf/arm_pmuv3.h
> @@ -78,6 +78,9 @@
> #define ARMV8_PMUV3_PERFCTR_STALL_SLOT_FRONTEND 0x003E
> #define ARMV8_PMUV3_PERFCTR_STALL_SLOT 0x003F
>
> +/* Base of the common events reported in PMCEID{0,1}[63:32] from PMUv3p1 */
> +#define ARMV8_PMUV3_EXT_COMMON_EVENT_BASE 0x4000
> +
> /* Statistical profiling extension microarchitectural events */
> #define ARMV8_SPE_PERFCTR_SAMPLE_POP 0x4000
> #define ARMV8_SPE_PERFCTR_SAMPLE_FEED 0x4001
On 26/10/2026 05:48, Amir Ayupov wrote:
> A feature loads on a device when the two share any match flag. That works
> for device classes, where a feature wants any one of them, but not for
> capabilities a device may or may not have: a feature that sets a class
> and a capability would still load on a device of that class without the
> capability.
>
> Split the match flags into CS_CFG_MATCH_CLASS_MASK for device classes and
> CS_CFG_MATCH_CAP_MASK for capabilities. A feature now loads on a device
> that shares one of its classes and has all of its capabilities. Check
> this once in cscfg_load_feat_csdev(), which both the feature and the
> device registration paths call. No capability flags are defined yet, so
> matching is unchanged.
>
> Assisted-by: Devmate:Opus-5.5
> Signed-off-by: Amir Ayupov <aaupov(a)fb.com>
> ---
> .../hwtracing/coresight/coresight-config.h | 8 ++++
> .../hwtracing/coresight/coresight-syscfg.c | 37 +++++++++++++------
> 2 files changed, 34 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/hwtracing/coresight/coresight-config.h b/drivers/hwtracing/coresight/coresight-config.h
> index ae6295e308131..f04e73b91d9a6 100644
> --- a/drivers/hwtracing/coresight/coresight-config.h
> +++ b/drivers/hwtracing/coresight/coresight-config.h
> @@ -7,6 +7,7 @@
> #ifndef _CORESIGHT_CORESIGHT_CONFIG_H
> #define _CORESIGHT_CORESIGHT_CONFIG_H
>
> +#include <linux/bits.h>
> #include <linux/cleanup.h>
> #include <linux/coresight.h>
> #include <linux/types.h>
> @@ -30,6 +31,13 @@
> */
> #define CS_CFG_MATCH_CLASS_SRC_ALL 0x0001 /* match any source */
> #define CS_CFG_MATCH_CLASS_SRC_ETM4 0x0002 /* match any ETMv4 device */
> +#define CS_CFG_MATCH_CLASS_MASK GENMASK_U32(7, 0)
> +
> +/*
> + * flags defining device capabilities a feature can require. A feature loads on
> + * a device that matches one of its classes and has all of its capabilities.
> + */
> +#define CS_CFG_MATCH_CAP_MASK GENMASK_U32(15, 8)
>
> /* flags defining device instance matching - used in config match desc data. */
> #define CS_CFG_MATCH_INST_ANY 0x80000000 /* any instance of a class */
> diff --git a/drivers/hwtracing/coresight/coresight-syscfg.c b/drivers/hwtracing/coresight/coresight-syscfg.c
> index a816c22c87bc8..3ee104e94a4cd 100644
> --- a/drivers/hwtracing/coresight/coresight-syscfg.c
> +++ b/drivers/hwtracing/coresight/coresight-syscfg.c
> @@ -171,8 +171,22 @@ cscfg_alloc_csdev_feat(struct coresight_device *csdev, struct cscfg_feature_desc
> return feat_csdev;
> }
>
> -/* load one feature into one coresight device */
> +static bool cscfg_match_feat_csdev(u32 feat_flags, u32 csdev_flags)
> +{
> + /* The device must be in one of the feature's classes */
> + if (!(feat_flags & csdev_flags & CS_CFG_MATCH_CLASS_MASK))
> + return false;
> +
> + /* The device must have every capability the feature requires */
> + if (feat_flags & ~csdev_flags & CS_CFG_MATCH_CAP_MASK)
> + return false;
Please could you address the following comment ?
https://lore.kernel.org/linux-arm-kernel/82adee03-a169-4106-99f2-7624a49260…
Cheers
Suzuki
> +
> + return true;
> +}
> +
> +/* Load the feature into the device if it matches, otherwise skip it. */
> static int cscfg_load_feat_csdev(struct coresight_device *csdev,
> + u32 match_flags,
> struct cscfg_feature_desc *feat_desc,
> struct cscfg_csdev_feat_ops *ops)
> {
> @@ -183,6 +197,9 @@ static int cscfg_load_feat_csdev(struct coresight_device *csdev,
> if (!ops->load_feat)
> return -EINVAL;
>
> + if (!cscfg_match_feat_csdev(feat_desc->match_flags, match_flags))
> + return 0;
> +
> feat_csdev = cscfg_alloc_csdev_feat(csdev, feat_desc);
> if (!feat_csdev)
> return -ENOMEM;
> @@ -212,11 +229,11 @@ static int cscfg_add_feat_to_csdevs(struct cscfg_feature_desc *feat_desc)
> int err;
>
> list_for_each_entry(csdev_item, &cscfg_mgr->csdev_desc_list, item) {
> - if (csdev_item->match_flags & feat_desc->match_flags) {
> - err = cscfg_load_feat_csdev(csdev_item->csdev, feat_desc, &csdev_item->ops);
> - if (err)
> - return err;
> - }
> + err = cscfg_load_feat_csdev(csdev_item->csdev,
> + csdev_item->match_flags, feat_desc,
> + &csdev_item->ops);
> + if (err)
> + return err;
> }
> return 0;
> }
> @@ -739,11 +756,9 @@ static int cscfg_add_feats_csdev(struct coresight_device *csdev,
> return -EINVAL;
>
> list_for_each_entry(feat_desc, &cscfg_mgr->feat_desc_list, item) {
> - if (feat_desc->match_flags & match_flags) {
> - err = cscfg_load_feat_csdev(csdev, feat_desc, ops);
> - if (err)
> - break;
> - }
> + err = cscfg_load_feat_csdev(csdev, match_flags, feat_desc, ops);
> + if (err)
> + break;
> }
> return err;
> }
On 26/10/2026 05:48, Amir Ayupov wrote:
> Some CPUs count the trace unit's external outputs as the PMU common
> events TRCEXTOUT0-3 (0x4010-0x4013), which lets a configuration drive a
> PMU event from trace unit events. Other CPUs, Neoverse N1 for example,
> have no such events, and a feature relying on them would silently do
> nothing there.
>
> Detect at probe, on the trace unit's CPU, whether the PMU counts every
> external output: the trace unit has an external output for each of its
> events (caps.nr_event), and pmuv3_event_implemented() reports whether
> the PMU implements TRCEXTOUTn. Only check ETE: per its specification, an
> implemented TRCEXTOUTn event counts external output n, while whether that
> output is also an external input is IMPLEMENTATION DEFINED.
>
> Add a CS_CFG_MATCH_CAP_TRCEXTOUT capability match flag, set it on those
> trace units, and show it in the feature's configfs 'matches' attribute.
> With capabilities matched separately from the device class, a feature
> that requires it only loads on those trace units.
>
> Assisted-by: Devmate:Opus-5.5
> Signed-off-by: Amir Ayupov <aaupov(a)fb.com>
> ---
> .../hwtracing/coresight/coresight-config.h | 2 ++
> .../hwtracing/coresight/coresight-etm4x-cfg.c | 6 ++++-
> .../coresight/coresight-etm4x-core.c | 26 +++++++++++++++++++
> drivers/hwtracing/coresight/coresight-etm4x.h | 3 +++
> .../coresight/coresight-syscfg-configfs.c | 3 +++
> 5 files changed, 39 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/hwtracing/coresight/coresight-config.h b/drivers/hwtracing/coresight/coresight-config.h
> index 220bc54f8e5e1..4cf53230854d0 100644
> --- a/drivers/hwtracing/coresight/coresight-config.h
> +++ b/drivers/hwtracing/coresight/coresight-config.h
> @@ -38,6 +38,8 @@
> * a device that matches one of its classes and has all of its capabilities.
> */
> #define CS_CFG_MATCH_CAP_MASK GENMASK_U32(15, 8)
> +/* the PMU counts the trace unit's external outputs as TRCEXTOUTn */
> +#define CS_CFG_MATCH_CAP_TRCEXTOUT BIT(8)
>
Please could we rename this to :
CS_CFG_MATCH_CAP_ETE_TRCEXTOUT
as this is only applicable for ETEs ?
> /*
> * Limit number of presets in a configuration
> diff --git a/drivers/hwtracing/coresight/coresight-etm4x-cfg.c b/drivers/hwtracing/coresight/coresight-etm4x-cfg.c
> index 52cb036bb04b9..9e4dc5950c13d 100644
> --- a/drivers/hwtracing/coresight/coresight-etm4x-cfg.c
> +++ b/drivers/hwtracing/coresight/coresight-etm4x-cfg.c
> @@ -176,9 +176,13 @@ static int etm4_cfg_load_feature(struct coresight_device *csdev,
>
> int etm4_cscfg_register(struct coresight_device *csdev)
> {
> + struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
> struct cscfg_csdev_feat_ops ops;
> + u32 match_flags = CS_CFG_ETM4_MATCH_FLAGS;
>
> ops.load_feat = &etm4_cfg_load_feature;
> + if (drvdata->caps.trcextout)
> + match_flags |= CS_CFG_MATCH_CAP_TRCEXTOUT;
>
> - return cscfg_register_csdev(csdev, CS_CFG_ETM4_MATCH_FLAGS, &ops);
> + return cscfg_register_csdev(csdev, match_flags, &ops);
> }
> diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
> index af017bd8a5477..cb5fbb0279b88 100644
> --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
> +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
> @@ -30,6 +30,7 @@
> #include <linux/uaccess.h>
> #include <linux/perf_event.h>
> #include <linux/perf/arm_pmu.h>
> +#include <linux/perf/arm_pmuv3.h>
> #include <linux/platform_device.h>
> #include <linux/pm_runtime.h>
> #include <linux/property.h>
> @@ -1399,6 +1400,29 @@ static void etm4_fixup_wrong_ccitmin(struct etmv4_caps *caps)
> }
> }
>
> +/*
> + * Whether the PMU counts each external output of this trace unit as
> + * TRCEXTOUTn. The trace unit has an external output for each of its events.
> + * Only ETE is checked: per its specification, an implemented TRCEXTOUTn event
> + * counts external output n.
> + *
> + * Must run on the CPU that owns the trace unit.
> + */
> +static bool etm4_pmu_counts_extout(struct etmv4_drvdata *drvdata)
> +{
> + int n;
> +
> + if (!etm4x_is_ete(drvdata) || !drvdata->caps.nr_event)
> + return false;
> +
> + for (n = 0; n < drvdata->caps.nr_event; n++) {
> + if (!pmuv3_event_implemented(ARMV8_PMUV3_PERFCTR_TRCEXTOUT0 + n))
> + return false;
> + }
> +
> + return true;
> +}
> +
> static void etm4_init_arch_data(void *info)
> {
> u32 etmidr0;
> @@ -1562,6 +1586,8 @@ static void etm4_init_arch_data(void *info)
> /* NUMCNTR, bits[30:28] number of counters available for tracing */
> caps->nr_cntr = FIELD_GET(TRCIDR5_NUMCNTR_MASK, etmidr5);
>
> + caps->trcextout = etm4_pmu_counts_extout(drvdata);
> +
> coresight_clear_self_claim_tag_unlocked(csa);
> etm4_cs_lock(drvdata, csa);
> cpu_detect_trace_filtering(drvdata);
> diff --git a/drivers/hwtracing/coresight/coresight-etm4x.h b/drivers/hwtracing/coresight/coresight-etm4x.h
> index 5417280cdb274..33994ad4b2641 100644
> --- a/drivers/hwtracing/coresight/coresight-etm4x.h
> +++ b/drivers/hwtracing/coresight/coresight-etm4x.h
> @@ -868,6 +868,8 @@ enum etm_impdef_type {
> * @lpoverride: If the implementation can support low-power state over.
> * @skip_power_up: Indicates if an implementation can skip powering up
> * the trace unit.
> + * @trcextout: The trace unit is an ETE whose PMU counts its external
> + * outputs as TRCEXTOUTn.
We don't seem to be using this after the CSFG flag initialisation, in
which case could we directly call etm4_pmu_counts_extout() from
etm4_cscfg_register() ?
Suzuki
> */
> struct etmv4_caps {
> u8 nr_pe;
> @@ -907,6 +909,7 @@ struct etmv4_caps {
> bool atbtrig : 1;
> bool lpoverride : 1;
> bool skip_power_up : 1;
> + bool trcextout : 1;
> };
>
> /**
> diff --git a/drivers/hwtracing/coresight/coresight-syscfg-configfs.c b/drivers/hwtracing/coresight/coresight-syscfg-configfs.c
> index 2b40e556be872..7b3800138f625 100644
> --- a/drivers/hwtracing/coresight/coresight-syscfg-configfs.c
> +++ b/drivers/hwtracing/coresight/coresight-syscfg-configfs.c
> @@ -250,6 +250,9 @@ static ssize_t cscfg_feat_matches_show(struct config_item *item, char *page)
> if (match_flags & CS_CFG_MATCH_CLASS_SRC_ETM4)
> used += scnprintf(page + used, PAGE_SIZE - used, "SRC_ETMV4 ");
>
> + if (match_flags & CS_CFG_MATCH_CAP_TRCEXTOUT)
> + used += scnprintf(page + used, PAGE_SIZE - used, "TRCEXTOUT ");
> +
> used += scnprintf(page + used, PAGE_SIZE - used, "\n");
> return used;
> }