On 6/12/2024 7:53 AM, Tomeu Vizoso wrote:
> diff --git a/include/uapi/drm/rocket_accel.h b/include/uapi/drm/rocket_accel.h
> index 888c9413e4cd..1539af0af4fe 100644
> --- a/include/uapi/drm/rocket_accel.h
> +++ b/include/uapi/drm/rocket_accel.h
> @@ -12,9 +12,13 @@ extern "C" {
> #endif
>
> #define DRM_ROCKET_CREATE_BO 0x00
> -#define DRM_ROCKET_SUBMIT 0x01
> +#define DRM_ROCKET_PREP_BO 0x01
> +#define DRM_ROCKET_FINI_BO 0x02
> +#define DRM_ROCKET_SUBMIT 0x03
This looks like a uAPI breaking change. Shouldn't you have defined
SUBMIT as 0x03 from the beginning, or put the new BO ioctls after it?
On 6/12/2024 7:53 AM, Tomeu Vizoso wrote:
> This uses the SHMEM DRM helpers and we map right away to the CPU and NPU
> sides, as all buffers are expected to be accessed from both.
>
> Signed-off-by: Tomeu Vizoso <tomeu(a)tomeuvizoso.net>
Reviewed-by: Jeffrey Hugo <quic_jhugo(a)quicinc.com>
On 2024-06-13 10:38 pm, Sebastian Reichel wrote:
> Hi,
>
> On Thu, Jun 13, 2024 at 11:34:02AM GMT, Tomeu Vizoso wrote:
>> On Thu, Jun 13, 2024 at 11:24 AM Tomeu Vizoso <tomeu(a)tomeuvizoso.net> wrote:
>>> On Thu, Jun 13, 2024 at 2:05 AM Sebastian Reichel
>>> <sebastian.reichel(a)collabora.com> wrote:
>>>> On Wed, Jun 12, 2024 at 03:52:55PM GMT, Tomeu Vizoso wrote:
>>>>> IOMMUs with multiple base addresses can also have multiple power
>>>>> domains.
>>>>>
>>>>> The base framework only takes care of a single power domain, as some
>>>>> devices will need for these power domains to be powered on in a specific
>>>>> order.
>>>>>
>>>>> Use a helper function to stablish links in the order in which they are
>>>>> in the DT.
>>>>>
>>>>> This is needed by the IOMMU used by the NPU in the RK3588.
>>>>>
>>>>> Signed-off-by: Tomeu Vizoso <tomeu(a)tomeuvizoso.net>
>>>>> ---
>>>>
>>>> To me it looks like this is multiple IOMMUs, which should each get
>>>> their own node. I don't see a good reason for merging these
>>>> together.
>>>
>>> I have made quite a few attempts at splitting the IOMMUs and also the
>>> cores, but I wasn't able to get things working stably. The TRM is
>>> really scant about how the 4 IOMMU instances relate to each other, and
>>> what the fourth one is for.
>>>
>>> Given that the vendor driver treats them as a single IOMMU with four
>>> instances and we don't have any information on them, I resigned myself
>>> to just have them as a single device.
>>>
>>> I would love to be proved wrong though and find a way fo getting
>>> things stably as different devices so they can be powered on and off
>>> as needed. We could save quite some code as well.
>>
>> FWIW, here a few ways how I tried to structure the DT nodes, none of
>> these worked reliably:
>>
>> https://gitlab.freedesktop.org/tomeu/linux/-/blob/6.10-rocket-multiple-devi…
>> https://gitlab.freedesktop.org/tomeu/linux/-/blob/6.10-rocket-schema-subnod…
>> https://gitlab.freedesktop.org/tomeu/linux/-/blob/6.10-rocket-multiple-devi…
>> https://gitlab.freedesktop.org/tomeu/linux/-/blob/6.10-rocket-multiple-iomm…
>>
>> I can very well imagine I missed some way of getting this to work, but
>> for every attempt, the domains, iommus and cores were resumed in
>> different orders that presumably caused problems during concurrent
>> execution fo workloads.
>>
>> So I fell back to what the vendor driver does, which works reliably
>> (but all cores have to be powered on at the same time).
>
> Mh. The "6.10-rocket-multiple-iommus" branch seems wrong. There is
> only one iommu node in that. I would have expected a test with
>
> rknn {
> // combined device
>
> iommus = <&iommu1>, <&iommu2>, ...;
> };
>
> Otherwise I think I would go with the schema-subnodes variant. The
> driver can initially walk through the sub-nodes and collect the
> resources into the main device, so on the driver side nothing would
> really change. But that has a couple of advantages:
>
> 1. DT and DT binding are easier to read
> 2. It's similar to e.g. CPU cores each having their own node
> 3. Easy to extend to more cores in the future
> 4. The kernel can easily switch to proper per-core device model when
> the problem has been identified
It also would seem to permit describing and associating the per-core
IOMMUs individually - apart from core 0's apparent coupling to whatever
shared "uncore" stuff exists for the whole thing, from the distinct
clocks, interrupts, power domains etc. lining up with each core I'd
guess those IOMMUs are not interrelated the same way the ISP's
read/write IOMMUs are (which was the main justification for adopting the
multiple-reg design originally vs. distinct DT nodes like Exynos does).
However, practically that would require the driver to at least populate
per-core child devices to make DMA API or IOMMU API mappings with, since
we couldn't spread the "collect the resources" trick into those
subsystems as well.
Thanks,
Robin.
On Fri, Jun 14, 2024 at 09:27:43AM +0200, Alexandre Mergnat wrote:
> This serie aim to add the following audio support for the Genio 350-evk:
> - Playback
> - 2ch Headset Jack (Earphone)
> - 1ch Line-out Jack (Speaker)
> - 8ch HDMI Tx
I seem to remember you had review comments that needed addressing from
AngeloGioacchino, why resend without addressing those?
On Wed, 12 Jun 2024 15:52:53 +0200, Tomeu Vizoso wrote:
> This series adds a new driver for the NPU that Rockchip includes in its
> newer SoCs, developed by them on the NVDLA base.
>
> In its current form, it supports the specific NPU in the RK3588 SoC.
>
> The userspace driver is part of Mesa and an initial draft can be found at:
>
> https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29698
>
> Signed-off-by: Tomeu Vizoso <tomeu(a)tomeuvizoso.net>
> ---
> Tomeu Vizoso (9):
> iommu/rockchip: Add compatible for rockchip,rk3588-iommu
> iommu/rockchip: Attach multiple power domains
> dt-bindings: mailbox: rockchip,rknn: Add bindings
> arm64: dts: rockchip: Add nodes for NPU and its MMU to rk3588s
> arm64: dts: rockchip: Enable the NPU on quartzpro64
> accel/rocket: Add a new driver for Rockchip's NPU
> accel/rocket: Add IOCTL for BO creation
> accel/rocket: Add job submission IOCTL
> accel/rocket: Add IOCTLs for synchronizing memory accesses
>
> .../devicetree/bindings/npu/rockchip,rknn.yaml | 123 +
> MAINTAINERS | 8 +
> .../arm64/boot/dts/rockchip/rk3588-quartzpro64.dts | 8 +
> arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 53 +
> drivers/accel/Kconfig | 1 +
> drivers/accel/Makefile | 1 +
> drivers/accel/rocket/Kconfig | 13 +
> drivers/accel/rocket/Makefile | 10 +
> drivers/accel/rocket/rocket_core.c | 155 +
> drivers/accel/rocket/rocket_core.h | 48 +
> drivers/accel/rocket/rocket_device.c | 39 +
> drivers/accel/rocket/rocket_device.h | 40 +
> drivers/accel/rocket/rocket_drv.c | 243 ++
> drivers/accel/rocket/rocket_drv.h | 16 +
> drivers/accel/rocket/rocket_gem.c | 136 +
> drivers/accel/rocket/rocket_gem.h | 33 +
> drivers/accel/rocket/rocket_job.c | 708 ++++
> drivers/accel/rocket/rocket_job.h | 49 +
> drivers/accel/rocket/rocket_registers.h | 4449 ++++++++++++++++++++
> drivers/iommu/rockchip-iommu.c | 39 +
> include/uapi/drm/rocket_accel.h | 116 +
> 21 files changed, 6288 insertions(+)
> ---
> base-commit: 83a7eefedc9b56fe7bfeff13b6c7356688ffa670
> change-id: 20240612-6-10-rocket-9316defc14c7
>
> Best regards,
> --
> Tomeu Vizoso <tomeu(a)tomeuvizoso.net>
>
>
>
My bot found new DTB warnings on the .dts files added or changed in this
series.
Some warnings may be from an existing SoC .dtsi. Or perhaps the warnings
are fixed by another series. Ultimately, it is up to the platform
maintainer whether these warnings are acceptable or not. No need to reply
unless the platform maintainer has comments.
If you already ran DT checks and didn't see these error(s), then
make sure dt-schema is up to date:
pip3 install dtschema --upgrade
New warnings running 'make CHECK_DTBS=y rockchip/rk3588-quartzpro64.dtb' for 20240612-6-10-rocket-v1-0-060e48eea250(a)tomeuvizoso.net:
arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: compatible: 'oneOf' conditional failed, one must be fixed:
['rockchip,rk3588-iommu'] is too short
'rockchip,rk3588-iommu' is not one of ['rockchip,iommu', 'rockchip,rk3568-iommu']
from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml#
arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: reg: [[0, 4255879168, 0, 256], [0, 4255883264, 0, 256], [0, 4255948800, 0, 256], [0, 4256014336, 0, 256]] is too long
from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml#
arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: interrupts: [[0, 110, 4, 0], [0, 111, 4, 0], [0, 112, 4, 0]] is too long
from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml#
arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: clocks: [[28, 287], [28, 276], [28, 278], [28, 288], [28, 277], [28, 279]] is too long
from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml#
arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: clock-names:0: 'aclk' was expected
from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml#
arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: clock-names:1: 'iface' was expected
from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml#
arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: clock-names: ['aclk0', 'aclk1', 'aclk2', 'iface0', 'iface1', 'iface2'] is too long
from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml#
arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: power-domains: [[30, 9], [30, 10], [30, 11]] is too long
from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml#
arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: 'interrupt-names', 'power-domain-names' do not match any of the regexes: 'pinctrl-[0-9]+'
from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml#
Hi Tomeu,
kernel test robot noticed the following build warnings:
[auto build test WARNING on 83a7eefedc9b56fe7bfeff13b6c7356688ffa670]
url: https://github.com/intel-lab-lkp/linux/commits/Tomeu-Vizoso/iommu-rockchip-…
base: 83a7eefedc9b56fe7bfeff13b6c7356688ffa670
patch link: https://lore.kernel.org/r/20240612-6-10-rocket-v1-8-060e48eea250%40tomeuviz…
patch subject: [PATCH 8/9] accel/rocket: Add job submission IOCTL
config: hexagon-allmodconfig (https://download.01.org/0day-ci/archive/20240613/202406131640.WbBaRMbr-lkp@…)
compiler: clang version 19.0.0git (https://github.com/llvm/llvm-project 4403cdbaf01379de96f8d0d6ea4f51a085e37766)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240613/202406131640.WbBaRMbr-lkp@…)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp(a)intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202406131640.WbBaRMbr-lkp@intel.com/
All warnings (new ones prefixed by >>):
In file included from drivers/accel/rocket/rocket_job.c:6:
In file included from include/drm/drm_file.h:39:
In file included from include/drm/drm_prime.h:37:
In file included from include/linux/scatterlist.h:8:
In file included from include/linux/mm.h:2253:
include/linux/vmstat.h:514:36: warning: arithmetic between different enumeration types ('enum node_stat_item' and 'enum lru_list') [-Wenum-enum-conversion]
514 | return node_stat_name(NR_LRU_BASE + lru) + 3; // skip "nr_"
| ~~~~~~~~~~~ ^ ~~~
In file included from drivers/accel/rocket/rocket_job.c:6:
In file included from include/drm/drm_file.h:39:
In file included from include/drm/drm_prime.h:37:
In file included from include/linux/scatterlist.h:9:
In file included from arch/hexagon/include/asm/io.h:328:
include/asm-generic/io.h:548:31: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
548 | val = __raw_readb(PCI_IOBASE + addr);
| ~~~~~~~~~~ ^
include/asm-generic/io.h:561:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
561 | val = __le16_to_cpu((__le16 __force)__raw_readw(PCI_IOBASE + addr));
| ~~~~~~~~~~ ^
include/uapi/linux/byteorder/little_endian.h:37:51: note: expanded from macro '__le16_to_cpu'
37 | #define __le16_to_cpu(x) ((__force __u16)(__le16)(x))
| ^
In file included from drivers/accel/rocket/rocket_job.c:6:
In file included from include/drm/drm_file.h:39:
In file included from include/drm/drm_prime.h:37:
In file included from include/linux/scatterlist.h:9:
In file included from arch/hexagon/include/asm/io.h:328:
include/asm-generic/io.h:574:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
574 | val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr));
| ~~~~~~~~~~ ^
include/uapi/linux/byteorder/little_endian.h:35:51: note: expanded from macro '__le32_to_cpu'
35 | #define __le32_to_cpu(x) ((__force __u32)(__le32)(x))
| ^
In file included from drivers/accel/rocket/rocket_job.c:6:
In file included from include/drm/drm_file.h:39:
In file included from include/drm/drm_prime.h:37:
In file included from include/linux/scatterlist.h:9:
In file included from arch/hexagon/include/asm/io.h:328:
include/asm-generic/io.h:585:33: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
585 | __raw_writeb(value, PCI_IOBASE + addr);
| ~~~~~~~~~~ ^
include/asm-generic/io.h:595:59: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
595 | __raw_writew((u16 __force)cpu_to_le16(value), PCI_IOBASE + addr);
| ~~~~~~~~~~ ^
include/asm-generic/io.h:605:59: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
605 | __raw_writel((u32 __force)cpu_to_le32(value), PCI_IOBASE + addr);
| ~~~~~~~~~~ ^
>> drivers/accel/rocket/rocket_job.c:353:11: warning: variable 'status' set but not used [-Wunused-but-set-variable]
353 | uint32_t status, raw_status;
| ^
>> drivers/accel/rocket/rocket_job.c:353:19: warning: variable 'raw_status' set but not used [-Wunused-but-set-variable]
353 | uint32_t status, raw_status;
| ^
drivers/accel/rocket/rocket_job.c:40:1: warning: unused function 'to_rocket_fence' [-Wunused-function]
40 | to_rocket_fence(struct dma_fence *fence)
| ^~~~~~~~~~~~~~~
10 warnings generated.
Kconfig warnings: (for reference only)
WARNING: unmet direct dependencies detected for IOMMU_IO_PGTABLE_LPAE
Depends on [n]: IOMMU_SUPPORT [=y] && (ARM || ARM64 || COMPILE_TEST [=y]) && !GENERIC_ATOMIC64 [=y]
Selected by [m]:
- DRM_ACCEL_ROCKET [=m] && DRM [=m] && (ARM64 || COMPILE_TEST [=y]) && MMU [=y]
vim +/status +353 drivers/accel/rocket/rocket_job.c
350
351 static void rocket_job_handle_irq(struct rocket_core *core)
352 {
> 353 uint32_t status, raw_status;
354
355 pm_runtime_mark_last_busy(core->dev->dev);
356
357 status = rocket_read(core, REG_PC_INTERRUPT_STATUS);
358 raw_status = rocket_read(core, REG_PC_INTERRUPT_RAW_STATUS);
359
360 rocket_write(core, REG_PC_OPERATION_ENABLE, 0x0);
361 rocket_write(core, REG_PC_INTERRUPT_CLEAR, 0x1ffff);
362
363 spin_lock(&core->job_lock);
364
365 if (core->in_flight_job)
366 rocket_job_handle_done(core, core->in_flight_job);
367
368 spin_unlock(&core->job_lock);
369 }
370
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
Hi Tomeu,
kernel test robot noticed the following build errors:
[auto build test ERROR on 83a7eefedc9b56fe7bfeff13b6c7356688ffa670]
url: https://github.com/intel-lab-lkp/linux/commits/Tomeu-Vizoso/iommu-rockchip-…
base: 83a7eefedc9b56fe7bfeff13b6c7356688ffa670
patch link: https://lore.kernel.org/r/20240612-6-10-rocket-v1-6-060e48eea250%40tomeuviz…
patch subject: [PATCH 6/9] accel/rocket: Add a new driver for Rockchip's NPU
config: s390-allyesconfig (https://download.01.org/0day-ci/archive/20240613/202406131022.1JKNS7me-lkp@…)
compiler: s390-linux-gcc (GCC) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240613/202406131022.1JKNS7me-lkp@…)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp(a)intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202406131022.1JKNS7me-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from include/linux/clk.h:13,
from drivers/accel/rocket/rocket_drv.c:4:
>> drivers/accel/rocket/rocket_drv.c:213:31: error: 'rocket_pm_ops' undeclared here (not in a function); did you mean 'rocket_probe'?
213 | .pm = pm_ptr(&rocket_pm_ops),
| ^~~~~~~~~~~~~
include/linux/kernel.h:48:44: note: in definition of macro 'PTR_IF'
48 | #define PTR_IF(cond, ptr) ((cond) ? (ptr) : NULL)
| ^~~
drivers/accel/rocket/rocket_drv.c:213:23: note: in expansion of macro 'pm_ptr'
213 | .pm = pm_ptr(&rocket_pm_ops),
| ^~~~~~
vim +213 drivers/accel/rocket/rocket_drv.c
207
208 static struct platform_driver rocket_driver = {
209 .probe = rocket_probe,
210 .remove_new = rocket_remove,
211 .driver = {
212 .name = "rocket",
> 213 .pm = pm_ptr(&rocket_pm_ops),
214 .of_match_table = dt_match,
215 },
216 };
217 module_platform_driver(rocket_driver);
218
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki