- Linaro-mm-sig - lists.linaro.org

Re: [PATCH 6/9] accel/rocket: Add a new driver for Rockchip's NPU

by Jeffrey Hugo

On 6/12/2024 7:52 AM, Tomeu Vizoso wrote: > This initial version supports the NPU as shipped in the RK3588 SoC and > described in the first part of its TRM, in Chapter 36. > > This NPU contains 3 independent cores that the driver can submit jobs > to. > > This commit adds just hardware initialization and power management. > > Signed-off-by: Tomeu Vizoso <tomeu(a)tomeuvizoso.net> > --- > MAINTAINERS | 8 + > drivers/accel/Kconfig | 1 + > drivers/accel/Makefile | 1 + > drivers/accel/rocket/Kconfig | 13 + > drivers/accel/rocket/Makefile | 8 + > drivers/accel/rocket/rocket_core.c | 149 ++ > drivers/accel/rocket/rocket_core.h | 32 + > drivers/accel/rocket/rocket_device.c | 37 + > drivers/accel/rocket/rocket_device.h | 38 + > drivers/accel/rocket/rocket_drv.c | 221 ++ > drivers/accel/rocket/rocket_drv.h | 13 + > drivers/accel/rocket/rocket_registers.h | 4449 +++++++++++++++++++++++++++++++ > include/uapi/drm/rocket_accel.h | 19 + > 13 files changed, 4989 insertions(+) How about something in Documentation? > > diff --git a/MAINTAINERS b/MAINTAINERS > index aacccb376c28..5a42ee39e2e6 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -6731,6 +6731,14 @@ T: git https://gitlab.freedesktop.org/drm/misc/kernel.git > F: drivers/accel/ivpu/ > F: include/uapi/drm/ivpu_accel.h > > +DRM ACCEL DRIVER FOR ROCKCHIP NPU > +M: Tomeu Vizoso <tomeu.vizoso(a)tomeuvizoso.net> > +L: dri-devel(a)lists.freedesktop.org > +S: Supported > +T: git https://gitlab.freedesktop.org/drm/misc/kernel.git > +F: drivers/accel/rocket/ > +F: include/uapi/drm/rocket_accel.h > + > DRM COMPUTE ACCELERATORS DRIVERS AND FRAMEWORK > M: Oded Gabbay <ogabbay(a)kernel.org> > L: dri-devel(a)lists.freedesktop.org > diff --git a/drivers/accel/Kconfig b/drivers/accel/Kconfig > index 64065fb8922b..8f79eea29e29 100644 > --- a/drivers/accel/Kconfig > +++ b/drivers/accel/Kconfig > @@ -27,5 +27,6 @@ menuconfig DRM_ACCEL > source "drivers/accel/habanalabs/Kconfig" > source "drivers/accel/ivpu/Kconfig" > source "drivers/accel/qaic/Kconfig" > +source "drivers/accel/rocket/Kconfig" > > endif > diff --git a/drivers/accel/Makefile b/drivers/accel/Makefile > index ab3df932937f..117844f94fc9 100644 > --- a/drivers/accel/Makefile > +++ b/drivers/accel/Makefile > @@ -3,3 +3,4 @@ > obj-$(CONFIG_DRM_ACCEL_HABANALABS) += habanalabs/ > obj-$(CONFIG_DRM_ACCEL_IVPU) += ivpu/ > obj-$(CONFIG_DRM_ACCEL_QAIC) += qaic/ > +obj-$(CONFIG_DRM_ACCEL_ROCKET) += rocket/ > \ No newline at end of file > diff --git a/drivers/accel/rocket/Kconfig b/drivers/accel/rocket/Kconfig > new file mode 100644 > index 000000000000..d50eaabb42cb > --- /dev/null > +++ b/drivers/accel/rocket/Kconfig > @@ -0,0 +1,13 @@ > +# SPDX-License-Identifier: GPL-2.0 > + > +config DRM_ACCEL_ROCKET > + tristate "Rocket (DRM support for Rockchip NPUs)" > + depends on DRM > + depends on ARM64 || COMPILE_TEST > + depends on MMU > + select DRM_SCHED > + select IOMMU_SUPPORT > + select IOMMU_IO_PGTABLE_LPAE > + select DRM_GEM_SHMEM_HELPER > + help > + DRM driver for Rockchip NPUs. This is way too short. At a minimum what is the module called? > diff --git a/drivers/accel/rocket/Makefile b/drivers/accel/rocket/Makefile > new file mode 100644 > index 000000000000..73a7280d260c > --- /dev/null > +++ b/drivers/accel/rocket/Makefile > @@ -0,0 +1,8 @@ > +# SPDX-License-Identifier: GPL-2.0 > + > +obj-$(CONFIG_DRM_ACCEL_ROCKET) := rocket.o > + > +rocket-y := \ > + rocket_core.o \ > + rocket_device.o \ > + rocket_drv.o > diff --git a/drivers/accel/rocket/rocket_core.c b/drivers/accel/rocket/rocket_core.c > new file mode 100644 > index 000000000000..d6680b00fb2f > --- /dev/null > +++ b/drivers/accel/rocket/rocket_core.c > @@ -0,0 +1,149 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* Copyright 2024 Tomeu Vizoso <tomeu(a)tomeuvizoso.net> */ > + > +#include <asm-generic/delay.h> Really? You need to go down into the asm specific headers? That seems wrong. > +#include <linux/err.h> > +#include <linux/clk.h> "c" comes before "e" in the English alphabet. > +#include <linux/device.h> > +#include <linux/platform_device.h> > +#include <linux/pm_domain.h> > +#include <linux/reset.h> > + > +#include "rocket_core.h" > +#include "rocket_device.h" > +#include "rocket_registers.h" > + > +static int rocket_clk_init(struct rocket_core *core) > +{ > + struct rocket_device *rdev = core->dev; > + int err; > + > + core->a_clk = devm_clk_get_enabled(rdev->dev, rdev->comp->clocks_a_names[core->index]); > + if (IS_ERR(core->a_clk)) { > + err = PTR_ERR(core->a_clk); > + dev_err(rdev->dev, "devm_clk_get_enabled failed %d for core %d\n", err, core->index); > + return err; > + } > + > + core->h_clk = devm_clk_get_enabled(rdev->dev, rdev->comp->clocks_h_names[core->index]); > + if (IS_ERR(core->h_clk)) { > + err = PTR_ERR(core->h_clk); > + dev_err(rdev->dev, "devm_clk_get_enabled failed %d for core %d\n", err, core->index); > + clk_disable_unprepare(core->a_clk); > + return err; > + } > + > + return 0; > +} > + > +static int rocket_reset_init(struct rocket_core *core) > +{ > + struct rocket_device *rdev = core->dev; > + struct reset_control *a_reset = NULL; > + struct reset_control *h_reset = NULL; > + > + a_reset = devm_reset_control_get( > + rdev->dev, > + rdev->comp->resets_a_names[core->index]); This fits on one line, why break it up? > + if (IS_ERR(a_reset)) > + return PTR_ERR(a_reset); > + > + core->a_reset = a_reset; > + > + h_reset = devm_reset_control_get( > + rdev->dev, > + rdev->comp->resets_h_names[core->index]); Same comment. > + if (IS_ERR(h_reset)) > + return PTR_ERR(h_reset); > + > + core->h_reset = h_reset; > + > + return 0; > +} > + > +static int rocket_pmdomain_init(struct rocket_core *core) > +{ > + struct rocket_device *rdev = core->dev; > + const char *pm_domain_name = rdev->comp->pm_domain_names[core->index]; Seems a bit weird to break reverse christmas tree. > + int err = 0; > + > + core->pm_domain = dev_pm_domain_attach_by_name(rdev->dev, pm_domain_name); > + if (IS_ERR_OR_NULL(core->pm_domain)) { > + err = PTR_ERR(core->pm_domain) ? : -ENODATA; > + core->pm_domain = NULL; > + dev_err(rdev->dev, > + "failed to get pm-domain %s(%d): %d\n", > + pm_domain_name, core->index, err); > + return err; > + } > + > + core->pm_domain_link = device_link_add(rdev->dev, > + core->pm_domain, > + DL_FLAG_PM_RUNTIME | DL_FLAG_STATELESS | DL_FLAG_RPM_ACTIVE); > + if (!core->pm_domain_link) { > + dev_err(core->pm_domain, "adding device link failed!\n"); > + dev_pm_domain_detach(core->pm_domain, true); > + return -ENODEV; > + } > + > + return err; > +} > + > +static void rocket_pmdomain_fini(struct rocket_core *core) > +{ > + dev_pm_domain_detach(core->pm_domain, true); > +} > + > +int rocket_core_init(struct rocket_core *core) > +{ > + struct rocket_device *rdev = core->dev; > + uint32_t version; > + int err = 0; > + > + err = rocket_clk_init(core); > + if (err) { > + dev_err(rdev->dev, "clk init failed %d\n", err); > + return err; > + } > + > + err = rocket_reset_init(core); > + if (err) { > + dev_err(rdev->dev, "reset init failed %d\n", err); > + return err; > + } > + > + err = rocket_pmdomain_init(core); > + if (err < 0) > + return err; > + > + core->iomem = devm_platform_ioremap_resource(rdev->pdev, core->index); > + if (IS_ERR(core->iomem)) { > + err = PTR_ERR(core->iomem); > + goto out_pm_domain; > + } > + > + version = rocket_read(core, REG_PC_VERSION) + (rocket_read(core, REG_PC_VERSION_NUM) & 0xffff); > + dev_info(rdev->dev, "Rockchip NPU core %d version: %d\n", core->index, version); A properly working driver should be silent by default. > + > + return 0; > + > +out_pm_domain: > + rocket_pmdomain_fini(core); > + return err; > +} > + > +void rocket_core_fini(struct rocket_core *core) > +{ > + rocket_pmdomain_fini(core); > +} > + > +void rocket_core_reset(struct rocket_core *core) > +{ > + reset_control_assert(core->a_reset); > + reset_control_assert(core->h_reset); > + > + udelay(10); Why? > + > + reset_control_deassert(core->a_reset); > + reset_control_deassert(core->h_reset); > +} > diff --git a/drivers/accel/rocket/rocket_core.h b/drivers/accel/rocket/rocket_core.h > new file mode 100644 > index 000000000000..e5d4c848c9f4 > --- /dev/null > +++ b/drivers/accel/rocket/rocket_core.h > @@ -0,0 +1,32 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* Copyright 2024 Tomeu Vizoso <tomeu(a)tomeuvizoso.net> */ > + > +#ifndef __ROCKET_CORE_H__ > +#define __ROCKET_CORE_H__ > + > +#include <linux/mutex_types.h> > +#include <asm/io.h> > +#include <asm-generic/io.h> Again, asm specific includes? > +#define rocket_read(core, reg) readl((core)->iomem + (reg)) > +#define rocket_write(core, reg, value) writel(value, (core)->iomem + (reg)) > + > +struct rocket_core { > + struct rocket_device *dev; > + unsigned int index; > + > + struct reset_control *a_reset; > + struct reset_control *h_reset; > + void __iomem *iomem; > + int irq; What does pahole say about this? > + struct clk *a_clk; > + struct clk *h_clk; > + struct device *pm_domain; > + struct device_link *pm_domain_link; > +}; > + > +int rocket_core_init(struct rocket_core *core); > +void rocket_core_fini(struct rocket_core *core); > +void rocket_core_reset(struct rocket_core *core); > + > +#endif > \ No newline at end of file > diff --git a/drivers/accel/rocket/rocket_device.c b/drivers/accel/rocket/rocket_device.c > new file mode 100644 > index 000000000000..3bd23f22d06a > --- /dev/null > +++ b/drivers/accel/rocket/rocket_device.c > @@ -0,0 +1,37 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* Copyright 2024 Tomeu Vizoso <tomeu(a)tomeuvizoso.net> */ > + > +#include <linux/clk.h> > +#include <linux/pm_runtime.h> > + > +#include "rocket_drv.h" > +#include "rocket_device.h" > + > +int rocket_device_init(struct rocket_device *rdev) > +{ > + int core, err; > + > + rdev->clk_npu = devm_clk_get_enabled(rdev->dev, "clk_npu"); > + rdev->pclk = devm_clk_get_enabled(rdev->dev, "pclk"); > + > + for (core = 0; core < rdev->comp->num_cores; core++) { > + rdev->cores[core].dev = rdev; > + rdev->cores[core].index = core; > + > + err = rocket_core_init(&rdev->cores[core]); > + if (err) { > + rocket_device_fini(rdev); > + return err; > + } > + } > + > + return 0; > +} > + > +void rocket_device_fini(struct rocket_device *rdev) > +{ > + int core; > + > + for (core = 0; core < rdev->comp->num_cores; core++) > + rocket_core_fini(&rdev->cores[core]); > +} > diff --git a/drivers/accel/rocket/rocket_device.h b/drivers/accel/rocket/rocket_device.h > new file mode 100644 > index 000000000000..6765d944a2c8 > --- /dev/null > +++ b/drivers/accel/rocket/rocket_device.h > @@ -0,0 +1,38 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* Copyright 2024 Tomeu Vizoso <tomeu(a)tomeuvizoso.net> */ > + > +#ifndef __ROCKET_DEVICE_H__ > +#define __ROCKET_DEVICE_H__ > + > +#include "rocket_core.h" > + > +#define MAX_NUM_CORES 3 > + > +struct rocket_compatible { > + int num_cores; > + const char * const *resets_a_names; > + const char * const *resets_h_names; > + const char * const *clocks_a_names; > + const char * const *clocks_h_names; > + const char * const *pm_domain_names; > + const char * const *irq_names; > +}; > + > +struct rocket_device { > + struct device *dev; > + struct drm_device *ddev; > + struct platform_device *pdev; > + > + const struct rocket_compatible *comp; > + > + struct rocket_core cores[MAX_NUM_CORES]; > + > + struct clk *clk_npu; > + struct clk *pclk; > +}; > + > +int rocket_device_init(struct rocket_device *rdev); > +void rocket_device_fini(struct rocket_device *rdev); > +void rocket_device_reset(struct rocket_device *rdev); > + > +#endif > diff --git a/drivers/accel/rocket/rocket_drv.c b/drivers/accel/rocket/rocket_drv.c > new file mode 100644 > index 000000000000..e339c337fcf9 > --- /dev/null > +++ b/drivers/accel/rocket/rocket_drv.c > @@ -0,0 +1,221 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* Copyright 2024 Tomeu Vizoso <tomeu(a)tomeuvizoso.net> */ > + > +#include "linux/clk.h" > +#include "linux/dma-mapping.h" Use of "" includes can't be right... > +#include <linux/of.h> > +#include <linux/module.h> > +#include <linux/platform_device.h> > +#include <linux/pm_runtime.h> > +#include <drm/drm_drv.h> > +#include <drm/drm_ioctl.h> > +#include <drm/drm_accel.h> Broken alphabetical order. > +#include <drm/drm_gem.h> > + > +#include "rocket_drv.h" > +#include "rocket_device.h" > + > +static const char * const rk3588_pm_domains[] = { "npu0", "npu1", "npu2" }; > +static const char * const rk3588_resets_a[] = { "srst_a0", "srst_a1", "srst_a2" }; > +static const char * const rk3588_resets_h[] = { "srst_h0", "srst_h1", "srst_h2" }; > +static const char * const rk3588_clocks_a[] = { "aclk0", "aclk1", "aclk2" }; > +static const char * const rk3588_clocks_h[] = { "hclk0", "hclk1", "hclk2" }; > +static const char * const rk3588_irqs[] = { "npu0_irq", "npu1_irq", "npu2_irq" }; > +static const struct rocket_compatible rk3588_data = { > + .num_cores = ARRAY_SIZE(rk3588_pm_domains), > + .pm_domain_names = rk3588_pm_domains, > + .resets_a_names = rk3588_resets_a, > + .resets_h_names = rk3588_resets_h, > + .clocks_a_names = rk3588_clocks_a, > + .clocks_h_names = rk3588_clocks_h, > + .irq_names = rk3588_irqs, > +}; > + > +static int > +rocket_open(struct drm_device *dev, struct drm_file *file) > +{ > + struct rocket_device *rdev = dev->dev_private; > + struct rocket_file_priv *rocket_priv; > + > + rocket_priv = kzalloc(sizeof(*rocket_priv), GFP_KERNEL); > + if (!rocket_priv) > + return -ENOMEM; > + > + rocket_priv->rdev = rdev; > + file->driver_priv = rocket_priv; > + > + return 0; > +} > + > +static void > +rocket_postclose(struct drm_device *dev, struct drm_file *file) > +{ > + struct rocket_file_priv *rocket_priv = file->driver_priv; > + > + kfree(rocket_priv); > +} > + > +static const struct drm_ioctl_desc rocket_drm_driver_ioctls[] = { > +#define ROCKET_IOCTL(n, func) \ > + DRM_IOCTL_DEF_DRV(ROCKET_##n, rocket_ioctl_##func, 0) > +}; > + > +static const struct file_operations rocket_drm_driver_fops = { > + .owner = THIS_MODULE, This is weird. I'd expect a use of DEFINE_DRM_ACCEL_FOPS() Are you sure you need to explictly set .owner? Thats been stripped out of the kernel as far as I've seen. > + DRM_ACCEL_FOPS, > +}; > + > +/* > + * Rocket driver version: > + * - 1.0 - initial interface > + */ > +static const struct drm_driver rocket_drm_driver = { > + .driver_features = DRIVER_COMPUTE_ACCEL, > + .open = rocket_open, > + .postclose = rocket_postclose, > + .ioctls = rocket_drm_driver_ioctls, > + .num_ioctls = ARRAY_SIZE(rocket_drm_driver_ioctls), > + .fops = &rocket_drm_driver_fops, > + .name = "rocket", > + .desc = "rocket DRM", > + .date = "20240521", > + .major = 1, > + .minor = 0, Major, minor has been deprecated. Pretty sure the data is being phased out right now if it hasn't been already. > +}; > + > +static int rocket_probe(struct platform_device *pdev) > +{ > + struct rocket_device *rdev; > + struct drm_device *ddev; > + int err; > + > + BUILD_BUG_ON(rk3588_data.num_cores > MAX_NUM_CORES); > + > + rdev = devm_kzalloc(&pdev->dev, sizeof(*rdev), GFP_KERNEL); > + if (!rdev) > + return -ENOMEM; > + > + rdev->pdev = pdev; > + rdev->dev = &pdev->dev; > + > + platform_set_drvdata(pdev, rdev); > + > + rdev->comp = of_device_get_match_data(&pdev->dev); > + if (!rdev->comp) > + return -ENODEV; > + > + /* Allocate and initialize the DRM device. */ Doesn't seem like this comment provides any value since it just reiterates what the code is. > + ddev = drm_dev_alloc(&rocket_drm_driver, &pdev->dev); Why not devm_drm_dev_alloc()? > + if (IS_ERR(ddev)) > + return PTR_ERR(ddev); > + > + ddev->dev_private = rdev; > + rdev->ddev = ddev; > + > + err = dma_set_mask_and_coherent(rdev->ddev->dev, DMA_BIT_MASK(40)); > + if (err) > + goto err_drm_dev; > + > + err = rocket_device_init(rdev); > + if (err) { > + if (err != -EPROBE_DEFER) > + dev_err(&pdev->dev, "Fatal error during NPU init\n"); Isn't there an EPROBE_DEFER helper for this usecase? > + goto err_drm_dev; > + } > + > + pm_runtime_use_autosuspend(rdev->dev); > + pm_runtime_set_autosuspend_delay(rdev->dev, 50); /* ~3 frames */ > + pm_runtime_set_active(rdev->dev); > + pm_runtime_enable(rdev->dev); > + pm_runtime_mark_last_busy(rdev->dev); > + > + /* > + * Register the DRM device with the core and the connectors with > + * sysfs > + */ Another no-value comment. > + err = drm_dev_register(ddev, 0); > + if (err < 0) > + goto err_pm_runtime; > + > + return 0; > + > +err_pm_runtime: > + pm_runtime_disable(rdev->dev); > + rocket_device_fini(rdev); > + pm_runtime_set_suspended(rdev->dev); > +err_drm_dev: > + drm_dev_put(ddev); > + return err; > +} > + > +static void rocket_remove(struct platform_device *pdev) > +{ > + struct rocket_device *rdev = platform_get_drvdata(pdev); > + struct drm_device *ddev = rdev->ddev; > + > + drm_dev_unregister(ddev); > + > + pm_runtime_disable(rdev->dev); > + rocket_device_fini(rdev); > + pm_runtime_set_suspended(rdev->dev); > + > + drm_dev_put(ddev); > +} > + > +static const struct of_device_id dt_match[] = { > + { .compatible = "rockchip,rk3588-rknn", .data = &rk3588_data, }, > + {} > +}; > +MODULE_DEVICE_TABLE(of, dt_match); > + > +static int rocket_device_runtime_resume(struct device *dev) > +{ > + struct rocket_device *rdev = dev_get_drvdata(dev); > + int core; > + > + clk_prepare_enable(rdev->clk_npu); > + clk_prepare_enable(rdev->pclk); > + > + for (core = 0; core < rdev->comp->num_cores; core++) { > + clk_prepare_enable(rdev->cores[core].a_clk); > + clk_prepare_enable(rdev->cores[core].h_clk); > + } > + > + return 0; > +} > + > +static int rocket_device_runtime_suspend(struct device *dev) > +{ > + struct rocket_device *rdev = dev_get_drvdata(dev); > + int core; > + > + for (core = 0; core < rdev->comp->num_cores; core++) { > + clk_disable_unprepare(rdev->cores[core].a_clk); > + clk_disable_unprepare(rdev->cores[core].h_clk); > + } > + > + clk_disable_unprepare(rdev->pclk); > + clk_disable_unprepare(rdev->clk_npu); > + > + return 0; > +} > + > +EXPORT_GPL_DEV_PM_OPS(rocket_pm_ops) = { > + RUNTIME_PM_OPS(rocket_device_runtime_suspend, rocket_device_runtime_resume, NULL) > + SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, pm_runtime_force_resume) > +}; > + > +static struct platform_driver rocket_driver = { > + .probe = rocket_probe, > + .remove_new = rocket_remove, > + .driver = { > + .name = "rocket", > + .pm = pm_ptr(&rocket_pm_ops), > + .of_match_table = dt_match, > + }, > +}; > +module_platform_driver(rocket_driver); > + > +MODULE_LICENSE("GPL v2"); "GPL v2" is invalid. Did you run checkpatch? > +MODULE_DESCRIPTION("DRM driver for the Rockchip NPU IP"); > +MODULE_AUTHOR("Tomeu Vizoso"); > diff --git a/drivers/accel/rocket/rocket_drv.h b/drivers/accel/rocket/rocket_drv.h > new file mode 100644 > index 000000000000..e1d9407c46d7 > --- /dev/null > +++ b/drivers/accel/rocket/rocket_drv.h > @@ -0,0 +1,13 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* Copyright 2024 Tomeu Vizoso <tomeu(a)tomeuvizoso.net> */ > + > +#ifndef __ROCKET_DRV_H__ > +#define __ROCKET_DRV_H__ > + > +#include <linux/io.h> Why? Doesn't look like this file uses this. However, no include for struct rocket_device either... > + > +struct rocket_file_priv { > + struct rocket_device *rdev; > +}; > + > +#endif > diff --git a/drivers/accel/rocket/rocket_registers.h b/drivers/accel/rocket/rocket_registers.h > new file mode 100644 > index 000000000000..9043fa922db5 > --- /dev/null > +++ b/drivers/accel/rocket/rocket_registers.h > @@ -0,0 +1,4449 @@ > +#ifndef REGISTERS_XML > +#define REGISTERS_XML > + > +/* Autogenerated file, DO NOT EDIT manually! > + > +This file was generated by the rules-ng-ng gen_header.py tool in this git repository: > +http://gitlab.freedesktop.org/mesa/mesa/ > +git clone https://gitlab.freedesktop.org/mesa/mesa.git > + > +The rules-ng-ng source files this header was generated from are: > + > +- /home/tomeu/src/mesa/src/gallium/drivers/rocket/registers.xml ( 60076 bytes, from Wed Jun 12 10:02:25 2024) > + > +Copyright (C) 2024-2024 by the following authors: Just checking, is the date range autogenerated? 2024-2024 is not correct per anything I've seen... > diff --git a/include/uapi/drm/rocket_accel.h b/include/uapi/drm/rocket_accel.h > new file mode 100644 > index 000000000000..7a179731ffe4 > --- /dev/null > +++ b/include/uapi/drm/rocket_accel.h > @@ -0,0 +1,19 @@ > +/* SPDX-License-Identifier: MIT */ > +/* > + * Copyright © 2024 Tomeu Vizoso Why the copyright symbol? > + */ > +#ifndef _ROCKET_DRM_H_ > +#define _ROCKET_DRM_H_ > + > +#include "drm.h" > + > +#if defined(__cplusplus) > +extern "C" { > +#endif > + > + > +#if defined(__cplusplus) > +} > +#endif > + > +#endif /* _ROCKET_DRM_H_ */ > Eh, feels like this could/should be moved later in the series since there is no content.

1 year, 8 months

2
1
0 0

Re: [PATCH 9/9] accel/rocket: Add IOCTLs for synchronizing memory accesses

by Jeffrey Hugo

On 6/12/2024 7:53 AM, Tomeu Vizoso wrote: > diff --git a/include/uapi/drm/rocket_accel.h b/include/uapi/drm/rocket_accel.h > index 888c9413e4cd..1539af0af4fe 100644 > --- a/include/uapi/drm/rocket_accel.h > +++ b/include/uapi/drm/rocket_accel.h > @@ -12,9 +12,13 @@ extern "C" { > #endif > > #define DRM_ROCKET_CREATE_BO 0x00 > -#define DRM_ROCKET_SUBMIT 0x01 > +#define DRM_ROCKET_PREP_BO 0x01 > +#define DRM_ROCKET_FINI_BO 0x02 > +#define DRM_ROCKET_SUBMIT 0x03 This looks like a uAPI breaking change. Shouldn't you have defined SUBMIT as 0x03 from the beginning, or put the new BO ioctls after it?

1 year, 8 months

1
0
0 0

Re: [PATCH 8/9] accel/rocket: Add job submission IOCTL

by Jeffrey Hugo

On 6/12/2024 7:53 AM, Tomeu Vizoso wrote: > Using the DRM GPU scheduler infrastructure, with a scheduler for each > core. > > Userspace can decide for a series of tasks to be executed sequentially > in the same core, so SRAM locality can be taken advantage of. > > The job submission code was intially based on Panfrost. intially -> initially > > Signed-off-by: Tomeu Vizoso <tomeu(a)tomeuvizoso.net> > --- > drivers/accel/rocket/Makefile | 3 +- > drivers/accel/rocket/rocket_core.c | 6 + > drivers/accel/rocket/rocket_core.h | 16 + > drivers/accel/rocket/rocket_device.c | 2 + > drivers/accel/rocket/rocket_device.h | 2 + > drivers/accel/rocket/rocket_drv.c | 15 + > drivers/accel/rocket/rocket_drv.h | 3 + > drivers/accel/rocket/rocket_job.c | 708 +++++++++++++++++++++++++++++++++++ > drivers/accel/rocket/rocket_job.h | 49 +++ > include/uapi/drm/rocket_accel.h | 55 +++ > 10 files changed, 858 insertions(+), 1 deletion(-) > > diff --git a/drivers/accel/rocket/Makefile b/drivers/accel/rocket/Makefile > index 875cac2243d9..4d59036af8d9 100644 > --- a/drivers/accel/rocket/Makefile > +++ b/drivers/accel/rocket/Makefile > @@ -6,4 +6,5 @@ rocket-y := \ > rocket_core.o \ > rocket_device.o \ > rocket_drv.o \ > - rocket_gem.o > + rocket_gem.o \ > + rocket_job.o > diff --git a/drivers/accel/rocket/rocket_core.c b/drivers/accel/rocket/rocket_core.c > index d6680b00fb2f..2b2d8be38f0a 100644 > --- a/drivers/accel/rocket/rocket_core.c > +++ b/drivers/accel/rocket/rocket_core.c > @@ -11,6 +11,7 @@ > > #include "rocket_core.h" > #include "rocket_device.h" > +#include "rocket_job.h" > #include "rocket_registers.h" > > static int rocket_clk_init(struct rocket_core *core) > @@ -122,6 +123,10 @@ int rocket_core_init(struct rocket_core *core) > goto out_pm_domain; > } > > + err = rocket_job_init(core); > + if (err) > + goto out_pm_domain; > + > version = rocket_read(core, REG_PC_VERSION) + (rocket_read(core, REG_PC_VERSION_NUM) & 0xffff); > dev_info(rdev->dev, "Rockchip NPU core %d version: %d\n", core->index, version); > > @@ -134,6 +139,7 @@ int rocket_core_init(struct rocket_core *core) > > void rocket_core_fini(struct rocket_core *core) > { > + rocket_job_fini(core); > rocket_pmdomain_fini(core); > } > > diff --git a/drivers/accel/rocket/rocket_core.h b/drivers/accel/rocket/rocket_core.h > index e5d4c848c9f4..e6401960a9b2 100644 > --- a/drivers/accel/rocket/rocket_core.h > +++ b/drivers/accel/rocket/rocket_core.h > @@ -8,6 +8,8 @@ > #include <asm/io.h> > #include <asm-generic/io.h> > > +#include <drm/gpu_scheduler.h> What about includes for workqueue or atomic? > +static struct dma_fence *rocket_job_run(struct drm_sched_job *sched_job) > +{ > + struct rocket_job *job = to_rocket_job(sched_job); > + struct rocket_device *rdev = job->rdev; > + struct rocket_core *core = sched_to_core(rdev, sched_job->sched); > + struct dma_fence *fence = NULL; > + int ret; > + > + if (unlikely(job->base.s_fence->finished.error)) > + return NULL; > + > + /* Nothing to execute: can happen if the job has finished while > + * we were resetting the GPU. > + */ Not the correct comment style > + if (job->next_task_idx == job->task_count) > + return NULL; > + > + fence = rocket_fence_create(core); > + if (IS_ERR(fence)) > + return fence; > + > + if (job->done_fence) > + dma_fence_put(job->done_fence); > + job->done_fence = dma_fence_get(fence); > + > + ret = pm_runtime_get_sync(rdev->dev); > + if (ret < 0) > + return fence; > + > + spin_lock(&core->job_lock); > + > + core->in_flight_job = job; > + rocket_job_hw_submit(core, job); > + > + spin_unlock(&core->job_lock); > + > + return fence; > +} > + > +static void rocket_job_handle_done(struct rocket_core *core, > + struct rocket_job *job) > +{ > + if (job->next_task_idx < job->task_count) { > + rocket_job_hw_submit(core, job); > + return; > + } > + > + core->in_flight_job = NULL; > + dma_fence_signal_locked(job->done_fence); > + pm_runtime_put_autosuspend(core->dev->dev); > +} > + > +static void rocket_job_handle_irq(struct rocket_core *core) > +{ > + uint32_t status, raw_status; > + > + pm_runtime_mark_last_busy(core->dev->dev); > + > + status = rocket_read(core, REG_PC_INTERRUPT_STATUS); > + raw_status = rocket_read(core, REG_PC_INTERRUPT_RAW_STATUS); > + > + rocket_write(core, REG_PC_OPERATION_ENABLE, 0x0); > + rocket_write(core, REG_PC_INTERRUPT_CLEAR, 0x1ffff); > + > + spin_lock(&core->job_lock); > + > + if (core->in_flight_job) > + rocket_job_handle_done(core, core->in_flight_job); > + > + spin_unlock(&core->job_lock); > +} > + > +static void > +rocket_reset(struct rocket_core *core, struct drm_sched_job *bad) > +{ > + struct rocket_device *rdev = core->dev; > + bool cookie; > + > + if (!atomic_read(&core->reset.pending)) > + return; > + > + /* Stop the scheduler. Not the correct comment style > + * > + * FIXME: We temporarily get out of the dma_fence_signalling section > + * because the cleanup path generate lockdep splats when taking locks > + * to release job resources. We should rework the code to follow this > + * pattern: > + * > + * try_lock > + * if (locked) > + * release > + * else > + * schedule_work_to_release_later > + */ > + drm_sched_stop(&core->sched, bad); > + > + cookie = dma_fence_begin_signalling(); > + > + if (bad) > + drm_sched_increase_karma(bad); > + > + /* Mask job interrupts and synchronize to make sure we won't be > + * interrupted during our reset. > + */ Not the correct comment style, again. This is the last time I'm going to mention it. > diff --git a/drivers/accel/rocket/rocket_job.h b/drivers/accel/rocket/rocket_job.h > new file mode 100644 > index 000000000000..0c3c90e47d39 > --- /dev/null > +++ b/drivers/accel/rocket/rocket_job.h > @@ -0,0 +1,49 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* Copyright 2024 Tomeu Vizoso <tomeu(a)tomeuvizoso.net> */ > + > +#ifndef __ROCKET_JOB_H__ > +#define __ROCKET_JOB_H__ > + > +#include <drm/gpu_scheduler.h> > +#include <drm/drm_drv.h> Alphabetical order > diff --git a/include/uapi/drm/rocket_accel.h b/include/uapi/drm/rocket_accel.h > index 8338726a83c3..888c9413e4cd 100644 > --- a/include/uapi/drm/rocket_accel.h > +++ b/include/uapi/drm/rocket_accel.h > @@ -12,8 +12,10 @@ extern "C" { > #endif > > #define DRM_ROCKET_CREATE_BO 0x00 > +#define DRM_ROCKET_SUBMIT 0x01 > > #define DRM_IOCTL_ROCKET_CREATE_BO DRM_IOWR(DRM_COMMAND_BASE + DRM_ROCKET_CREATE_BO, struct drm_rocket_create_bo) > +#define DRM_IOCTL_ROCKET_SUBMIT DRM_IOW(DRM_COMMAND_BASE + DRM_ROCKET_SUBMIT, struct drm_rocket_submit) > > /** > * struct drm_rocket_create_bo - ioctl argument for creating Rocket BOs. > @@ -36,6 +38,59 @@ struct drm_rocket_create_bo { > __u64 offset; > }; > > +/** > + * struct drm_rocket_task - A task to be run on the NPU > + * > + * A task is the smallest unit of work that can be run on the NPU. > + */ > +struct drm_rocket_task { > + /** DMA address to NPU mapping of register command buffer */ > + __u64 regcmd; > + > + /** Number of commands in the register command buffer */ > + __u32 regcmd_count; > +}; > + > +/** > + * struct drm_rocket_job - A job to be run on the NPU > + * > + * The kernel will schedule the execution of this job taking into account its > + * dependencies with other jobs. All tasks in the same job will be executed > + * sequentially on the same core, to benefit from memory residency in SRAM. > + */ > +struct drm_rocket_job { > + /** Pointer to an array of struct drm_rocket_task. */ > + __u64 tasks; > + > + /** Number of tasks passed in. */ > + __u32 task_count; > + > + /** Pointer to a u32 array of the BOs that are read by the job. */ > + __u64 in_bo_handles; > + > + /** Number of input BO handles passed in (size is that times 4). */ > + __u32 in_bo_handle_count; > + > + /** Pointer to a u32 array of the BOs that are written to by the job. */ > + __u64 out_bo_handles; > + > + /** Number of output BO handles passed in (size is that times 4). */ > + __u32 out_bo_handle_count; > +}; I feels like the mixing of 32-bit and 64-bit fields violates the guidelines on defining ioctls due to implicit padding that might or might not be present. > + > +/** > + * struct drm_rocket_submit - ioctl argument for submitting commands to the NPU. > + * > + * The kernel will schedule the execution of these jobs in dependency order. > + */ > +struct drm_rocket_submit { > + /** Pointer to an array of struct drm_rocket_job. */ > + __u64 jobs; > + > + /** Number of jobs passed in. */ > + __u32 job_count; > +}; > + > #if defined(__cplusplus) > } > #endif >

1 year, 8 months

1
0
0 0

Re: [PATCH 7/9] accel/rocket: Add IOCTL for BO creation

by Jeffrey Hugo

On 6/12/2024 7:53 AM, Tomeu Vizoso wrote: > This uses the SHMEM DRM helpers and we map right away to the CPU and NPU > sides, as all buffers are expected to be accessed from both. > > Signed-off-by: Tomeu Vizoso <tomeu(a)tomeuvizoso.net> Reviewed-by: Jeffrey Hugo <quic_jhugo(a)quicinc.com>

1 year, 8 months

1
0
0 0

Re: [PATCH 2/9] iommu/rockchip: Attach multiple power domains

by Robin Murphy

On 2024-06-13 10:38 pm, Sebastian Reichel wrote: > Hi, > > On Thu, Jun 13, 2024 at 11:34:02AM GMT, Tomeu Vizoso wrote: >> On Thu, Jun 13, 2024 at 11:24 AM Tomeu Vizoso <tomeu(a)tomeuvizoso.net> wrote: >>> On Thu, Jun 13, 2024 at 2:05 AM Sebastian Reichel >>> <sebastian.reichel(a)collabora.com> wrote: >>>> On Wed, Jun 12, 2024 at 03:52:55PM GMT, Tomeu Vizoso wrote: >>>>> IOMMUs with multiple base addresses can also have multiple power >>>>> domains. >>>>> >>>>> The base framework only takes care of a single power domain, as some >>>>> devices will need for these power domains to be powered on in a specific >>>>> order. >>>>> >>>>> Use a helper function to stablish links in the order in which they are >>>>> in the DT. >>>>> >>>>> This is needed by the IOMMU used by the NPU in the RK3588. >>>>> >>>>> Signed-off-by: Tomeu Vizoso <tomeu(a)tomeuvizoso.net> >>>>> --- >>>> >>>> To me it looks like this is multiple IOMMUs, which should each get >>>> their own node. I don't see a good reason for merging these >>>> together. >>> >>> I have made quite a few attempts at splitting the IOMMUs and also the >>> cores, but I wasn't able to get things working stably. The TRM is >>> really scant about how the 4 IOMMU instances relate to each other, and >>> what the fourth one is for. >>> >>> Given that the vendor driver treats them as a single IOMMU with four >>> instances and we don't have any information on them, I resigned myself >>> to just have them as a single device. >>> >>> I would love to be proved wrong though and find a way fo getting >>> things stably as different devices so they can be powered on and off >>> as needed. We could save quite some code as well. >> >> FWIW, here a few ways how I tried to structure the DT nodes, none of >> these worked reliably: >> >> https://gitlab.freedesktop.org/tomeu/linux/-/blob/6.10-rocket-multiple-devi… >> https://gitlab.freedesktop.org/tomeu/linux/-/blob/6.10-rocket-schema-subnod… >> https://gitlab.freedesktop.org/tomeu/linux/-/blob/6.10-rocket-multiple-devi… >> https://gitlab.freedesktop.org/tomeu/linux/-/blob/6.10-rocket-multiple-iomm… >> >> I can very well imagine I missed some way of getting this to work, but >> for every attempt, the domains, iommus and cores were resumed in >> different orders that presumably caused problems during concurrent >> execution fo workloads. >> >> So I fell back to what the vendor driver does, which works reliably >> (but all cores have to be powered on at the same time). > > Mh. The "6.10-rocket-multiple-iommus" branch seems wrong. There is > only one iommu node in that. I would have expected a test with > > rknn { > // combined device > > iommus = <&iommu1>, <&iommu2>, ...; > }; > > Otherwise I think I would go with the schema-subnodes variant. The > driver can initially walk through the sub-nodes and collect the > resources into the main device, so on the driver side nothing would > really change. But that has a couple of advantages: > > 1. DT and DT binding are easier to read > 2. It's similar to e.g. CPU cores each having their own node > 3. Easy to extend to more cores in the future > 4. The kernel can easily switch to proper per-core device model when > the problem has been identified It also would seem to permit describing and associating the per-core IOMMUs individually - apart from core 0's apparent coupling to whatever shared "uncore" stuff exists for the whole thing, from the distinct clocks, interrupts, power domains etc. lining up with each core I'd guess those IOMMUs are not interrelated the same way the ISP's read/write IOMMUs are (which was the main justification for adopting the multiple-reg design originally vs. distinct DT nodes like Exynos does). However, practically that would require the driver to at least populate per-core child devices to make DMA API or IOMMU API mappings with, since we couldn't spread the "collect the resources" trick into those subsystems as well. Thanks, Robin.

1 year, 8 months

1
0
0 0

Re: [PATCH RESEND v5 00/16] Add audio support for the MediaTek Genio 350-evk board

by Mark Brown

On Fri, Jun 14, 2024 at 09:27:43AM +0200, Alexandre Mergnat wrote: > This serie aim to add the following audio support for the Genio 350-evk: > - Playback > - 2ch Headset Jack (Earphone) > - 1ch Line-out Jack (Speaker) > - 8ch HDMI Tx I seem to remember you had review comments that needed addressing from AngeloGioacchino, why resend without addressing those?

1 year, 8 months

1
0
0 0

Re: [PATCH 3/9] dt-bindings: mailbox: rockchip,rknn: Add bindings

by Rob Herring

On Wed, Jun 12, 2024 at 03:52:56PM +0200, Tomeu Vizoso wrote: > Add the bindings for the Neural Processing Unit IP from Rockchip. Subject is wrong. Not a mailbox... > Signed-off-by: Tomeu Vizoso <tomeu(a)tomeuvizoso.net> > --- > .../devicetree/bindings/npu/rockchip,rknn.yaml | 123 +++++++++++++++++++++ > 1 file changed, 123 insertions(+) > > diff --git a/Documentation/devicetree/bindings/npu/rockchip,rknn.yaml b/Documentation/devicetree/bindings/npu/rockchip,rknn.yaml > new file mode 100644 > index 000000000000..570a4889c11c > --- /dev/null > +++ b/Documentation/devicetree/bindings/npu/rockchip,rknn.yaml > @@ -0,0 +1,123 @@ > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) > +%YAML 1.2 > +--- > +$id: http://devicetree.org/schemas/npu/rockchip,rknn.yaml# > +$schema: http://devicetree.org/meta-schemas/core.yaml# > + > +title: Neural Processing Unit IP from Rockchip, based on NVIDIA's NVDLA > + > +maintainers: > + - Tomeu Vizoso <tomeu(a)tomeuvizoso.net> > + > +description: |+ > + Rockchip IP for accelerating inference of neural networks, based on NVIDIA's open source NVDLA IP. Wrap at 80. > + > +properties: > + compatible: > + items: > + - enum: > + - rockchip,rk3588-rknn > + - const: rockchip,rknn Is there any evidence this block is 'the same' on multiple chips? > + > + reg: > + description: Base registers for NPU cores > + minItems: 1 > + maxItems: 20 > + > + interrupts: > + minItems: 1 > + maxItems: 20 > + > + interrupt-names: > + minItems: 1 > + maxItems: 20 > + > + clocks: > + minItems: 1 > + maxItems: 20 > + > + clock-names: > + minItems: 1 > + maxItems: 20 > + > + assigned-clocks: > + maxItems: 1 > + > + assigned-clock-rates: > + maxItems: 1 You don't need assigned-clocks in schemas. > + > + resets: > + minItems: 1 > + maxItems: 20 > + > + reset-names: > + minItems: 1 > + maxItems: 20 > + > + power-domains: > + minItems: 1 > + maxItems: 20 > + > + power-domain-names: > + minItems: 1 > + maxItems: 20 > + > + iommus: > + items: > + - description: IOMMU for all cores > + > +required: > + - compatible > + - reg > + - interrupts > + - interrupt-names > + - clocks > + - clock-names > + - assigned-clocks > + - assigned-clock-rates And never should be required. > + - resets > + - reset-names > + - power-domains > + - power-domain-names > + - iommus > + > +additionalProperties: false > + > +examples: > + - | > + #include <dt-bindings/interrupt-controller/arm-gic.h> > + > + bus { > + #address-cells = <2>; > + #size-cells = <2>; > + > + rknn: npu@fdab0000 { > + compatible = "rockchip,rk3588-rknn", "rockchip,rknn"; > + reg = <0x0 0xfdab0000 0x0 0x9000>, > + <0x0 0xfdac0000 0x0 0x9000>, > + <0x0 0xfdad0000 0x0 0x9000>; > + interrupts = <GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH 0>, > + <GIC_SPI 111 IRQ_TYPE_LEVEL_HIGH 0>, > + <GIC_SPI 112 IRQ_TYPE_LEVEL_HIGH 0>; > + interrupt-names = "npu0_irq", "npu1_irq", "npu2_irq"; 'irq' is redundant. Names with the index are also kind of pointless unless they can be not contiguous. > + clocks = <&scmi_clk 0>, <&cru 1>, > + <&cru 2>, <&cru 3>, > + <&cru 4>, <&cru 5>, > + <&cru 6>, <&cru 7>; > + clock-names = "clk_npu", 'clk_' is redundant. > + "aclk0", "aclk1", "aclk2", > + "hclk0", "hclk1", "hclk2", > + "pclk"; Assuming 0, 1, 2 are cores and may vary, put all the fixed clocks first and then better to do "aclk0", "hclk0", "aclk1", "hclk1",... > + assigned-clocks = <&scmi_clk 0>; > + assigned-clock-rates = <200000000>; > + resets = <&cru 0>, <&cru 1>, <&cru 2>, > + <&cru 3>, <&cru 4>, <&cru 5>; > + reset-names = "srst_a0", "srst_a1", "srst_a2", > + "srst_h0", "srst_h1", "srst_h2"; And similar order here. > + power-domains = <&power 0>, <&power 1>, <&power 2>; > + power-domain-names = "npu0", "npu1", "npu2"; > + iommus = <&rknpu_mmu>; > + status = "disabled"; > + }; > + }; > +... > > -- > 2.45.2 >

1 year, 8 months

1
0
0 0

Re: [PATCH 0/9] New DRM accel driver for Rockchip's RKNN NPU

by Rob Herring (Arm)

On Wed, 12 Jun 2024 15:52:53 +0200, Tomeu Vizoso wrote: > This series adds a new driver for the NPU that Rockchip includes in its > newer SoCs, developed by them on the NVDLA base. > > In its current form, it supports the specific NPU in the RK3588 SoC. > > The userspace driver is part of Mesa and an initial draft can be found at: > > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29698 > > Signed-off-by: Tomeu Vizoso <tomeu(a)tomeuvizoso.net> > --- > Tomeu Vizoso (9): > iommu/rockchip: Add compatible for rockchip,rk3588-iommu > iommu/rockchip: Attach multiple power domains > dt-bindings: mailbox: rockchip,rknn: Add bindings > arm64: dts: rockchip: Add nodes for NPU and its MMU to rk3588s > arm64: dts: rockchip: Enable the NPU on quartzpro64 > accel/rocket: Add a new driver for Rockchip's NPU > accel/rocket: Add IOCTL for BO creation > accel/rocket: Add job submission IOCTL > accel/rocket: Add IOCTLs for synchronizing memory accesses > > .../devicetree/bindings/npu/rockchip,rknn.yaml | 123 + > MAINTAINERS | 8 + > .../arm64/boot/dts/rockchip/rk3588-quartzpro64.dts | 8 + > arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 53 + > drivers/accel/Kconfig | 1 + > drivers/accel/Makefile | 1 + > drivers/accel/rocket/Kconfig | 13 + > drivers/accel/rocket/Makefile | 10 + > drivers/accel/rocket/rocket_core.c | 155 + > drivers/accel/rocket/rocket_core.h | 48 + > drivers/accel/rocket/rocket_device.c | 39 + > drivers/accel/rocket/rocket_device.h | 40 + > drivers/accel/rocket/rocket_drv.c | 243 ++ > drivers/accel/rocket/rocket_drv.h | 16 + > drivers/accel/rocket/rocket_gem.c | 136 + > drivers/accel/rocket/rocket_gem.h | 33 + > drivers/accel/rocket/rocket_job.c | 708 ++++ > drivers/accel/rocket/rocket_job.h | 49 + > drivers/accel/rocket/rocket_registers.h | 4449 ++++++++++++++++++++ > drivers/iommu/rockchip-iommu.c | 39 + > include/uapi/drm/rocket_accel.h | 116 + > 21 files changed, 6288 insertions(+) > --- > base-commit: 83a7eefedc9b56fe7bfeff13b6c7356688ffa670 > change-id: 20240612-6-10-rocket-9316defc14c7 > > Best regards, > -- > Tomeu Vizoso <tomeu(a)tomeuvizoso.net> > > > My bot found new DTB warnings on the .dts files added or changed in this series. Some warnings may be from an existing SoC .dtsi. Or perhaps the warnings are fixed by another series. Ultimately, it is up to the platform maintainer whether these warnings are acceptable or not. No need to reply unless the platform maintainer has comments. If you already ran DT checks and didn't see these error(s), then make sure dt-schema is up to date: pip3 install dtschema --upgrade New warnings running 'make CHECK_DTBS=y rockchip/rk3588-quartzpro64.dtb' for 20240612-6-10-rocket-v1-0-060e48eea250(a)tomeuvizoso.net: arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: compatible: 'oneOf' conditional failed, one must be fixed: ['rockchip,rk3588-iommu'] is too short 'rockchip,rk3588-iommu' is not one of ['rockchip,iommu', 'rockchip,rk3568-iommu'] from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml# arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: reg: [[0, 4255879168, 0, 256], [0, 4255883264, 0, 256], [0, 4255948800, 0, 256], [0, 4256014336, 0, 256]] is too long from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml# arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: interrupts: [[0, 110, 4, 0], [0, 111, 4, 0], [0, 112, 4, 0]] is too long from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml# arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: clocks: [[28, 287], [28, 276], [28, 278], [28, 288], [28, 277], [28, 279]] is too long from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml# arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: clock-names:0: 'aclk' was expected from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml# arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: clock-names:1: 'iface' was expected from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml# arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: clock-names: ['aclk0', 'aclk1', 'aclk2', 'iface0', 'iface1', 'iface2'] is too long from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml# arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: power-domains: [[30, 9], [30, 10], [30, 11]] is too long from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml# arch/arm64/boot/dts/rockchip/rk3588-quartzpro64.dtb: iommu@fdab9000: 'interrupt-names', 'power-domain-names' do not match any of the regexes: 'pinctrl-[0-9]+' from schema $id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml#

1 year, 8 months

1
0
0 0

Re: [PATCH 6/9] accel/rocket: Add a new driver for Rockchip's NPU

by kernel test robot

Hi Tomeu, kernel test robot noticed the following build warnings: [auto build test WARNING on 83a7eefedc9b56fe7bfeff13b6c7356688ffa670] url: https://github.com/intel-lab-lkp/linux/commits/Tomeu-Vizoso/iommu-rockchip-… base: 83a7eefedc9b56fe7bfeff13b6c7356688ffa670 patch link: https://lore.kernel.org/r/20240612-6-10-rocket-v1-6-060e48eea250%40tomeuviz… patch subject: [PATCH 6/9] accel/rocket: Add a new driver for Rockchip's NPU config: arc-kismet-CONFIG_IOMMU_IO_PGTABLE_LPAE-CONFIG_DRM_ACCEL_ROCKET-0-0 (https://download.01.org/0day-ci/archive/20240613/202406131802.9chtX0Ci-lkp@…) reproduce: (https://download.01.org/0day-ci/archive/20240613/202406131802.9chtX0Ci-lkp@…) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp(a)intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202406131802.9chtX0Ci-lkp@intel.com/ kismet warnings: (new ones prefixed by >>) >> kismet: WARNING: unmet direct dependencies detected for IOMMU_IO_PGTABLE_LPAE when selected by DRM_ACCEL_ROCKET WARNING: unmet direct dependencies detected for IOMMU_IO_PGTABLE_LPAE Depends on [n]: IOMMU_SUPPORT [=y] && (ARM || ARM64 || COMPILE_TEST [=y]) && !GENERIC_ATOMIC64 [=y] Selected by [y]: - DRM_ACCEL_ROCKET [=y] && DRM [=y] && (ARM64 || COMPILE_TEST [=y]) && MMU [=y] -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 year, 8 months

1
0
0 0

Re: [PATCH 8/9] accel/rocket: Add job submission IOCTL

by kernel test robot

Hi Tomeu, kernel test robot noticed the following build warnings: [auto build test WARNING on 83a7eefedc9b56fe7bfeff13b6c7356688ffa670] url: https://github.com/intel-lab-lkp/linux/commits/Tomeu-Vizoso/iommu-rockchip-… base: 83a7eefedc9b56fe7bfeff13b6c7356688ffa670 patch link: https://lore.kernel.org/r/20240612-6-10-rocket-v1-8-060e48eea250%40tomeuviz… patch subject: [PATCH 8/9] accel/rocket: Add job submission IOCTL config: hexagon-allmodconfig (https://download.01.org/0day-ci/archive/20240613/202406131640.WbBaRMbr-lkp@…) compiler: clang version 19.0.0git (https://github.com/llvm/llvm-project 4403cdbaf01379de96f8d0d6ea4f51a085e37766) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240613/202406131640.WbBaRMbr-lkp@…) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp(a)intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202406131640.WbBaRMbr-lkp@intel.com/ All warnings (new ones prefixed by >>): In file included from drivers/accel/rocket/rocket_job.c:6: In file included from include/drm/drm_file.h:39: In file included from include/drm/drm_prime.h:37: In file included from include/linux/scatterlist.h:8: In file included from include/linux/mm.h:2253: include/linux/vmstat.h:514:36: warning: arithmetic between different enumeration types ('enum node_stat_item' and 'enum lru_list') [-Wenum-enum-conversion] 514 | return node_stat_name(NR_LRU_BASE + lru) + 3; // skip "nr_" | ~~~~~~~~~~~ ^ ~~~ In file included from drivers/accel/rocket/rocket_job.c:6: In file included from include/drm/drm_file.h:39: In file included from include/drm/drm_prime.h:37: In file included from include/linux/scatterlist.h:9: In file included from arch/hexagon/include/asm/io.h:328: include/asm-generic/io.h:548:31: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] 548 | val = __raw_readb(PCI_IOBASE + addr); | ~~~~~~~~~~ ^ include/asm-generic/io.h:561:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] 561 | val = __le16_to_cpu((__le16 __force)__raw_readw(PCI_IOBASE + addr)); | ~~~~~~~~~~ ^ include/uapi/linux/byteorder/little_endian.h:37:51: note: expanded from macro '__le16_to_cpu' 37 | #define __le16_to_cpu(x) ((__force __u16)(__le16)(x)) | ^ In file included from drivers/accel/rocket/rocket_job.c:6: In file included from include/drm/drm_file.h:39: In file included from include/drm/drm_prime.h:37: In file included from include/linux/scatterlist.h:9: In file included from arch/hexagon/include/asm/io.h:328: include/asm-generic/io.h:574:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] 574 | val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr)); | ~~~~~~~~~~ ^ include/uapi/linux/byteorder/little_endian.h:35:51: note: expanded from macro '__le32_to_cpu' 35 | #define __le32_to_cpu(x) ((__force __u32)(__le32)(x)) | ^ In file included from drivers/accel/rocket/rocket_job.c:6: In file included from include/drm/drm_file.h:39: In file included from include/drm/drm_prime.h:37: In file included from include/linux/scatterlist.h:9: In file included from arch/hexagon/include/asm/io.h:328: include/asm-generic/io.h:585:33: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] 585 | __raw_writeb(value, PCI_IOBASE + addr); | ~~~~~~~~~~ ^ include/asm-generic/io.h:595:59: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] 595 | __raw_writew((u16 __force)cpu_to_le16(value), PCI_IOBASE + addr); | ~~~~~~~~~~ ^ include/asm-generic/io.h:605:59: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] 605 | __raw_writel((u32 __force)cpu_to_le32(value), PCI_IOBASE + addr); | ~~~~~~~~~~ ^ >> drivers/accel/rocket/rocket_job.c:353:11: warning: variable 'status' set but not used [-Wunused-but-set-variable] 353 | uint32_t status, raw_status; | ^ >> drivers/accel/rocket/rocket_job.c:353:19: warning: variable 'raw_status' set but not used [-Wunused-but-set-variable] 353 | uint32_t status, raw_status; | ^ drivers/accel/rocket/rocket_job.c:40:1: warning: unused function 'to_rocket_fence' [-Wunused-function] 40 | to_rocket_fence(struct dma_fence *fence) | ^~~~~~~~~~~~~~~ 10 warnings generated. Kconfig warnings: (for reference only) WARNING: unmet direct dependencies detected for IOMMU_IO_PGTABLE_LPAE Depends on [n]: IOMMU_SUPPORT [=y] && (ARM || ARM64 || COMPILE_TEST [=y]) && !GENERIC_ATOMIC64 [=y] Selected by [m]: - DRM_ACCEL_ROCKET [=m] && DRM [=m] && (ARM64 || COMPILE_TEST [=y]) && MMU [=y] vim +/status +353 drivers/accel/rocket/rocket_job.c 350 351 static void rocket_job_handle_irq(struct rocket_core *core) 352 { > 353 uint32_t status, raw_status; 354 355 pm_runtime_mark_last_busy(core->dev->dev); 356 357 status = rocket_read(core, REG_PC_INTERRUPT_STATUS); 358 raw_status = rocket_read(core, REG_PC_INTERRUPT_RAW_STATUS); 359 360 rocket_write(core, REG_PC_OPERATION_ENABLE, 0x0); 361 rocket_write(core, REG_PC_INTERRUPT_CLEAR, 0x1ffff); 362 363 spin_lock(&core->job_lock); 364 365 if (core->in_flight_job) 366 rocket_job_handle_done(core, core->in_flight_job); 367 368 spin_unlock(&core->job_lock); 369 } 370 -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 year, 8 months

1
0
0 0

Re: [PATCH 6/9] accel/rocket: Add a new driver for Rockchip's NPU

by kernel test robot

Hi Tomeu, kernel test robot noticed the following build errors: [auto build test ERROR on 83a7eefedc9b56fe7bfeff13b6c7356688ffa670] url: https://github.com/intel-lab-lkp/linux/commits/Tomeu-Vizoso/iommu-rockchip-… base: 83a7eefedc9b56fe7bfeff13b6c7356688ffa670 patch link: https://lore.kernel.org/r/20240612-6-10-rocket-v1-6-060e48eea250%40tomeuviz… patch subject: [PATCH 6/9] accel/rocket: Add a new driver for Rockchip's NPU config: s390-allyesconfig (https://download.01.org/0day-ci/archive/20240613/202406131022.1JKNS7me-lkp@…) compiler: s390-linux-gcc (GCC) 13.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240613/202406131022.1JKNS7me-lkp@…) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp(a)intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202406131022.1JKNS7me-lkp@intel.com/ All errors (new ones prefixed by >>): In file included from include/linux/clk.h:13, from drivers/accel/rocket/rocket_drv.c:4: >> drivers/accel/rocket/rocket_drv.c:213:31: error: 'rocket_pm_ops' undeclared here (not in a function); did you mean 'rocket_probe'? 213 | .pm = pm_ptr(&rocket_pm_ops), | ^~~~~~~~~~~~~ include/linux/kernel.h:48:44: note: in definition of macro 'PTR_IF' 48 | #define PTR_IF(cond, ptr) ((cond) ? (ptr) : NULL) | ^~~ drivers/accel/rocket/rocket_drv.c:213:23: note: in expansion of macro 'pm_ptr' 213 | .pm = pm_ptr(&rocket_pm_ops), | ^~~~~~ vim +213 drivers/accel/rocket/rocket_drv.c 207 208 static struct platform_driver rocket_driver = { 209 .probe = rocket_probe, 210 .remove_new = rocket_remove, 211 .driver = { 212 .name = "rocket", > 213 .pm = pm_ptr(&rocket_pm_ops), 214 .of_match_table = dt_match, 215 }, 216 }; 217 module_platform_driver(rocket_driver); 218 -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 year, 8 months

1
0
0 0

Re: [PATCH 6/9] accel/rocket: Add a new driver for Rockchip's NPU

by kernel test robot

Hi Tomeu, kernel test robot noticed the following build errors: [auto build test ERROR on 83a7eefedc9b56fe7bfeff13b6c7356688ffa670] url: https://github.com/intel-lab-lkp/linux/commits/Tomeu-Vizoso/iommu-rockchip-… base: 83a7eefedc9b56fe7bfeff13b6c7356688ffa670 patch link: https://lore.kernel.org/r/20240612-6-10-rocket-v1-6-060e48eea250%40tomeuviz… patch subject: [PATCH 6/9] accel/rocket: Add a new driver for Rockchip's NPU config: loongarch-allmodconfig (https://download.01.org/0day-ci/archive/20240613/202406130901.oiofrkFe-lkp@…) compiler: loongarch64-linux-gcc (GCC) 13.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240613/202406130901.oiofrkFe-lkp@…) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp(a)intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202406130901.oiofrkFe-lkp@intel.com/ All errors (new ones prefixed by >>): In file included from arch/loongarch/include/asm/processor.h:17, from arch/loongarch/include/asm/thread_info.h:15, from include/linux/thread_info.h:60, from include/asm-generic/current.h:6, from ./arch/loongarch/include/generated/asm/current.h:1, from include/linux/mutex.h:14, from include/linux/notifier.h:14, from include/linux/clk.h:14, from drivers/accel/rocket/rocket_core.c:6: >> arch/loongarch/include/uapi/asm/ptrace.h:25:25: error: expected identifier before '(' token 25 | #define PC (GPR_END + 2) | ^ drivers/accel/rocket/rocket_registers.h:53:9: note: in expansion of macro 'PC' 53 | PC = 0x00000100, | ^~ vim +25 arch/loongarch/include/uapi/asm/ptrace.h 803b0fc5c3f2ba Huacai Chen 2022-05-31 16 803b0fc5c3f2ba Huacai Chen 2022-05-31 17 /* 803b0fc5c3f2ba Huacai Chen 2022-05-31 18 * For PTRACE_{POKE,PEEK}USR. 0 - 31 are GPRs, 803b0fc5c3f2ba Huacai Chen 2022-05-31 19 * 32 is syscall's original ARG0, 33 is PC, 34 is BADVADDR. 803b0fc5c3f2ba Huacai Chen 2022-05-31 20 */ 803b0fc5c3f2ba Huacai Chen 2022-05-31 21 #define GPR_BASE 0 803b0fc5c3f2ba Huacai Chen 2022-05-31 22 #define GPR_NUM 32 803b0fc5c3f2ba Huacai Chen 2022-05-31 23 #define GPR_END (GPR_BASE + GPR_NUM - 1) 803b0fc5c3f2ba Huacai Chen 2022-05-31 24 #define ARG0 (GPR_END + 1) 803b0fc5c3f2ba Huacai Chen 2022-05-31 @25 #define PC (GPR_END + 2) 803b0fc5c3f2ba Huacai Chen 2022-05-31 26 #define BADVADDR (GPR_END + 3) 803b0fc5c3f2ba Huacai Chen 2022-05-31 27 -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 year, 8 months

1
0
0 0

Re: [PATCH 3/9] dt-bindings: mailbox: rockchip,rknn: Add bindings

by Conor Dooley

On Wed, Jun 12, 2024 at 03:52:56PM +0200, Tomeu Vizoso wrote: > Add the bindings for the Neural Processing Unit IP from Rockchip. > > Signed-off-by: Tomeu Vizoso <tomeu(a)tomeuvizoso.net> > --- > .../devicetree/bindings/npu/rockchip,rknn.yaml | 123 +++++++++++++++++++++ > 1 file changed, 123 insertions(+) > > diff --git a/Documentation/devicetree/bindings/npu/rockchip,rknn.yaml b/Documentation/devicetree/bindings/npu/rockchip,rknn.yaml > new file mode 100644 > index 000000000000..570a4889c11c > --- /dev/null > +++ b/Documentation/devicetree/bindings/npu/rockchip,rknn.yaml > @@ -0,0 +1,123 @@ > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) > +%YAML 1.2 > +--- > +$id: http://devicetree.org/schemas/npu/rockchip,rknn.yaml# > +$schema: http://devicetree.org/meta-schemas/core.yaml# > + > +title: Neural Processing Unit IP from Rockchip, based on NVIDIA's NVDLA > + > +maintainers: > + - Tomeu Vizoso <tomeu(a)tomeuvizoso.net> > + > +description: |+ The |+ chomping operator is not needed here. > + Rockchip IP for accelerating inference of neural networks, based on NVIDIA's open source NVDLA IP. > + > +properties: > + compatible: > + items: > + - enum: > + - rockchip,rk3588-rknn > + - const: rockchip,rknn > + > + reg: > + description: Base registers for NPU cores > + minItems: 1 > + maxItems: 20 For all of these properties, you need to describe the individual items. > + > + interrupts: > + minItems: 1 > + maxItems: 20 > + > + interrupt-names: > + minItems: 1 > + maxItems: 20 > + > + clocks: > + minItems: 1 > + maxItems: 20 > + > + clock-names: > + minItems: 1 > + maxItems: 20 > + > + assigned-clocks: > + maxItems: 1 > + > + assigned-clock-rates: > + maxItems: 1 > + > + resets: > + minItems: 1 > + maxItems: 20 > + > + reset-names: > + minItems: 1 > + maxItems: 20 > + > + power-domains: > + minItems: 1 > + maxItems: 20 > + > + power-domain-names: > + minItems: 1 > + maxItems: 20 > + > + iommus: > + items: > + - description: IOMMU for all cores > + > +required: > + - compatible > + - reg > + - interrupts > + - interrupt-names > + - clocks > + - clock-names > + - assigned-clocks > + - assigned-clock-rates > + - resets > + - reset-names > + - power-domains > + - power-domain-names > + - iommus > + > +additionalProperties: false > + > +examples: > + - | > + #include <dt-bindings/interrupt-controller/arm-gic.h> > + > + bus { > + #address-cells = <2>; > + #size-cells = <2>; > + > + rknn: npu@fdab0000 { Drop the label here, it's not used. > + compatible = "rockchip,rk3588-rknn", "rockchip,rknn"; > + reg = <0x0 0xfdab0000 0x0 0x9000>, > + <0x0 0xfdac0000 0x0 0x9000>, > + <0x0 0xfdad0000 0x0 0x9000>; > + interrupts = <GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH 0>, > + <GIC_SPI 111 IRQ_TYPE_LEVEL_HIGH 0>, > + <GIC_SPI 112 IRQ_TYPE_LEVEL_HIGH 0>; > + interrupt-names = "npu0_irq", "npu1_irq", "npu2_irq"; > + clocks = <&scmi_clk 0>, <&cru 1>, > + <&cru 2>, <&cru 3>, > + <&cru 4>, <&cru 5>, > + <&cru 6>, <&cru 7>; > + clock-names = "clk_npu", > + "aclk0", "aclk1", "aclk2", > + "hclk0", "hclk1", "hclk2", > + "pclk"; > + assigned-clocks = <&scmi_clk 0>; > + assigned-clock-rates = <200000000>; > + resets = <&cru 0>, <&cru 1>, <&cru 2>, > + <&cru 3>, <&cru 4>, <&cru 5>; > + reset-names = "srst_a0", "srst_a1", "srst_a2", > + "srst_h0", "srst_h1", "srst_h2"; > + power-domains = <&power 0>, <&power 1>, <&power 2>; > + power-domain-names = "npu0", "npu1", "npu2"; > + iommus = <&rknpu_mmu>; > + status = "disabled"; A disabled example is useless. > + }; > + }; > +... > > -- > 2.45.2 >

1 year, 8 months

1
0
0 0

Re: [PATCH] dma-buf/heaps: Correct the types of fd_flags and heap_flags

by John Stultz

On Wed, Jun 5, 2024 at 7:02 PM Barry Song <21cnbao(a)gmail.com> wrote: > > From: Barry Song <v-songbaohua(a)oppo.com> > > dma_heap_allocation_data defines the UAPI as follows: > > struct dma_heap_allocation_data { > __u64 len; > __u32 fd; > __u32 fd_flags; > __u64 heap_flags; > }; > > But dma heaps are casting both fd_flags and heap_flags into > unsigned long. This patch makes dma heaps - cma heap and > system heap have consistent types with UAPI. > > Signed-off-by: Barry Song <v-songbaohua(a)oppo.com> Thanks for submitting this additional cleanup! Acked-by: John Stultz <jstultz(a)google.com>

1 year, 8 months

1
0
0 0

Re: [PATCH v2] dma-buf: align fd_flags and heap_flags with dma_heap_allocation_data

by Sumit Semwal

Hello Barry, On Wed, 5 Jun 2024 at 06:56, Barry Song <21cnbao(a)gmail.com> wrote: > > From: Barry Song <v-songbaohua(a)oppo.com> > > dma_heap_allocation_data defines the UAPI as follows: > > struct dma_heap_allocation_data { > __u64 len; > __u32 fd; > __u32 fd_flags; > __u64 heap_flags; > }; > > However, dma_heap_buffer_alloc() casts both fd_flags and heap_flags > into unsigned int. We're inconsistent with types in the non UAPI > arguments. This patch fixes it. Thank you for your patch; I've pushed it to drm-misc-next. Best, Sumit. > > Signed-off-by: Barry Song <v-songbaohua(a)oppo.com> > Acked-by: John Stultz <jstultz(a)google.com> > --- > -v2: > collect ack of John, thanks! > refine commit message; > > drivers/dma-buf/dma-heap.c | 4 ++-- > include/uapi/linux/dma-heap.h | 2 +- > 2 files changed, 3 insertions(+), 3 deletions(-) > > diff --git a/drivers/dma-buf/dma-heap.c b/drivers/dma-buf/dma-heap.c > index 84ae708fafe7..2298ca5e112e 100644 > --- a/drivers/dma-buf/dma-heap.c > +++ b/drivers/dma-buf/dma-heap.c > @@ -50,8 +50,8 @@ static struct class *dma_heap_class; > static DEFINE_XARRAY_ALLOC(dma_heap_minors); > > static int dma_heap_buffer_alloc(struct dma_heap *heap, size_t len, > - unsigned int fd_flags, > - unsigned int heap_flags) > + u32 fd_flags, > + u64 heap_flags) > { > struct dma_buf *dmabuf; > int fd; > diff --git a/include/uapi/linux/dma-heap.h b/include/uapi/linux/dma-heap.h > index 6f84fa08e074..a4cf716a49fa 100644 > --- a/include/uapi/linux/dma-heap.h > +++ b/include/uapi/linux/dma-heap.h > @@ -19,7 +19,7 @@ > #define DMA_HEAP_VALID_FD_FLAGS (O_CLOEXEC | O_ACCMODE) > > /* Currently no heap flags */ > -#define DMA_HEAP_VALID_HEAP_FLAGS (0) > +#define DMA_HEAP_VALID_HEAP_FLAGS (0ULL) > > /** > * struct dma_heap_allocation_data - metadata passed from userspace for > -- > 2.34.1 > -- Thanks and regards, Sumit Semwal (he / him) Tech Lead - LCG, Vertical Technologies Linaro.org │ Open source software for ARM SoCs

1 year, 8 months

1
0
0 0

Re: [RFC PATCH v1] dma-buf: heaps: move the verification of heap_flags to the corresponding heap

by John Stultz

On Mon, Jun 3, 2024 at 11:30 PM Hailong Liu <hailong.liu(a)oppo.com> wrote: > On 6/4/2024 2:06 AM, John Stultz wrote: > > On Mon, Jun 3, 2024 at 10:21 AM Hailong Liu <hailong.liu(a)oppo.com> wrote: > >> We now aim to improve priority dma-buf allocation. Consider android > >> animations scene: > >> > >> when device is in low memory, Allocating dma-buf as animation > >> buffers enter direct_reclaimation, longer allocation time result in a > >> laggy UI. But if we know the usage of the dma-buf, we can use some > >> mechanisms to boost, e.g. animation-memory-pool. > > > > Can you generalize this a bit further? When would userland know to use > > this new flag? > > If it is aware, would it make sense to just use a separate heap name instead? > > > > (Also: These other mechanisms you mention should probably also be > > submitted upstream, however for upstream there's also the requirement > > that we have open users and are not just enabling proprietary blob > > userspace, which makes any changes to dma-buf heaps for out of tree > > code quite difficult) > > > >> However, dma-buf usage identification becomes a challenge. A potential > >> solution could be heap_flags. the use of heap_flags seems ugly and > >> contrary to the intended design as you said, How aboult extending > >> dma_heap_allocation_data as follows? > >> > >> struct dma_heap_allocation_data { > >> __u64 len; > >> __u32 fd; > >> __u32 fd_flags; > >> __u64 heap_flags; > >> __u64 buf_flags: // buf usage > >> }; > > > > This would affect the ABI (forcing a new ioctl number). And it's > > unclear what flags you envision as buffer specific (rather than heap > > specific as this patch suggested). > > > > I think we need more details about the specific problem you're seeing > > and trying to resolve. > This patch mainly focuses on optimization for Android scenarios. Let’s > discuss it on the issue website. > Bug: 344501512 Ok, we can do that if you need. But if this is ever going to go upstream (and it's more and more important that we minimize out of tree technical debt), conversations about how to generalize this will need to happen on the list. thanks -john

1 year, 8 months

1
0
0 0

Re: [RFC PATCH v1] dma-buf: heaps: move the verification of heap_flags to the corresponding heap

by John Stultz

On Mon, Jun 3, 2024 at 10:21 AM Hailong Liu <hailong.liu(a)oppo.com> wrote: > On Mon, 03. Jun 09:01, John Stultz wrote: > > On Mon, Jun 3, 2024 at 4:40 AM <hailong.liu(a)oppo.com> wrote: > > > > > > From: "Hailong.Liu" <hailong.liu(a)oppo.com> > > > > > > This help module use heap_flags to determine the type of dma-buf, > > > so that some mechanisms can be used to speed up allocation, such as > > > memory_pool, to optimize the allocation time of dma-buf. > > > > This feels like it's trying to introduce heap specific flags, but > > doesn't introduce any details about what those flags might be? > > > > This seems like it would re-allow the old opaque vendor specific heap > > flags that we saw in the ION days, which was problematic as different > > userspaces would use the same interface with potentially colliding > > heap flags with different meanings. Resulting in no way to properly > > move to an upstream solution. > > > > With the dma-heaps interface, we're trying to make sure it is well > > defined. One can register a number of heaps with different behaviors, > > and the heap name is used to differentiate the behavior. Any flags > > introduced will need to be well defined and behaviorally consistent > > between heaps. That way when an upstream solution lands, if necessary > > we can provide backwards compatibility via symlinks. > > > > So I don't think this is a good direction to go for dma-heaps. > > > > It would be better if you were able to clarify what flag requirements > > you need, so we can better understand how they might apply to other > > heaps, and see if it was something we would want to define as a flag > > (see the discussion here for similar thoughts: > > https://lore.kernel.org/lkml/CANDhNCoOKwtpstFE2VDcUvzdXUWkZ-Zx+fz6xrdPWTyci… > > ) > > > > But if your vendor heap really needs some sort of flags argument that > > you can't generalize, you can always implement your own dmabuf > > exporter driver with whatever ioctl interface you'd prefer. > > Thanks for your reply. Let’s continue our discussion here instead > of on android-review. We aim to enhance memory allocation on each > all heaps. Your pointer towards heap_flags used in /dev/ion for heap > identification was helpful. > > We now aim to improve priority dma-buf allocation. Consider android > animations scene: > > when device is in low memory, Allocating dma-buf as animation > buffers enter direct_reclaimation, longer allocation time result in a > laggy UI. But if we know the usage of the dma-buf, we can use some > mechanisms to boost, e.g. animation-memory-pool. Can you generalize this a bit further? When would userland know to use this new flag? If it is aware, would it make sense to just use a separate heap name instead? (Also: These other mechanisms you mention should probably also be submitted upstream, however for upstream there's also the requirement that we have open users and are not just enabling proprietary blob userspace, which makes any changes to dma-buf heaps for out of tree code quite difficult) > However, dma-buf usage identification becomes a challenge. A potential > solution could be heap_flags. the use of heap_flags seems ugly and > contrary to the intended design as you said, How aboult extending > dma_heap_allocation_data as follows? > > struct dma_heap_allocation_data { > __u64 len; > __u32 fd; > __u32 fd_flags; > __u64 heap_flags; > __u64 buf_flags: // buf usage > }; This would affect the ABI (forcing a new ioctl number). And it's unclear what flags you envision as buffer specific (rather than heap specific as this patch suggested). I think we need more details about the specific problem you're seeing and trying to resolve. thanks -john

1 year, 8 months

1
0
0 0

Re: [RFC PATCH v1] dma-buf: heaps: move the verification of heap_flags to the corresponding heap

by John Stultz

On Mon, Jun 3, 2024 at 4:40 AM <hailong.liu(a)oppo.com> wrote: > > From: "Hailong.Liu" <hailong.liu(a)oppo.com> > > This help module use heap_flags to determine the type of dma-buf, > so that some mechanisms can be used to speed up allocation, such as > memory_pool, to optimize the allocation time of dma-buf. This feels like it's trying to introduce heap specific flags, but doesn't introduce any details about what those flags might be? This seems like it would re-allow the old opaque vendor specific heap flags that we saw in the ION days, which was problematic as different userspaces would use the same interface with potentially colliding heap flags with different meanings. Resulting in no way to properly move to an upstream solution. With the dma-heaps interface, we're trying to make sure it is well defined. One can register a number of heaps with different behaviors, and the heap name is used to differentiate the behavior. Any flags introduced will need to be well defined and behaviorally consistent between heaps. That way when an upstream solution lands, if necessary we can provide backwards compatibility via symlinks. So I don't think this is a good direction to go for dma-heaps. It would be better if you were able to clarify what flag requirements you need, so we can better understand how they might apply to other heaps, and see if it was something we would want to define as a flag (see the discussion here for similar thoughts: https://lore.kernel.org/lkml/CANDhNCoOKwtpstFE2VDcUvzdXUWkZ-Zx+fz6xrdPWTyci… ) But if your vendor heap really needs some sort of flags argument that you can't generalize, you can always implement your own dmabuf exporter driver with whatever ioctl interface you'd prefer. thanks -john

1 year, 8 months

1
0
0 0

Re: [PATCH v6 0/7] Add mediate-drm secure flow for SVP

by mripard＠kernel.org

On Tue, May 28, 2024 at 07:15:34AM GMT, Jason-JH Lin (林睿祥) wrote: > Hi Maxime, > > On Mon, 2024-05-27 at 16:06 +0200, Maxime Ripard wrote: > > Hi, > > > > On Sun, May 26, 2024 at 07:29:21AM GMT, Jason-JH.Lin wrote: > > > From: Jason-jh Lin <jason-jh.lin(a)mediatek.corp-partner.google.com> > > > > > > Memory Definitions: > > > secure memory - Memory allocated in the TEE (Trusted Execution > > > Environment) which is inaccessible in the REE (Rich Execution > > > Environment, i.e. linux kernel/userspace). > > > secure handle - Integer value which acts as reference to 'secure > > > memory'. Used in communication between TEE and REE to reference > > > 'secure memory'. > > > secure buffer - 'secure memory' that is used to store decrypted, > > > compressed video or for other general purposes in the TEE. > > > secure surface - 'secure memory' that is used to store graphic > > > buffers. > > > > > > Memory Usage in SVP: > > > The overall flow of SVP starts with encrypted video coming in from > > > an > > > outside source into the REE. The REE will then allocate a 'secure > > > buffer' and send the corresponding 'secure handle' along with the > > > encrypted, compressed video data to the TEE. The TEE will then > > > decrypt > > > the video and store the result in the 'secure buffer'. The REE will > > > then allocate a 'secure surface'. The REE will pass the 'secure > > > handles' for both the 'secure buffer' and 'secure surface' into the > > > TEE for video decoding. The video decoder HW will then decode the > > > contents of the 'secure buffer' and place the result in the 'secure > > > surface'. The REE will then attach the 'secure surface' to the > > > overlay > > > plane for rendering of the video. > > > > > > Everything relating to ensuring security of the actual contents of > > > the > > > 'secure buffer' and 'secure surface' is out of scope for the REE > > > and > > > is the responsibility of the TEE. > > > > > > DRM driver handles allocation of gem objects that are backed by a > > > 'secure > > > surface' and for displaying a 'secure surface' on the overlay > > > plane. > > > This introduces a new flag for object creation called > > > DRM_MTK_GEM_CREATE_RESTRICTED which indicates it should be a > > > 'secure > > > surface'. All changes here are in MediaTek specific code. > > > --- > > > TODO: > > > 1) Drop MTK_DRM_IOCTL_GEM_CREATE and use DMA_HEAP_IOCTL_ALLOC in > > > userspace > > > 2) DRM driver use secure mailbox channel to handle normal and > > > secure flow > > > 3) Implement setting mmsys routing table in the secure world series > > > > I'm not sure what you mean here. Why are you trying to upstream > > something that still needs to be removed from your patch series? > > > Because their is too much patches need to be fixed in this series, so I > list down the remaining TODO items and send to review for the other > patches. > > Sorry for the bothering, I'll drop this at the next version. If you don't intend to use it, we just shouldn't add it. Removing the TODO item doesn't make sense, even more so if heaps should be the way you handle this. > > Also, I made some comments on the previous version that have been > > entirely ignored and still apply on this version: > > > https://lore.kernel.org/dri-devel/20240415-guppy-of-perpetual-current-3a797… > > > > I lost that mail in my mailbox, so I didn't reply at that time. > I have imported that mail and replied to you. Hope you don't mind :) I haven't received that answer Maxime

1 year, 8 months

1
0
0 0

Re: [RFC PATCH] dma-buf: align fd_flags and heap_flags with dma_heap_allocation_data

by John Stultz

On Wed, May 22, 2024 at 2:02 AM Barry Song <21cnbao(a)gmail.com> wrote: > > From: Barry Song <v-songbaohua(a)oppo.com> > > dma_heap_allocation_data defines the UAPI as follows: > > struct dma_heap_allocation_data { > __u64 len; > __u32 fd; > __u32 fd_flags; > __u64 heap_flags; > }; > > However, dma_heap_buffer_alloc() casts them into unsigned int. It's unclear > whether this is intentional or what the purpose is, but it can be quite > confusing for users. > > Adding to the confusion, dma_heap_ops.allocate defines both of these as > unsigned long. Fortunately, since dma_heap_ops is not part of the UAPI, > it is less of a concern. > > struct dma_heap_ops { > struct dma_buf *(*allocate)(struct dma_heap *heap, > unsigned long len, > unsigned long fd_flags, > unsigned long heap_flags); > }; > > I am sending this RFC in hopes of clarifying these confusions. > > If the goal is to constrain both flags to 32 bits while ensuring the struct > is aligned to 64 bits, it would have been more suitable to define > dma_heap_allocation_data accordingly from the beginning, like so: > > struct dma_heap_allocation_data { > __u64 len; > __u32 fd; > __u32 fd_flags; > __u32 heap_flags; > __u32 padding; > }; So here, if I recall, the intent was to keep 64bits for potential future heap_flags. But your point above that we're inconsistent with types in the non UAPI arguments is valid. So I think your patch makes sense. Thanks for raising this issue! Acked-by: John Stultz <jstultz(a)google.com>

1 year, 8 months

1
0
0 0

[PATCH] dma-buf: add a warning when drv try to reserve 0 fence slots

by Christian König

When dma_resv_reserve_fences() is called with num_fences=0 it usually means that a driver or other component messed up its calculation how many fences are needed. Warn in that situation. When no fence are needed the function shouldn't be called in the first place. Signed-off-by: Christian König <christian.koenig(a)amd.com> --- drivers/dma-buf/dma-resv.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c index e2869fb31140..5f8d010516f0 100644 --- a/drivers/dma-buf/dma-resv.c +++ b/drivers/dma-buf/dma-resv.c @@ -186,6 +186,13 @@ int dma_resv_reserve_fences(struct dma_resv *obj, unsigned int num_fences) dma_resv_assert_held(obj); + /* Driver and component code should never call this function with + * num_fences=0. If they do it usually points to bugs when calculating + * the number of needed fences dynamically. + */ + if (WARN_ON(!num_fences)) + return -EINVAL; + old = dma_resv_fences_list(obj); if (old && old->max_fences) { if ((old->num_fences + num_fences) <= old->max_fences) -- 2.34.1

1 year, 8 months

1
0
0 0

Re: [PATCH] udmabuf: add CONFIG_MMU dependency

by David Hildenbrand

Am 28.05.24 um 15:31 schrieb Arnd Bergmann: > From: Arnd Bergmann <arnd(a)arndb.de> > > There is no !CONFIG_MMU version of vmf_insert_pfn(): > > arm-linux-gnueabi-ld: drivers/dma-buf/udmabuf.o: in function `udmabuf_vm_fault': > udmabuf.c:(.text+0xaa): undefined reference to `vmf_insert_pfn' > > Fixes: f7254e043ff1 ("udmabuf: use vmf_insert_pfn and VM_PFNMAP for handling mmap") > Signed-off-by: Arnd Bergmann <arnd(a)arndb.de> > --- > drivers/dma-buf/Kconfig | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/drivers/dma-buf/Kconfig b/drivers/dma-buf/Kconfig > index e4dc53a36428..b46eb8a552d7 100644 > --- a/drivers/dma-buf/Kconfig > +++ b/drivers/dma-buf/Kconfig > @@ -35,6 +35,7 @@ config UDMABUF > default n > depends on DMA_SHARED_BUFFER > depends on MEMFD_CREATE || COMPILE_TEST > + depends on MMU > help > A driver to let userspace turn memfd regions into dma-bufs. > Qemu can use this to create host dmabufs for guest framebuffers. Acked-by: David Hildenbrand <david(a)redhat.com> -- Thanks, David / dhildenb

1 year, 8 months

1
0
0 0

Re: [PATCH v4 1/3] dt-bindings: mailbox: Add mediatek,gce-props.yaml

by AngeloGioacchino Del Regno

Il 26/05/24 17:04, Jason-JH Lin (林睿祥) ha scritto: > Hi Angelo, Jassi, > > Could you help me apply this series? > Thanks! > That's not me, it's Jassi - green light from me, btw. Cheers, Angelo > Regards, > Jason-JH.Lin > > On Wed, 2024-01-24 at 09:57 +0100, AngeloGioacchino Del Regno wrote: >> Il 24/01/24 02:14, Jason-JH.Lin ha scritto: >>> Add mediatek,gce-props.yaml for common GCE properties that is used >>> for >>> both mailbox providers and consumers. We place the common property >>> "mediatek,gce-events" in this binding currently. >>> >>> The property "mediatek,gce-events" is used for GCE event ID >>> corresponding >>> to a hardware event signal sent by the hardware or a software >>> driver. >>> If the mailbox providers or consumers want to manipulate the value >>> of >>> the event ID, they need to know the specific event ID. >>> >>> Signed-off-by: Jason-JH.Lin <jason-jh.lin(a)mediatek.com> >>> Reviewed-by: Conor Dooley <conor.dooley(a)microchip.com> >> >> Reviewed-by: AngeloGioacchino Del Regno < >> angelogioacchino.delregno(a)collabora.com> >> >

1 year, 8 months

1
0
0 0

Re: [PATCH v2] dma-buf: handle testing kthreads creation failure

by T.J. Mercier

On Wed, May 22, 2024 at 11:14 AM Fedor Pchelkin <pchelkin(a)ispras.ru> wrote: > > kthread creation may possibly fail inside race_signal_callback(). In > such a case stop the already started threads, put the already taken > references to them and return with error code. > > Found by Linux Verification Center (linuxtesting.org). > > Fixes: 2989f6451084 ("dma-buf: Add selftests for dma-fence") > Cc: stable(a)vger.kernel.org > Signed-off-by: Fedor Pchelkin <pchelkin(a)ispras.ru> Reviewed-by: T.J. Mercier <tjmercier(a)google.com> > --- > v2: use kthread_stop_put() to actually put the last reference as > T.J. Mercier noticed; > link to v1: https://lore.kernel.org/lkml/20240522122326.696928-1-pchelkin@ispras.ru/ > > drivers/dma-buf/st-dma-fence.c | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/drivers/dma-buf/st-dma-fence.c b/drivers/dma-buf/st-dma-fence.c > index b7c6f7ea9e0c..6a1bfcd0cc21 100644 > --- a/drivers/dma-buf/st-dma-fence.c > +++ b/drivers/dma-buf/st-dma-fence.c > @@ -540,6 +540,12 @@ static int race_signal_callback(void *arg) > t[i].before = pass; > t[i].task = kthread_run(thread_signal_callback, &t[i], > "dma-fence:%d", i); > + if (IS_ERR(t[i].task)) { > + ret = PTR_ERR(t[i].task); > + while (--i >= 0) > + kthread_stop_put(t[i].task); > + return ret; > + } > get_task_struct(t[i].task); > } > > -- > 2.39.2 >

1 year, 8 months

2
1
0 0

[PATCH v6 0/7] Add mediate-drm secure flow for SVP

by Jason-JH.Lin

From: Jason-jh Lin <jason-jh.lin(a)mediatek.corp-partner.google.com> Memory Definitions: secure memory - Memory allocated in the TEE (Trusted Execution Environment) which is inaccessible in the REE (Rich Execution Environment, i.e. linux kernel/userspace). secure handle - Integer value which acts as reference to 'secure memory'. Used in communication between TEE and REE to reference 'secure memory'. secure buffer - 'secure memory' that is used to store decrypted, compressed video or for other general purposes in the TEE. secure surface - 'secure memory' that is used to store graphic buffers. Memory Usage in SVP: The overall flow of SVP starts with encrypted video coming in from an outside source into the REE. The REE will then allocate a 'secure buffer' and send the corresponding 'secure handle' along with the encrypted, compressed video data to the TEE. The TEE will then decrypt the video and store the result in the 'secure buffer'. The REE will then allocate a 'secure surface'. The REE will pass the 'secure handles' for both the 'secure buffer' and 'secure surface' into the TEE for video decoding. The video decoder HW will then decode the contents of the 'secure buffer' and place the result in the 'secure surface'. The REE will then attach the 'secure surface' to the overlay plane for rendering of the video. Everything relating to ensuring security of the actual contents of the 'secure buffer' and 'secure surface' is out of scope for the REE and is the responsibility of the TEE. DRM driver handles allocation of gem objects that are backed by a 'secure surface' and for displaying a 'secure surface' on the overlay plane. This introduces a new flag for object creation called DRM_MTK_GEM_CREATE_RESTRICTED which indicates it should be a 'secure surface'. All changes here are in MediaTek specific code. --- TODO: 1) Drop MTK_DRM_IOCTL_GEM_CREATE and use DMA_HEAP_IOCTL_ALLOC in userspace 2) DRM driver use secure mailbox channel to handle normal and secure flow 3) Implement setting mmsys routing table in the secure world series --- Based on 3 series: [1] v3 dma-buf: heaps: Add MediaTek secure heap - https://patchwork.kernel.org/project/linux-mediatek/list/?series=809023 [2] v6 media: mediatek: add driver to support secure video decoder - https://patchwork.kernel.org/project/linux-mediatek/list/?series=853689 [3] v6 Add CMDQ secure driver for SVP - https://patchwork.kernel.org/project/linux-mediatek/list/?series=855884 --- Change in v6: 1. Rebase to linux-next then rebased on 3 series [1][2][3] 2. Remove some parameter and security settings in normal world 3. Drop secure port related patches 4. Fix some build error and warning Changes in v5: 1. Sync the local changes Changes in v4: 1. Rebase on mediatek-drm-next(278640d4d74cd) and fix the conflicts 2. This series is based on 20240129063025.29251-1-yunfei.dong(a)mediatek.com 3. This series is based on 20240322052829.9893-1-shawn.sung(a)mediatek.com 4. This series is based on 20240403065603.21920-1-shawn.sung(a)mediatek.com Changes in v3: 1. fix kerneldoc problems 2. fix typo in title and commit message 3. adjust naming for secure variable 4. add the missing part for is_suecure plane implementation 5. use BIT_ULL macro to replace bit shifting 6. move modification of ovl_adaptor part to the correct patch 7. add TODO list in commit message 8. add commit message for using share memory to store execute count Changes in v2: 1. remove the DRIVER_RDNDER flag for mtk_drm_ioctl 2. move cmdq_insert_backup_cookie into client driver 3. move secure gce node define from mt8195-cherry.dtsi to mt8195.dtsi --- CK Hu (1): drm/mediatek: Add interface to allocate MediaTek GEM buffer Jason-JH.Lin (6): drm/mediatek/uapi: Add DRM_MTK_GEM_CREATE_RESTRICTED flag drm/mediatek: Add secure buffer control flow to mtk_drm_gem drm/mediatek: Add secure identify flag and funcution to mtk_drm_plane drm/mediatek: Add mtk_ddp_sec_write() to config secure buffer info drm/mediatek: Add secure flow support to mediatek-drm drm/mediatek: Add cmdq_insert_backup_cookie before secure pkt finalize drivers/gpu/drm/mediatek/mtk_crtc.c | 263 +++++++++++++++++- drivers/gpu/drm/mediatek/mtk_crtc.h | 1 + drivers/gpu/drm/mediatek/mtk_ddp_comp.c | 14 + drivers/gpu/drm/mediatek/mtk_ddp_comp.h | 5 + drivers/gpu/drm/mediatek/mtk_disp_ovl.c | 9 +- .../gpu/drm/mediatek/mtk_disp_ovl_adaptor.c | 1 + drivers/gpu/drm/mediatek/mtk_drm_drv.c | 13 +- drivers/gpu/drm/mediatek/mtk_gem.c | 114 ++++++++ drivers/gpu/drm/mediatek/mtk_gem.h | 12 + drivers/gpu/drm/mediatek/mtk_mdp_rdma.c | 8 +- drivers/gpu/drm/mediatek/mtk_mdp_rdma.h | 1 + drivers/gpu/drm/mediatek/mtk_plane.c | 25 ++ drivers/gpu/drm/mediatek/mtk_plane.h | 2 + include/uapi/drm/mediatek_drm.h | 47 ++++ 14 files changed, 500 insertions(+), 15 deletions(-) create mode 100644 include/uapi/drm/mediatek_drm.h -- 2.18.0

1 year, 8 months

2
8
0 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Linaro-mm-sig