- Linaro-mm-sig - lists.linaro.org

Re: [RFC PATCH 0/5] drm/panthor: Protected mode support for Mali CSF GPUs

by Maxime Ripard

Hi, I started to review it, but it's probably best to discuss it here. On Thu, Jan 30, 2025 at 01:08:56PM +0000, Florent Tomasin wrote: > Hi, > > This is a patch series covering the support for protected mode execution in > Mali Panthor CSF kernel driver. > > The Mali CSF GPUs come with the support for protected mode execution at the > HW level. This feature requires two main changes in the kernel driver: > > 1) Configure the GPU with a protected buffer. The system must provide a DMA > heap from which the driver can allocate a protected buffer. > It can be a carved-out memory or dynamically allocated protected memory region. > Some system includes a trusted FW which is in charge of the protected memory. > Since this problem is integration specific, the Mali Panthor CSF kernel > driver must import the protected memory from a device specific exporter. Why do you need a heap for it in the first place? My understanding of your series is that you have a carved out memory region somewhere, and you want to allocate from that carved out memory region your buffers. How is that any different from using a reserved-memory region, adding the reserved-memory property to the GPU device and doing all your allocation through the usual dma_alloc_* API? Or do you expect to have another dma-buf heap that would call into the firmware to perform the allocations? The semantics of the CMA heap allocations is a concern too. Another question is how would you expect something like a secure video-playback pipeline to operate with that kind of interface. Assuming you have a secure codec, you would likely get that protected buffer from the codec, right? So the most likely scenario would be to then import that dma-buf into the GPU driver, but not allocate the buffer from it. Overall, I think a better abstraction would be to have a heap indeed to allocate your protected buffers from, and then import them in the devices that need them. The responsibility would be on the userspace to do so, but it already kind of does with your design anyway. Maxime

1 year

2
3
0 0

Re: [RFC PATCH 2/5] cma-heap: Allow registration of custom cma heaps

by Maxime Ripard

Hi, On Thu, Jan 30, 2025 at 01:08:58PM +0000, Florent Tomasin wrote: > This patch introduces a cma-heap probe function, allowing > users to register custom cma heaps in the device tree. > > A "memory-region" is bound to the cma heap at probe time > allowing allocation of DMA buffers from that heap. > > Use cases: > - registration of carved out secure heaps. Some devices > are implementing secure memory by reserving a specific > memory regions for that purpose. For example, this is the > case of platforms making use of early version of > ARM TrustZone. In such a case, the CMA heap would de-facto become un-mappable for userspace, right? > - registration of multiple memory regions at different > locations for efficiency or HW integration reasons. > For example, a peripheral may expect to share data at a > specific location in RAM. This information could have been > programmed by a FW prior to the kernel boot. How would you differentiate between them? Maxime

1 year

1
0
0 0

Re: [RFC PATCH 1/5] dt-bindings: dma: Add CMA Heap bindings

by Maxime Ripard

Hi, On Thu, Jan 30, 2025 at 01:08:57PM +0000, Florent Tomasin wrote: > Introduce a CMA Heap dt-binding allowing custom > CMA heap registrations. > > * Note to the reviewers: > The patch was used for the development of the protected mode > feature in Panthor CSF kernel driver and is not initially thought > to land in the Linux kernel. It is mostly relevant if someone > wants to reproduce the environment of testing. Please, raise > interest if you think the patch has value in the Linux kernel. > > Signed-off-by: Florent Tomasin <florent.tomasin(a)arm.com> > --- > .../devicetree/bindings/dma/linux,cma.yml | 43 +++++++++++++++++++ > 1 file changed, 43 insertions(+) > create mode 100644 Documentation/devicetree/bindings/dma/linux,cma.yml > > diff --git a/Documentation/devicetree/bindings/dma/linux,cma.yml b/Documentation/devicetree/bindings/dma/linux,cma.yml > new file mode 100644 > index 000000000000..c532e016bbe5 > --- /dev/null > +++ b/Documentation/devicetree/bindings/dma/linux,cma.yml > @@ -0,0 +1,43 @@ > +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause > +%YAML 1.2 > +--- > +$id: http://devicetree.org/schemas/dma/linux-cma.yaml# > +$schema: http://devicetree.org/meta-schemas/core.yaml# > + > +title: Custom Linux CMA heap > + > +description: > + The custom Linux CMA heap device tree node allows registering > + of multiple CMA heaps. > + > + The CMA heap name will match the node name of the "memory-region". > + > +properties: > + compatible: > + enum: > + - linux,cma > + > + memory-region: > + maxItems: 1 > + description: | > + Phandle to the reserved memory node associated with the CMA Heap. > + The reserved memory node must follow this binding convention: > + - Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > + > +examples: > + - | > + reserved-memory { > + #address-cells = <2>; > + #size-cells = <2>; > + > + custom_cma_heap: custom-cma-heap { > + compatible = "shared-dma-pool"; > + reg = <0x0 0x90600000 0x0 0x1000000>; > + reusable; > + }; > + }; > + > + device_cma_heap: device-cma-heap { > + compatible = "linux,cma"; > + memory-region = <&custom_cma_heap>; > + }; Isn't it redundant with the linux,cma-default shared-dma-pool property already? Maxime

1 year

1
0
0 0

Re: [RFC PATCH 3/5] dt-bindings: gpu: Add protected heap name to Mali Valhall CSF binding

by Krzysztof Kozlowski

On 30/01/2025 14:08, Florent Tomasin wrote: > Allow mali-valhall-csf driver to retrieve a protected > heap at probe time by passing the name of the heap > as attribute to the device tree GPU node. Please wrap commit message according to Linux coding style / submission process (neither too early nor over the limit): https://elixir.bootlin.com/linux/v6.4-rc1/source/Documentation/process/subm… Why this cannot be passed by phandle, just like all reserved regions? From where do you take these protected heaps? Firmware? This would explain why no relation is here (no probe ordering, no device links, nothing connecting separate devices). Best regards, Krzysztof

1 year

1
0
0 0

[PATCH] dma-buf: fix timeout handling in dma_resv_wait_timeout v2

by Christian König

Even the kerneldoc says that with a zero timeout the function should not wait for anything, but still return 1 to indicate that the fences are signaled now. Unfortunately that isn't what was implemented, instead of only returning 1 we also waited for at least one jiffies. Fix that by adjusting the handling to what the function is actually documented to do. v2: improve code readability Reported-by: Marek Olšák <marek.olsak(a)amd.com> Reported-by: Lucas Stach <l.stach(a)pengutronix.de> Signed-off-by: Christian König <christian.koenig(a)amd.com> Cc: <stable(a)vger.kernel.org> --- drivers/dma-buf/dma-resv.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c index 5f8d010516f0..c78cdae3deaf 100644 --- a/drivers/dma-buf/dma-resv.c +++ b/drivers/dma-buf/dma-resv.c @@ -684,11 +684,13 @@ long dma_resv_wait_timeout(struct dma_resv *obj, enum dma_resv_usage usage, dma_resv_iter_begin(&cursor, obj, usage); dma_resv_for_each_fence_unlocked(&cursor, fence) { - ret = dma_fence_wait_timeout(fence, intr, ret); - if (ret <= 0) { - dma_resv_iter_end(&cursor); - return ret; - } + ret = dma_fence_wait_timeout(fence, intr, timeout); + if (ret <= 0) + break; + + /* Even for zero timeout the return value is 1 */ + if (timeout) + timeout = ret; } dma_resv_iter_end(&cursor); -- 2.34.1

1 year

2
1
0 0

[PATCH] dma-buf: fix timeout handling in dma_resv_wait_timeout

by Christian König

Even the kerneldoc says that with a zero timeout the function should not wait for anything, but still return 1 to indicate that the fences are signaled now. Unfortunately that isn't what was implemented, instead of only returning 1 we also waited for at least one jiffies. Fix that by adjusting the handling to what the function is actually documented to do. Reported-by: Marek Olšák <marek.olsak(a)amd.com> Reported-by: Lucas Stach <l.stach(a)pengutronix.de> Signed-off-by: Christian König <christian.koenig(a)amd.com> Cc: <stable(a)vger.kernel.org> --- drivers/dma-buf/dma-resv.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c index 5f8d010516f0..ae92d9d2394d 100644 --- a/drivers/dma-buf/dma-resv.c +++ b/drivers/dma-buf/dma-resv.c @@ -684,11 +684,12 @@ long dma_resv_wait_timeout(struct dma_resv *obj, enum dma_resv_usage usage, dma_resv_iter_begin(&cursor, obj, usage); dma_resv_for_each_fence_unlocked(&cursor, fence) { - ret = dma_fence_wait_timeout(fence, intr, ret); - if (ret <= 0) { - dma_resv_iter_end(&cursor); - return ret; - } + ret = dma_fence_wait_timeout(fence, intr, timeout); + if (ret <= 0) + break; + + /* Even for zero timeout the return value is 1 */ + timeout = min(timeout, ret); } dma_resv_iter_end(&cursor); -- 2.34.1

1 year

2
4
0 0

Re: [RFC PATCH 01/12] dma-buf: Introduce dma_buf_get_pfn_unlocked() kAPI

by Christian König

Am 22.01.25 um 12:04 schrieb Simona Vetter: > On Tue, Jan 21, 2025 at 01:36:33PM -0400, Jason Gunthorpe wrote: >> On Tue, Jan 21, 2025 at 05:11:32PM +0100, Simona Vetter wrote: >>> On Mon, Jan 20, 2025 at 03:48:04PM -0400, Jason Gunthorpe wrote: >>>> On Mon, Jan 20, 2025 at 07:50:23PM +0100, Simona Vetter wrote: >>>>> On Mon, Jan 20, 2025 at 01:59:01PM -0400, Jason Gunthorpe wrote: >>>>>> On Mon, Jan 20, 2025 at 01:14:12PM +0100, Christian König wrote: >>>>>> What is going wrong with your email? You replied to Simona, but >>>>>> Simona Vetter <simona.vetter(a)ffwll.ch> is dropped from the To/CC >>>>>> list??? I added the address back, but seems like a weird thing to >>>>>> happen. >>>>> Might also be funny mailing list stuff, depending how you get these. I >>>>> read mails over lore and pretty much ignore cc (unless it's not also on >>>>> any list, since those tend to be security issues) because I get cc'ed on >>>>> way too much stuff for that to be a useful signal. >>>> Oh I see, you are sending a Mail-followup-to header that excludes your >>>> address, so you don't get any emails at all.. My mutt is dropping you >>>> as well. I'm having all kind of funny phenomena with AMDs mail servers since coming back from xmas vacation. From the news it looks like Outlook on Windows has a new major security issue where just viewing a mail can compromise the system and my educated guess is that our IT guys went into panic mode because of this and has changed something. >> [SNIP] >> I have been assuming that dmabuf mmap remains unchanged, that >> exporters will continue to implement that mmap() callback as today. That sounds really really good to me because that was my major concern when you noted that you want to have PFNs to build up KVM page tables. But you don't want to handle mmap() on your own, you basically don't want to have a VMA for this stuff at all, correct? >> My main interest has been what data structure is produced in the >> attach APIs. >> >> Eg today we have a struct dma_buf_attachment that returns a sg_table. >> >> I'm expecting some kind of new data structure, lets call it "physical >> list" that is some efficient coding of meta/addr/len tuples that works >> well with the new DMA API. Matthew has been calling this thing phyr.. I would not use a data structure at all. Instead we should have something like an iterator/cursor based approach similar to what the new DMA API is doing. >> So, I imagine, struct dma_buf_attachment gaining an optional >> feature negotiation and then we have in dma_buf_attachment: >> >> union { >> struct sg_table *sgt; >> struct physical_list *phyr; >> }; >> >> That's basicaly it, an alternative to scatterlist that has a clean >> architecture. I would rather suggest something like dma_buf_attachment() gets offset and size to map and returns a cursor object you can use to get your address, length and access attributes. And then you can iterate over this cursor and fill in your importer data structure with the necessary information. This way neither the exporter nor the importer need to convert their data back and forth between their specific representations of the information. >> Now, if you are asking if the current dmabuf mmap callback can be >> improved with the above? Maybe? phyr should have the neccessary >> information inside it to populate a VMA - eventually even fully >> correctly with all the right cachable/encrypted/forbidden/etc flags. That won't work like this. See the exporter needs to be informed about page faults on the VMA to eventually wait for operations to end and sync caches. Otherwise we either potentially allow access to freed up or re-used memory or run into issues with device cache coherency. >> So, you could imagine that exporters could just have one routine to >> generate the phyr list and that goes into the attachment, goes into >> some common code to fill VMA PTEs, and some other common code that >> will convert it into the DMABUF scatterlist. If performance is not a >> concern with these data structure conversions it could be an appealing >> simplification. >> >> And yes, I could imagine the meta information being descriptive enough >> to support the private interconnect cases, the common code could >> detect private meta information and just cleanly fail. > I'm kinda leaning towards entirely separate dma-buf interfaces for the new > phyr stuff, because I fear that adding that to the existing ones will only > make the chaos worse. But that aside sounds all reasonable, and even that > could just be too much worry on my side and mixing phyr into existing > attachments (with a pile of importer/exporter flags probably) is fine. I lean into the other direction. Dmitry and Thomas have done a really good job at cleaning up all the interaction between dynamic and static exporters / importers. Especially that we now have consistent locking for map_dma_buf() and unmap_dma_buf() should make that transition rather straight forward. > For the existing dma-buf importers/exporters I'm kinda hoping for a pure > dma_addr_t based list eventually. Going all the way to a phyr based > approach for everyone might be too much churn, there's some real bad cruft > there. It's not going to work for every case, but it covers a lot of them > and might be less pain for existing importers. The point is we have use cases that won't work without exchanging DMA addresses any more. For example we have cases with multiple devices are in the same IOMMU domain and re-using their DMA address mappings. > But in theory it should be possible to use phyr everywhere eventually, as > long as there's no obviously api-rules-breaking way to go from a phyr back to > a struct page even when that exists. I would rather say we should stick to DMA addresses as much as possible. What we can do is to add an address space description to the addresses, e.g. if it's a PCIe BUS addr in IOMMU domain X, or of it's a device private bus addr or in the case of sharing with iommufd and KVM PFNs. Regards, Christian. >>> At least the device mapping / dma_buf_attachment >>> side should be doable with just the pfn and the new dma-api? >> Yes, that would be my first goal post. Figure out some meta >> information and a container data structure that allows struct >> page-less P2P mapping through the new DMA API. >> >>>> I'm hoping we can get to something where we describe not just how the >>>> pfns should be DMA mapped, but also can describe how they should be >>>> CPU mapped. For instance that this PFN space is always mapped >>>> uncachable, in CPU and in IOMMU. >>> I was pondering whether dma_mmap and friends would be a good place to >>> prototype this and go for a fully generic implementation. But then even >>> those have _wc/_uncached variants. >> Given that the inability to correctly DMA map P2P MMIO without struct >> page is a current pain point and current source of hacks in dmabuf >> exporters, I wanted to make resolving that a priority. >> >> However, if you mean what I described above for "fully generic [dmabuf >> mmap] implementation", then we'd have the phyr datastructure as a >> dependency to attempt that work. >> >> phyr, and particularly the meta information, has a number of >> stakeholders. I was thinking of going first with rdma's memory >> registration flow because we are now pretty close to being able to do >> such a big change, and it can demonstrate most of the requirements. >> >> But that doesn't mean mmap couldn't go concurrently on the same agreed >> datastructure if people are interested. > Yeah cpu mmap needs a lot more, going with a very limited p2p use-case > first only makes sense. > >>>> We also have current bugs in the iommu/vfio side where we are fudging >>>> CC stuff, like assuming CPU memory is encrypted (not always true) and >>>> that MMIO is non-encrypted (not always true) >>> tbf CC pte flags I just don't grok at all. I've once tried to understand >>> what current exporters and gpu drivers do and just gave up. But that's >>> also a bit why I'm worried here because it's an enigma to me. >> For CC, inside the secure world, is some information if each PFN >> inside the VM is 'encrypted' or not. Any VM PTE (including the IOPTEs) >> pointing at the PFN must match the secure world's view of >> 'encrypted'. The VM can ask the secure world to change its view at >> runtime. >> >> The way CC has been bolted on to the kernel so far laregly hides this >> from drivers, so it is troubled to tell in driver code if the PFN you >> have is 'encrypted' or not. Right now the general rule (that is not >> always true) is that struct page CPU memory is encrypted and >> everything else is decrypted. >> >> So right now, you can mostly ignore it and the above assumption >> largely happens for you transparently. >> >> However, soon we will have encrypted P2P MMIO which will stress this >> hiding strategy. > It's already breaking with stuff like virtual gpu drivers, vmwgfx is > fiddling around with these bits (at least last I tried to understand this > all) and I think a few others do too. > >>>>> I thought iommuv2 (or whatever linux calls these) has full fault support >>>>> and could support current move semantics. But yeah for iommu without >>>>> fault support we need some kind of pin or a newly formalized revoke model. >>>> No, this is HW dependent, including PCI device, and I'm aware of no HW >>>> that fully implements this in a way that could be useful to implement >>>> arbitary move semantics for VFIO.. >>> Hm I thought we've had at least prototypes floating around of device fault >>> repair, but I guess that only works with ATS/pasid stuff and not general >>> iommu traffic from devices. Definitely needs some device cooperation since >>> the timeouts of a full fault are almost endless. >> Yes, exactly. What all real devices I'm aware have done is make a >> subset of their traffic work with ATS and PRI, but not all their >> traffic. Without *all* traffic you can't make any generic assumption >> in the iommu that a transient non-present won't be fatal to the >> device. >> >> Stuff like dmabuf move semantics rely on transient non-present being >> non-disruptive... > Ah now I get it, at the iommu level you have to pessimistically assume > whether a device can handle a fault, and none can for all traffic. I was > thinking too much about the driver level where generally the dma-buf you > importer are only used for the subset of device functions that can cope > with faults on many devices. > > Cheers, Sima

1 year

2
8
0 0

Re: [RFC PATCH 08/12] vfio/pci: Create host unaccessible dma-buf for private device

by Jason Gunthorpe

On Thu, Jan 23, 2025 at 03:41:58PM +0800, Xu Yilun wrote: > I don't have a complete idea yet. But the goal is not to make any > existing driver seamlessly work with secure device. It is to provide a > generic way for bind/attestation/accept, and may save driver's effort > if they don't care about this startup process. There are plenty of > operations that a driver can't do to a secure device, FLR is one of > them. The TDISP SPEC has described some general rules but some are even > device specific. You can FLR a secure device, it just has to be re-secured and re-attested after. Otherwise no VFIO for you. > So I think a driver (including VFIO) expects change to support trusted > device, but may not have to cover bind/attestation/accept flow. I expect changes, but not fundamental ones. VFIO will still have to FLR devices as part of it's security architecture. The entire flow needs to have options for drivers to be involved in the flow, somehow. Jason

1 year

1
0
0 0

Re: [RFC PATCH 01/12] dma-buf: Introduce dma_buf_get_pfn_unlocked() kAPI

by Jason Gunthorpe

On Wed, Jan 22, 2025 at 12:04:19PM +0100, Simona Vetter wrote: > I'm kinda leaning towards entirely separate dma-buf interfaces for the new > phyr stuff, because I fear that adding that to the existing ones will only > make the chaos worse. Lets see when some patches come up, if importers have to deal with several formats a single attach interface would probably be simpler for them.. > For the existing dma-buf importers/exporters I'm kinda hoping for a pure > dma_addr_t based list eventually. IMHO the least churn would be to have the old importers call a helper (perhaps implicitly in the core code) to convert phyr into a dma mapped scatterlist and then just keep using their existing code exactly as is. Performance improvement would come from importers switching to use the new dma api internally and avoiding the scatterlist allocation, but even the extra alocation is not so bad. Then, perhaps, and I really hesitate to say this, but perhaps to ease the migration we could store a dma mapped list in a phyr using the meta information. That would allow a dmabuf scatterlist exporter to be converted to a phyr with a helper. The importer would have to have a special path to detect the dma mapped mode and skip the normal mapping. The point would be to let maintained drivers use the new data structures and flows easily and still interoperate with older drivers. Otherwise we have to somehow build parallel scatterlist and phyr code flows into importers :\ > Going all the way to a phyr based > approach for everyone might be too much churn, there's some real bad cruft > there. It's not going to work for every case, but it covers a lot of them > and might be less pain for existing importers. I'd hope we can reach every case.. But I don't know what kind of horrors you guys have :) > But in theory it should be possible to use phyr everywhere eventually, as > long as there's no obviously api-rules-breaking way to go from a phyr back to > a struct page even when that exists. I'd say getting a struct page is a perfectly safe operation, from a phyr API perspective. The dmabuf issue seems to be entirely about following the locking rules, an importer is not allowed to get a struct page and switch from reservation locking to page refcount locking. I get the appeal for DRM of blocking struct page use because that directly prevents the above. But, in RDMA I want to re-use the same phyr datastructure for tracking pin_user_pages() CPU memory, and I must get the struct page back so I can put_user_page() it when done. Perhaps we can find some compromise where the phyr data structure has some kind of flag 'disable struct page' and then the phyr_entry_to_page() API will WARN_ON() or something like that. > > However, soon we will have encrypted P2P MMIO which will stress this > > hiding strategy. > > It's already breaking with stuff like virtual gpu drivers, vmwgfx is > fiddling around with these bits (at least last I tried to understand this > all) and I think a few others do too. IMHO, most places I've seen touching this out side arch code feel very hacky. :\ Jason

1 year

1
0
0 0

Re: [RFC PATCH 08/12] vfio/pci: Create host unaccessible dma-buf for private device

by Jason Gunthorpe

On Wed, Jan 22, 2025 at 12:32:56PM +0800, Xu Yilun wrote: > On Tue, Jan 21, 2025 at 01:43:03PM -0400, Jason Gunthorpe wrote: > > On Tue, Jun 25, 2024 at 05:12:10AM +0800, Xu Yilun wrote: > > > > > When VFIO works as a TEE user in VM, it means an attester (e.g. PCI > > > subsystem) has already moved the device to RUN state. So VFIO & DPDK > > > are all TEE users, no need to manipulate TDISP state between them. > > > AFAICS, this is the most preferred TIO usage in CoCo-VM. > > > > No, unfortunately. Part of the motivation to have the devices be > > unlocked when the VM starts is because there is an expectation that a > > driver in the VM will need to do untrusted operations to boot up the > > I assume these operations are device specific. Yes > > device before it can be switched to the run state. > > > > So any vfio use case needs to imagine that VFIO starts with an > > untrusted device, does stuff to it, then pushes everything through to > > I have concern that VFIO has to do device specific stuff. Our current > expectation is a specific device driver deals with the untrusted > operations, then user writes a 'bind' device sysfs node which detaches > the driver for untrusted, do the attestation and accept, and try match > the driver for trusted (e.g. VFIO). I don't see this as working, VFIO will FLR the device which will destroy anything that was done prior. VFIO itself has to do the sequence and the VFIO userspace has to contain the device specific stuff. The bind/unbind dance for untrusted->trusted would need to be internalized in VFIO without unbinding. The main motivation for the bind/unbind flow was to manage the DMA API, which VFIO does not use. Jason

1 year

1
0
0 0

Re: [RFC PATCH 08/12] vfio/pci: Create host unaccessible dma-buf for private device

by Jason Gunthorpe

On Tue, Jun 25, 2024 at 05:12:10AM +0800, Xu Yilun wrote: > When VFIO works as a TEE user in VM, it means an attester (e.g. PCI > subsystem) has already moved the device to RUN state. So VFIO & DPDK > are all TEE users, no need to manipulate TDISP state between them. > AFAICS, this is the most preferred TIO usage in CoCo-VM. No, unfortunately. Part of the motivation to have the devices be unlocked when the VM starts is because there is an expectation that a driver in the VM will need to do untrusted operations to boot up the device before it can be switched to the run state. So any vfio use case needs to imagine that VFIO starts with an untrusted device, does stuff to it, then pushes everything through to run. The exact mirror as what a kernel driver should be able to do. How exactly all this very complex stuff works, I have no idea, but this is what I've understood is the target. :\ Jason

1 year

1
0
0 0

Re: [RFC PATCH 01/12] dma-buf: Introduce dma_buf_get_pfn_unlocked() kAPI

by Jason Gunthorpe

On Tue, Jan 21, 2025 at 05:11:32PM +0100, Simona Vetter wrote: > On Mon, Jan 20, 2025 at 03:48:04PM -0400, Jason Gunthorpe wrote: > > On Mon, Jan 20, 2025 at 07:50:23PM +0100, Simona Vetter wrote: > > > On Mon, Jan 20, 2025 at 01:59:01PM -0400, Jason Gunthorpe wrote: > > > > On Mon, Jan 20, 2025 at 01:14:12PM +0100, Christian König wrote: > > > > What is going wrong with your email? You replied to Simona, but > > > > Simona Vetter <simona.vetter(a)ffwll.ch> is dropped from the To/CC > > > > list??? I added the address back, but seems like a weird thing to > > > > happen. > > > > > > Might also be funny mailing list stuff, depending how you get these. I > > > read mails over lore and pretty much ignore cc (unless it's not also on > > > any list, since those tend to be security issues) because I get cc'ed on > > > way too much stuff for that to be a useful signal. > > > > Oh I see, you are sending a Mail-followup-to header that excludes your > > address, so you don't get any emails at all.. My mutt is dropping you > > as well. > > > > > Yeah I'm not worried about cpu mmap locking semantics. drm/ttm is a pretty > > > clear example that you can implement dma-buf mmap with the rules we have, > > > except the unmap_mapping_range might need a bit fudging with a separate > > > address_space. > > > > From my perspective the mmap thing is a bit of a side/DRM-only thing > > as nothing I'm interested in wants to mmap dmabuf into a VMA. > > I guess we could just skip mmap on these pfn exporters, but it also means > a bit more boilerplate. I have been assuming that dmabuf mmap remains unchanged, that exporters will continue to implement that mmap() callback as today. My main interest has been what data structure is produced in the attach APIs. Eg today we have a struct dma_buf_attachment that returns a sg_table. I'm expecting some kind of new data structure, lets call it "physical list" that is some efficient coding of meta/addr/len tuples that works well with the new DMA API. Matthew has been calling this thing phyr.. So, I imagine, struct dma_buf_attachment gaining an optional feature negotiation and then we have in dma_buf_attachment: union { struct sg_table *sgt; struct physical_list *phyr; }; That's basicaly it, an alternative to scatterlist that has a clean architecture. Now, if you are asking if the current dmabuf mmap callback can be improved with the above? Maybe? phyr should have the neccessary information inside it to populate a VMA - eventually even fully correctly with all the right cachable/encrypted/forbidden/etc flags. So, you could imagine that exporters could just have one routine to generate the phyr list and that goes into the attachment, goes into some common code to fill VMA PTEs, and some other common code that will convert it into the DMABUF scatterlist. If performance is not a concern with these data structure conversions it could be an appealing simplification. And yes, I could imagine the meta information being descriptive enough to support the private interconnect cases, the common code could detect private meta information and just cleanly fail. > At least the device mapping / dma_buf_attachment > side should be doable with just the pfn and the new dma-api? Yes, that would be my first goal post. Figure out some meta information and a container data structure that allows struct page-less P2P mapping through the new DMA API. > > I'm hoping we can get to something where we describe not just how the > > pfns should be DMA mapped, but also can describe how they should be > > CPU mapped. For instance that this PFN space is always mapped > > uncachable, in CPU and in IOMMU. > > I was pondering whether dma_mmap and friends would be a good place to > prototype this and go for a fully generic implementation. But then even > those have _wc/_uncached variants. Given that the inability to correctly DMA map P2P MMIO without struct page is a current pain point and current source of hacks in dmabuf exporters, I wanted to make resolving that a priority. However, if you mean what I described above for "fully generic [dmabuf mmap] implementation", then we'd have the phyr datastructure as a dependency to attempt that work. phyr, and particularly the meta information, has a number of stakeholders. I was thinking of going first with rdma's memory registration flow because we are now pretty close to being able to do such a big change, and it can demonstrate most of the requirements. But that doesn't mean mmap couldn't go concurrently on the same agreed datastructure if people are interested. > > We also have current bugs in the iommu/vfio side where we are fudging > > CC stuff, like assuming CPU memory is encrypted (not always true) and > > that MMIO is non-encrypted (not always true) > > tbf CC pte flags I just don't grok at all. I've once tried to understand > what current exporters and gpu drivers do and just gave up. But that's > also a bit why I'm worried here because it's an enigma to me. For CC, inside the secure world, is some information if each PFN inside the VM is 'encrypted' or not. Any VM PTE (including the IOPTEs) pointing at the PFN must match the secure world's view of 'encrypted'. The VM can ask the secure world to change its view at runtime. The way CC has been bolted on to the kernel so far laregly hides this from drivers, so it is troubled to tell in driver code if the PFN you have is 'encrypted' or not. Right now the general rule (that is not always true) is that struct page CPU memory is encrypted and everything else is decrypted. So right now, you can mostly ignore it and the above assumption largely happens for you transparently. However, soon we will have encrypted P2P MMIO which will stress this hiding strategy. > > > I thought iommuv2 (or whatever linux calls these) has full fault support > > > and could support current move semantics. But yeah for iommu without > > > fault support we need some kind of pin or a newly formalized revoke model. > > > > No, this is HW dependent, including PCI device, and I'm aware of no HW > > that fully implements this in a way that could be useful to implement > > arbitary move semantics for VFIO.. > > Hm I thought we've had at least prototypes floating around of device fault > repair, but I guess that only works with ATS/pasid stuff and not general > iommu traffic from devices. Definitely needs some device cooperation since > the timeouts of a full fault are almost endless. Yes, exactly. What all real devices I'm aware have done is make a subset of their traffic work with ATS and PRI, but not all their traffic. Without *all* traffic you can't make any generic assumption in the iommu that a transient non-present won't be fatal to the device. Stuff like dmabuf move semantics rely on transient non-present being non-disruptive... Jason

1 year

1
0
0 0

Re: [RFC PATCH 01/12] dma-buf: Introduce dma_buf_get_pfn_unlocked() kAPI

by Jason Gunthorpe

On Mon, Jan 20, 2025 at 07:50:23PM +0100, Simona Vetter wrote: > On Mon, Jan 20, 2025 at 01:59:01PM -0400, Jason Gunthorpe wrote: > > On Mon, Jan 20, 2025 at 01:14:12PM +0100, Christian König wrote: > > What is going wrong with your email? You replied to Simona, but > > Simona Vetter <simona.vetter(a)ffwll.ch> is dropped from the To/CC > > list??? I added the address back, but seems like a weird thing to > > happen. > > Might also be funny mailing list stuff, depending how you get these. I > read mails over lore and pretty much ignore cc (unless it's not also on > any list, since those tend to be security issues) because I get cc'ed on > way too much stuff for that to be a useful signal. Oh I see, you are sending a Mail-followup-to header that excludes your address, so you don't get any emails at all.. My mutt is dropping you as well. > Yeah I'm not worried about cpu mmap locking semantics. drm/ttm is a pretty > clear example that you can implement dma-buf mmap with the rules we have, > except the unmap_mapping_range might need a bit fudging with a separate > address_space. From my perspective the mmap thing is a bit of a side/DRM-only thing as nothing I'm interested in wants to mmap dmabuf into a VMA. However, I think if you have locking rules that can fit into a VMA fault path and link move_notify to unmap_mapping_range() then you've got a pretty usuable API. > For cpu mmaps I'm more worried about the arch bits in the pte, stuff like > caching mode or encrypted memory bits and things like that. There's > vma->vm_pgprot, but it's a mess. But maybe this all is an incentive to > clean up that mess a bit. I'm convinced we need meta-data along with pfns, there is too much stuff that needs more information than just the address. Cachability, CC encryption, exporting device, etc. This is a topic to partially cross when we talk about how to fully remove struct page requirements from the new DMA API. I'm hoping we can get to something where we describe not just how the pfns should be DMA mapped, but also can describe how they should be CPU mapped. For instance that this PFN space is always mapped uncachable, in CPU and in IOMMU. We also have current bugs in the iommu/vfio side where we are fudging CC stuff, like assuming CPU memory is encrypted (not always true) and that MMIO is non-encrypted (not always true) > I thought iommuv2 (or whatever linux calls these) has full fault support > and could support current move semantics. But yeah for iommu without > fault support we need some kind of pin or a newly formalized revoke model. No, this is HW dependent, including PCI device, and I'm aware of no HW that fully implements this in a way that could be useful to implement arbitary move semantics for VFIO.. Jason

1 year

1
0
0 0

Re: [RFC PATCH 01/12] dma-buf: Introduce dma_buf_get_pfn_unlocked() kAPI

by Christian König

Am 17.01.25 um 15:42 schrieb Simona Vetter: > On Wed, Jan 15, 2025 at 11:06:53AM +0100, Christian König wrote: >> [SNIP] >>> Anything missing? >> Well as far as I can see this use case is not a good fit for the DMA-buf >> interfaces in the first place. DMA-buf deals with devices and buffer >> exchange. >> >> What's necessary here instead is to give an importing VM full access on some >> memory for their specific use case. >> >> That full access includes CPU and DMA mappings, modifying caching >> attributes, potentially setting encryption keys for specific ranges etc.... >> etc... >> >> In other words we have a lot of things the importer here should be able to >> do which we don't want most of the DMA-buf importers to do. > This proposal isn't about forcing existing exporters to allow importers to > do new stuff. That stays as-is, because it would break things. > > It's about adding yet another interface to get at the underlying data, and > we have tons of those already. The only difference is that if we don't > butcher the design, we'll be able to implement all the existing dma-buf > interfaces on top of this new pfn interface, for some neat maximal > compatibility. That sounds like you missed my concern. When an exporter and an importer agree that they want to exchange PFNs instead of DMA addresses then that is perfectly fine. The problems start when you define the semantics how those PFNs, DMA address, private bus addresses, whatever is echanged different to what we have documented for DMA-buf. This semantics is very well defined for DMA-buf now, because that is really important or otherwise things usually seem to work under testing (e.g. without memory pressure) and then totally fall apart in production environments. In other words we have defined what lock you need to hold when calling functions, what a DMA fence is, when exchanged addresses are valid etc... > But fundamentally there's never been an expectation that you can take any > arbitrary dma-buf and pass it any arbitrary importer, and that is must > work. The fundamental promise is that if it _does_ work, then > - it's zero copy > - and fast, or as fast as we can make it > > I don't see this any different than all the much more specific prposals > and existing code, where a subset of importers/exporters have special > rules so that e.g. gpu interconnect or vfio uuid based sharing works. > pfn-based sharing is just yet another flavor that exists to get the max > amount of speed out of interconnects. Please take another look at what is proposed here. The function is called dma_buf_get_pfn_*unlocked* ! This is not following DMA-buf semantics for exchanging addresses and keeping them valid, but rather something more like userptrs. Inserting PFNs into CPU (or probably also IOMMU) page tables have a different semantics than what DMA-buf usually does, because as soon as the address is written into the page table it is made public. So you need some kind of mechanism to make sure that this addr you made public stays valid as long as it is public. The usual I/O operation we encapsulate with DMA-fences have a fundamentally different semantics because we have the lock which enforces that stuff stays valid and then have a DMA-fence which notes how long the stuff needs to stay valid for an operation to complete. Regards, Christian. > > Cheers, Sima > >> The semantics for things like pin vs revocable vs dynamic/moveable seems >> similar, but that's basically it. >> >> As far as I know the TEE subsystem also represents their allocations as file >> descriptors. If I'm not completely mistaken this use case most likely fit's >> better there. >> >>> I feel like this is small enough that m-l archives is good enough. For >>> some of the bigger projects we do in graphics we sometimes create entries >>> in our kerneldoc with wip design consensus and things like that. But >>> feels like overkill here. >>> >>>> My general desire is to move all of RDMA's MR process away from >>>> scatterlist and work using only the new DMA API. This will save *huge* >>>> amounts of memory in common workloads and be the basis for non-struct >>>> page DMA support, including P2P. >>> Yeah a more memory efficient structure than the scatterlist would be >>> really nice. That would even benefit the very special dma-buf exporters >>> where you cannot get a pfn and only the dma_addr_t, altough most of those >>> (all maybe even?) have contig buffers, so your scatterlist has only one >>> entry. But it would definitely be nice from a design pov. >> Completely agree on that part. >> >> Scatterlist have a some design flaws, especially mixing the input and out >> parameters of the DMA API into the same structure. >> >> Additional to that DMA addresses are basically missing which bus they belong >> to and details how the access should be made (e.g. snoop vs no-snoop >> etc...). >> >>> Aside: A way to more efficiently create compressed scatterlists would be >>> neat too, because a lot of drivers hand-roll that and it's a bit brittle >>> and kinda silly to duplicate. With compressed I mean just a single entry >>> for a contig range, in practice thanks to huge pages/folios and allocators >>> trying to hand out contig ranges if there's plenty of memory that saves a >>> lot of memory too. But currently it's a bit a pain to construct these >>> efficiently, mostly it's just a two-pass approach and then trying to free >>> surplus memory or krealloc to fit. Also I don't have good ideas here, but >>> dma-api folks might have some from looking at too many things that create >>> scatterlists. >> I mailed with Christoph about that a while back as well and we both agreed >> that it would probably be a good idea to start defining a data structure to >> better encapsulate DMA addresses. >> >> It's just that nobody had time for that yet and/or I wasn't looped in in the >> final discussion about it. >> >> Regards, >> Christian. >> >>> -Sima

1 year

2
1
0 0

Re: [RFC PATCH 01/12] dma-buf: Introduce dma_buf_get_pfn_unlocked() kAPI

by Christian König

Am 21.06.24 um 00:02 schrieb Xu Yilun: > On Thu, Jan 16, 2025 at 04:13:13PM +0100, Christian König wrote: >> Am 15.01.25 um 18:09 schrieb Jason Gunthorpe: >> >> On Wed, Jan 15, 2025 at 05:34:23PM +0100, Christian König wrote: >> >> Granted, let me try to improve this. >> Here is a real world example of one of the issues we ran into and why >> CPU mappings of importers are redirected to the exporter. >> We have a good bunch of different exporters who track the CPU mappings >> of their backing store using address_space objects in one way or >> another and then uses unmap_mapping_range() to invalidate those CPU >> mappings. >> But when importers get the PFNs of the backing store they can look >> behind the curtain and directly insert this PFN into the CPU page >> tables. >> We had literally tons of cases like this where drivers developers cause >> access after free issues because the importer created a CPU mappings on >> their own without the exporter knowing about it. >> This is just one example of what we ran into. Additional to that >> basically the whole synchronization between drivers was overhauled as >> well because we found that we can't trust importers to always do the >> right thing. >> >> But this, fundamentally, is importers creating attachments and then >> *ignoring the lifetime rules of DMABUF*. If you created an attachment, >> got a move and *ignored the move* because you put the PFN in your own >> VMA, then you are not following the attachment lifetime rules! >> >> Move notify is solely for informing the importer that they need to >> re-fresh their DMA mappings and eventually block for ongoing DMA to end. >> >> This semantics doesn't work well for CPU mappings because you need to hold >> the reservation lock to make sure that the information stay valid and you >> can't hold a lock while returning from a page fault. > Dealing with CPU mapping and resource invalidation is a little hard, but is > resolvable, by using other types of locks. And I guess for now dma-buf > exporters should always handle this CPU mapping VS. invalidation contention if > they support mmap(). > > It is resolvable so with some invalidation notify, a decent importers could > also handle the contention well. That doesn't work like this. See page tables updates under DMA-buf works by using the same locking approach for both the validation and invalidation side. In other words we hold the same lock while inserting and removing entries into/from the page tables. That this here should be an unlocked API means that can only use it with pre-allocated and hard pinned memory without any chance to invalidate it while running. Otherwise you can never be sure of the validity of the address information you got from the exporter. > IIUC now the only concern is importer device drivers are easier to do > something wrong, so move CPU mapping things to exporter. But most of the > exporters are also device drivers, why they are smarter? Exporters always use their invalidation code path no matter if they are exporting their buffers for other to use or if they are stand alone. If you do the invalidation on the importer side you always need both exporter and importer around to test it. Additional to that we have much more importers than exporters. E.g. a lot of simple drivers only import DMA-heap buffers and never exports anything. > And there are increasing mapping needs, today exporters help handle CPU primary > mapping, tomorrow should they also help on all other mappings? Clearly it is > not feasible. So maybe conditionally give trust to some importers. Why should that be necessary? Exporters *must* know what somebody does with their buffers. If you have an use case the exporter doesn't support in their mapping operation then that use case most likely doesn't work in the first place. For example direct I/O is enabled/disabled by exporters on their CPU mappings based on if that works correctly for them. And importer simply doesn't know if they should use vm_insert_pfn() or vm_insert_page(). We could of course implement that logic into each importer to chose between the different approaches, but than each importer gains logic it only exercises with a specific exporter. And that doesn't seem to be a good idea at all. Regards, Christian. > > Thanks, > Yilun

1 year

1
0
0 0

Re: [PATCH v5 2/2] i2c: i2c-qcom-geni: Add Block event interrupt support

by kernel test robot

Hi Jyothi, kernel test robot noticed the following build warnings: [auto build test WARNING on 55bcd2e0d04c1171d382badef1def1fd04ef66c5] url: https://github.com/intel-lab-lkp/linux/commits/Jyothi-Kumar-Seerapu/dmaengi… base: 55bcd2e0d04c1171d382badef1def1fd04ef66c5 patch link: https://lore.kernel.org/r/20250120095753.25539-3-quic_jseerapu%40quicinc.com patch subject: [PATCH v5 2/2] i2c: i2c-qcom-geni: Add Block event interrupt support config: arc-randconfig-001-20250120 (https://download.01.org/0day-ci/archive/20250120/202501202159.wLRVO16t-lkp@…) compiler: arceb-elf-gcc (GCC) 13.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250120/202501202159.wLRVO16t-lkp@…) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp(a)intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202501202159.wLRVO16t-lkp@intel.com/ All warnings (new ones prefixed by >>): >> drivers/i2c/busses/i2c-qcom-geni.c:599: warning: Excess function parameter 'dev' description in 'geni_i2c_gpi_multi_desc_unmap' Kconfig warnings: (for reference only) WARNING: unmet direct dependencies detected for OMAP2PLUS_MBOX Depends on [n]: MAILBOX [=y] && (ARCH_OMAP2PLUS || ARCH_K3) Selected by [m]: - TI_K3_M4_REMOTEPROC [=m] && REMOTEPROC [=y] && (ARCH_K3 || COMPILE_TEST [=y]) vim +599 drivers/i2c/busses/i2c-qcom-geni.c 589 590 /** 591 * geni_i2c_gpi_multi_desc_unmap() - unmaps the buffers post multi message TX transfers 592 * @dev: pointer to the corresponding dev node 593 * @gi2c: i2c dev handle 594 * @msgs: i2c messages array 595 * @peripheral: pointer to the gpi_i2c_config 596 */ 597 static void geni_i2c_gpi_multi_desc_unmap(struct geni_i2c_dev *gi2c, struct i2c_msg msgs[], 598 struct gpi_i2c_config *peripheral) > 599 { 600 u32 msg_xfer_cnt, wr_idx = 0; 601 struct geni_i2c_gpi_multi_desc_xfer *tx_multi_xfer = &gi2c->i2c_multi_desc_config; 602 603 /* 604 * In error case, need to unmap all messages based on the msg_idx_cnt. 605 * Non-error case unmap all the processed messages. 606 */ 607 if (gi2c->err) 608 msg_xfer_cnt = tx_multi_xfer->msg_idx_cnt; 609 else 610 msg_xfer_cnt = tx_multi_xfer->irq_cnt * QCOM_I2C_GPI_NUM_MSGS_PER_IRQ; 611 612 /* Unmap the processed DMA buffers based on the received interrupt count */ 613 for (; tx_multi_xfer->unmap_msg_cnt < msg_xfer_cnt; tx_multi_xfer->unmap_msg_cnt++) { 614 if (tx_multi_xfer->unmap_msg_cnt == gi2c->num_msgs) 615 break; 616 wr_idx = tx_multi_xfer->unmap_msg_cnt % QCOM_I2C_GPI_MAX_NUM_MSGS; 617 geni_i2c_gpi_unmap(gi2c, &msgs[tx_multi_xfer->unmap_msg_cnt], 618 tx_multi_xfer->dma_buf[wr_idx], 619 tx_multi_xfer->dma_addr[wr_idx], 620 NULL, (dma_addr_t)NULL); 621 tx_multi_xfer->freed_msg_cnt++; 622 } 623 } 624 -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 year

1
0
0 0

Re: [RFC PATCH 08/12] vfio/pci: Create host unaccessible dma-buf for private device

by Jason Gunthorpe

On Mon, Jun 24, 2024 at 03:59:53AM +0800, Xu Yilun wrote: > > But it also seems to me that VFIO should be able to support putting > > the device into the RUN state > > Firstly I think VFIO should support putting device into *LOCKED* state. > From LOCKED to RUN, there are many evidence fetching and attestation > things that only guest cares. I don't think VFIO needs to opt-in. VFIO is not just about running VMs. If someone wants to run DPDK on VFIO they should be able to get the device into a RUN state and work with secure memory without requiring a KVM. Yes there are many steps to this, but we should imagine how it can work. > > without involving KVM or cVMs. > > It may not be feasible for all vendors. It must be. A CC guest with an in kernel driver can definately get the PCI device into RUN, so VFIO running in the guest should be able as well. > I believe AMD would have one firmware call that requires cVM handle > *AND* move device into LOCKED state. It really depends on firmware > implementation. IMHO, you would not use the secure firmware if you are not using VMs. > Yes, the secure EPT is in the secure world and managed by TDX firmware. > Now a SW Mirror Secure EPT is introduced in KVM and managed by KVM > directly, and KVM will finally use firmware calls to propagate Mirror > Secure EPT changes to secure EPT. If the secure world managed it then the secure world can have rules that work with the IOMMU as well.. Jason

1 year

1
0
0 0

[PATCH v2] Documentation: dma-buf: heaps: Add heap name definitions

by Maxime Ripard

Following a recent discussion at last Plumbers, John Stultz, Sumit Sewal, TJ Mercier and I came to an agreement that we should document what the dma-buf heaps names are expected to be, and what the buffers attributes you'll get should be documented. Let's create that doc to make sure those attributes and names are guaranteed going forward. Signed-off-by: Maxime Ripard <mripard(a)kernel.org> --- Changes from v1: - Add the mention that the cma / reserved heap is optional. To: Jonathan Corbet <corbet(a)lwn.net> To: Sumit Semwal <sumit.semwal(a)linaro.org> Cc: Benjamin Gaignard <benjamin.gaignard(a)collabora.com> Cc: Brian Starkey <Brian.Starkey(a)arm.com> Cc: John Stultz <jstultz(a)google.com> Cc: "T.J. Mercier" <tjmercier(a)google.com> Cc: "Christian König" <christian.koenig(a)amd.com> Cc: dri-devel(a)lists.freedesktop.org Cc: linaro-mm-sig(a)lists.linaro.org Cc: linux-media(a)vger.kernel.org Cc: linux-doc(a)vger.kernel.org --- Documentation/userspace-api/dma-buf-heaps.rst | 76 +++++++++++++++++++ Documentation/userspace-api/index.rst | 1 + 2 files changed, 77 insertions(+) create mode 100644 Documentation/userspace-api/dma-buf-heaps.rst diff --git a/Documentation/userspace-api/dma-buf-heaps.rst b/Documentation/userspace-api/dma-buf-heaps.rst new file mode 100644 index 000000000000..68be7ddea150 --- /dev/null +++ b/Documentation/userspace-api/dma-buf-heaps.rst @@ -0,0 +1,76 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============================== +Allocating dma-buf using heaps +============================== + +Dma-buf Heaps are a way for userspace to allocate dma-buf objects. They are +typically used to allocate buffers from a specific allocation pool, or to share +buffers across frameworks. + +Heaps +===== + +A heap represent a specific allocator. The Linux kernel currently supports the +following heaps: + + - The ``system`` heap allocates virtually contiguous, cacheable, buffers + + - The ``reserved`` heap allocates physically contiguous, cacheable, + buffers. Only present if a CMA region is present. Such a region is + usually created either through the kernel commandline through the + `cma` parameter, a memory region Device-Tree node with the + `linux,cma-default` property set, or through the `CMA_SIZE_MBYTES` or + `CMA_SIZE_PERCENTAGE` Kconfig options. Depending on the platform, it + might be called differently: + + - Acer Iconia Tab A500: ``linux,cma`` + - Allwinner sun4i, sun5i and sun7i families: ``default-pool`` + - Amlogic A1: ``linux,cma`` + - Amlogic G12A/G12B/SM1: ``linux,cma`` + - Amlogic GXBB/GXL: ``linux,cma`` + - ASUS EeePad Transformer TF101: ``linux,cma`` + - ASUS Google Nexus 7 (Project Bach / ME370TG) E1565: ``linux,cma`` + - ASUS Google Nexus 7 (Project Nakasi / ME370T) E1565: ``linux,cma`` + - ASUS Google Nexus 7 (Project Nakasi / ME370T) PM269: ``linux,cma`` + - Asus Transformer Infinity TF700T: ``linux,cma`` + - Asus Transformer Pad 3G TF300TG: ``linux,cma`` + - Asus Transformer Pad TF300T: ``linux,cma`` + - Asus Transformer Pad TF701T: ``linux,cma`` + - Asus Transformer Prime TF201: ``linux,cma`` + - ASUS Vivobook S 15: ``linux,cma`` + - Cadence KC705: ``linux,cma`` + - Digi International ConnectCore 6UL: ``linux,cma`` + - Freescale i.MX8DXL EVK: ``linux,cma`` + - Freescale TQMa8Xx: ``linux,cma`` + - Hisilicon Hikey: ``linux,cma`` + - Lenovo ThinkPad T14s Gen 6: ``linux,cma`` + - Lenovo ThinkPad X13s: ``linux,cma`` + - Lenovo Yoga Slim 7x: ``linux,cma`` + - LG Optimus 4X HD P880: ``linux,cma`` + - LG Optimus Vu P895: ``linux,cma`` + - Loongson 2k0500, 2k1000 and 2k2000: ``linux,cma`` + - Microsoft Romulus: ``linux,cma`` + - NXP i.MX8ULP EVK: ``linux,cma`` + - NXP i.MX93 9x9 QSB: ``linux,cma`` + - NXP i.MX93 11X11 EVK: ``linux,cma`` + - NXP i.MX93 14X14 EVK: ``linux,cma`` + - NXP i.MX95 19X19 EVK: ``linux,cma`` + - Ouya Game Console: ``linux,cma`` + - Pegatron Chagall: ``linux,cma`` + - PHYTEC phyCORE-AM62A SOM: ``linux,cma`` + - PHYTEC phyCORE-i.MX93 SOM: ``linux,cma`` + - Qualcomm SC8280XP CRD: ``linux,cma`` + - Qualcomm X1E80100 CRD: ``linux,cma`` + - Qualcomm X1E80100 QCP: ``linux,cma`` + - RaspberryPi: ``linux,cma`` + - Texas Instruments AM62x SK board family: ``linux,cma`` + - Texas Instruments AM62A7 SK: ``linux,cma`` + - Toradex Apalis iMX8: ``linux,cma`` + - TQ-Systems i.MX8MM TQMa8MxML: ``linux,cma`` + - TQ-Systems i.MX8MN TQMa8MxNL: ``linux,cma`` + - TQ-Systems i.MX8MPlus TQMa8MPxL: ``linux,cma`` + - TQ-Systems i.MX8MQ TQMa8MQ: ``linux,cma`` + - TQ-Systems i.MX93 TQMa93xxLA/TQMa93xxCA SOM: ``linux,cma`` + - TQ-Systems MBA6ULx Baseboard: ``linux,cma`` + diff --git a/Documentation/userspace-api/index.rst b/Documentation/userspace-api/index.rst index 274cc7546efc..4901ce7c6cb7 100644 --- a/Documentation/userspace-api/index.rst +++ b/Documentation/userspace-api/index.rst @@ -41,10 +41,11 @@ Devices and I/O .. toctree:: :maxdepth: 1 accelerators/ocxl + dma-buf-heaps dma-buf-alloc-exchange gpio/index iommufd media/index dcdbas -- 2.47.1

1 year

3
6
0 0

Re: [RFC PATCH 08/12] vfio/pci: Create host unaccessible dma-buf for private device

by Jason Gunthorpe

On Fri, Jan 17, 2025 at 09:57:40AM +0800, Baolu Lu wrote: > On 1/15/25 21:01, Jason Gunthorpe wrote: > > On Wed, Jan 15, 2025 at 11:57:05PM +1100, Alexey Kardashevskiy wrote: > > > On 15/1/25 00:35, Jason Gunthorpe wrote: > > > > On Tue, Jun 18, 2024 at 07:28:43AM +0800, Xu Yilun wrote: > > > > > > > > > > is needed so the secure world can prepare anything it needs prior to > > > > > > starting the VM. > > > > > OK. From Dan's patchset there are some touch point for vendor tsm > > > > > drivers to do secure world preparation. e.g. pci_tsm_ops::probe(). > > > > > > > > > > Maybe we could move to Dan's thread for discussion. > > > > > > > > > > https://lore.kernel.org/linux- > > > > > coco/173343739517.1074769.13134786548545925484.stgit@dwillia2- > > > > > xfh.jf.intel.com/ > > > > I think Dan's series is different, any uapi from that series should > > > > not be used in the VMM case. We need proper vfio APIs for the VMM to > > > > use. I would expect VFIO to be calling some of that infrastructure. > > > Something like this experiment? > > > > > > https://github.com/aik/linux/commit/ > > > ce052512fb8784e19745d4cb222e23cabc57792e > > Yeah, maybe, though I don't know which of vfio/iommufd/kvm should be > > hosting those APIs, the above does seem to be a reasonable direction. > > > > When the various fds are closed I would expect the kernel to unbind > > and restore the device back. > > I am curious about the value of tsm binding against an iomnufd_vdevice > instead of the physical iommufd_device. Interesting question > It is likely that the kvm pointer should be passed to iommufd during the > creation of a viommu object. Yes, I fully expect this > If my recollection is correct, the arm > smmu-v3 needs it to obtain the vmid to setup the userspace event queue: Right now it will use a VMID unrelated to KVM. BTM support on ARM will require syncing the VMID with KVM. AMD and Intel may require the KVM for some reason as well. For CC I'm expecting the KVM fd to be the handle for the cVM, so any RPCs that want to call into the secure world need the KVM FD to get the cVM's identifier. Ie a "bind to cVM" RPC will need the PCI information and the cVM's handle. From that perspective it does make sense that any cVM related APIs, like "bind to cVM" would be against the VDEVICE where we have a link to the VIOMMU which has the KVM. On the iommufd side the VIOMMU is part of the object hierarchy, but does not necessarily have to force a vIOMMU to appear in the cVM. But it also seems to me that VFIO should be able to support putting the device into the RUN state without involving KVM or cVMs. > Intel TDX connect implementation also needs a reference to the kvm > pointer to obtain the secure EPT information. This is crucial because > the CPU's page table must be shared with the iommu. I thought kvm folks were NAKing this sharing entirely? Or is the secure EPT in the secure world and not directly managed by Linux? AFAIK AMD is going to mirror the iommu page table like today. ARM, I suspect, will not have an "EPT" under Linux control, so whatever happens will be hidden in their secure world. Jason

1 year

1
0
0 0

Re: [RFC PATCH 01/12] dma-buf: Introduce dma_buf_get_pfn_unlocked() kAPI

by Christian König

Am 08.01.25 um 20:22 schrieb Xu Yilun: > On Wed, Jan 08, 2025 at 07:44:54PM +0100, Simona Vetter wrote: >> On Wed, Jan 08, 2025 at 12:22:27PM -0400, Jason Gunthorpe wrote: >>> On Wed, Jan 08, 2025 at 04:25:54PM +0100, Christian König wrote: >>>> Am 08.01.25 um 15:58 schrieb Jason Gunthorpe: >>>>> I have imagined a staged approach were DMABUF gets a new API that >>>>> works with the new DMA API to do importer mapping with "P2P source >>>>> information" and a gradual conversion. >>>> To make it clear as maintainer of that subsystem I would reject such a step >>>> with all I have. >>> This is unexpected, so you want to just leave dmabuf broken? Do you >>> have any plan to fix it, to fix the misuse of the DMA API, and all >>> the problems I listed below? This is a big deal, it is causing real >>> problems today. >>> >>> If it going to be like this I think we will stop trying to use dmabuf >>> and do something simpler for vfio/kvm/iommufd :( >> As the gal who help edit the og dma-buf spec 13 years ago, I think adding >> pfn isn't a terrible idea. By design, dma-buf is the "everything is >> optional" interface. And in the beginning, even consistent locking was >> optional, but we've managed to fix that by now :-/ Well you were also the person who mangled the struct page pointers in the scatterlist because people were abusing this and getting a bloody nose :) >> Where I do agree with Christian is that stuffing pfn support into the >> dma_buf_attachment interfaces feels a bit much wrong. > So it could a dmabuf interface like mmap/vmap()? I was also wondering > about that. But finally I start to use dma_buf_attachment interface > because of leveraging existing buffer pin and move_notify. Exactly that's the point, sharing pfn doesn't work with the pin and move_notify interfaces because of the MMU notifier approach Sima mentioned. >>>> We have already gone down that road and it didn't worked at all and >>>> was a really big pain to pull people back from it. >>> Nobody has really seriously tried to improve the DMA API before, so I >>> don't think this is true at all. >> Aside, I really hope this finally happens! Sorry my fault. I was not talking about the DMA API, but rather that people tried to look behind the curtain of DMA-buf backing stores. In other words all the fun we had with scatterlists and that people try to modify the struct pages inside of them. Improving the DMA API is something I really really hope for as well. >>>>> 3) Importing devices need to know if they are working with PCI P2P >>>>> addresses during mapping because they need to do things like turn on >>>>> ATS on their DMA. As for multi-path we have the same hacks inside mlx5 >>>>> today that assume DMABUFs are always P2P because we cannot determine >>>>> if things are P2P or not after being DMA mapped. >>>> Why would you need ATS on PCI P2P and not for system memory accesses? >>> ATS has a significant performance cost. It is mandatory for PCI P2P, >>> but ideally should be avoided for CPU memory. >> Huh, I didn't know that. And yeah kinda means we've butchered the pci p2p >> stuff a bit I guess ... Hui? Why should ATS be mandatory for PCI P2P? We have tons of production systems using PCI P2P without ATS. And it's the first time I hear that. >>>>> 5) iommufd and kvm are both using CPU addresses without DMA. No >>>>> exporter mapping is possible >>>> We have customers using both KVM and XEN with DMA-buf, so I can clearly >>>> confirm that this isn't true. >>> Today they are mmaping the dma-buf into a VMA and then using KVM's >>> follow_pfn() flow to extract the CPU pfn from the PTE. Any mmapable >>> dma-buf must have a CPU PFN. >>> >>> Here Xu implements basically the same path, except without the VMA >>> indirection, and it suddenly not OK? Illogical. >> So the big difference is that for follow_pfn() you need mmu_notifier since >> the mmap might move around, whereas with pfn smashed into >> dma_buf_attachment you need dma_resv_lock rules, and the move_notify >> callback if you go dynamic. >> >> So I guess my first question is, which locking rules do you want here for >> pfn importers? > follow_pfn() is unwanted for private MMIO, so dma_resv_lock. As Sima explained you either have follow_pfn() and mmu_notifier or you have DMA addresses and dma_resv lock / dma_fence. Just giving out PFNs without some lifetime associated with them is one of the major problems we faced before and really not something you can do. >> If mmu notifiers is fine, then I think the current approach of follow_pfn >> should be ok. But if you instead dma_resv_lock rules (or the cpu mmap >> somehow is an issue itself), then I think the clean design is create a new > cpu mmap() is an issue, this series is aimed to eliminate userspace > mapping for private MMIO resources. Why? >> separate access mechanism just for that. It would be the 5th or so (kernel >> vmap, userspace mmap, dma_buf_attach and driver private stuff like >> virtio_dma_buf.c where you access your buffer with a uuid), so really not >> a big deal. > OK, will think more about that. Please note that we have follow_pfn() + mmu_notifier working for KVM/XEN with MMIO mappings and P2P. And that required exactly zero DMA-buf changes :) I don't fully understand your use case, but I think it's quite likely that we already have that working. Regards, Christian. > > Thanks, > Yilun > >> And for non-contrived exporters we might be able to implement the other >> access methods in terms of the pfn method generically, so this wouldn't >> even be a terrible maintenance burden going forward. And meanwhile all the >> contrived exporters just keep working as-is. >> >> The other part is that cpu mmap is optional, and there's plenty of strange >> exporters who don't implement. But you can dma map the attachment into >> plenty devices. This tends to mostly be a thing on SoC devices with some >> very funky memory. But I guess you don't care about these use-case, so >> should be ok. >> >> I couldn't come up with a good name for these pfn users, maybe >> dma_buf_pfn_attachment? This does _not_ have a struct device, but maybe >> some of these new p2p source specifiers (or a list of those which are >> allowed, no idea how this would need to fit into the new dma api). >> >> Cheers, Sima >> -- >> Simona Vetter >> Software Engineer, Intel Corporation >> http://blog.ffwll.ch

1 year

2
14
0 0

Re: [RFC PATCH] driver: dma-buf: use vmf_insert_page for cma_heap_vm_fault

by Christian König

Am 16.01.25 um 02:46 schrieb Zhaoyang Huang: > On Wed, Jan 15, 2025 at 7:49 PM Christian König > <christian.koenig(a)amd.com> wrote: >> Am 15.01.25 um 07:18 schrieb zhaoyang.huang: >>> From: Zhaoyang Huang<zhaoyang.huang(a)unisoc.com> >>> >>> When using dma-buf as memory pool for VMM. The vmf_insert_pfn will >>> apply PTE_SPECIAL on pte which have vm_normal_page report bad_pte and >>> return NULL. This commit would like to suggest to replace >>> vmf_insert_pfn by vmf_insert_page. >> Setting PTE_SPECIAL is completely intentional here to prevent >> get_user_pages() from working on DMA-buf mappings. > ok. May I ask the reason? Drivers using this interface own the backing store for their specific use cases. There are a couple of things get_user_pages(), pin_user_pages(), direct I/O etc.. do which usually clash with those use cases. So that is intentionally completely disabled. We have the possibility to create a DMA-buf from memfd object and you can then do direct I/O to the memfd and still use the DMA-buf with GPUs or V4L for example. >> So absolutely clear NAK to this patch here. >> >> What exactly are you trying to do? > I would like to have pkvm have guest kernel be faulted of its second > stage page fault(ARM64's memory virtualization method) on dma-buf > which use pin_user_pages. Yeah, exactly that's one of the use case which we intentionally prevent here. The backing store drivers use don't care about the pin count of the memory and happily give it back to memory pools and/or swap it with device local memory if necessary. When this happens the ARM VM wouldn't be informed of the change and potentially accesses the wrong address. So sorry, but this approach won't work. You could try with the memfd+DMA-buf approach I mentioned earlier, but that won't give you all functionality on all DMA-buf supporting devices. For example GPUs usually can't scan out to a monitor from such buffers because of hardware limitations. Regards, Christian. >> Regards, >> Christian. >> >>> [ 103.402787] kvm [5276]: gfn(ipa)=0x80000 hva=0x7d4a400000 write_fault=0 >>> [ 103.403822] BUG: Bad page map in process crosvm_vcpu0 pte:168000140000f43 pmd:8000000c1ca0003 >>> [ 103.405144] addr:0000007d4a400000 vm_flags:040400fb anon_vma:0000000000000000 mapping:ffffff8085163df0 index:0 >>> [ 103.406536]file:dmabuf fault:cma_heap_vm_fault [cma_heap] mmap:dma_buf_mmap_internal read_folio:0x0 >>> [ 103.407877] CPU: 3 PID: 5276 Comm: crosvm_vcpu0 Tainted: G W OE 6.6.46-android15-8-g8bab72b63c20-dirty-4k #1 1e474a12dac4553a3ebba3a911f3b744176a5d2d >>> [ 103.409818] Hardware name: Unisoc UMS9632-base Board (DT) >>> [ 103.410613] Call trace: >>> [ 103.411038] dump_backtrace+0xf4/0x140 >>> [ 103.411641] show_stack+0x20/0x30 >>> [ 103.412184] dump_stack_lvl+0x60/0x84 >>> [ 103.412766] dump_stack+0x18/0x24 >>> [ 103.413304] print_bad_pte+0x1b8/0x1cc >>> [ 103.413909] vm_normal_page+0xc8/0xd0 >>> [ 103.414491] follow_page_pte+0xb0/0x304 >>> [ 103.415096] follow_page_mask+0x108/0x240 >>> [ 103.415721] __get_user_pages+0x168/0x4ac >>> [ 103.416342] __gup_longterm_locked+0x15c/0x864 >>> [ 103.417023] pin_user_pages+0x70/0xcc >>> [ 103.417609] pkvm_mem_abort+0xf8/0x5c0 >>> [ 103.418207] kvm_handle_guest_abort+0x3e0/0x3e4 >>> [ 103.418906] handle_exit+0xac/0x33c >>> [ 103.419472] kvm_arch_vcpu_ioctl_run+0x48c/0x8d8 >>> [ 103.420176] kvm_vcpu_ioctl+0x504/0x5bc >>> [ 103.420785] __arm64_sys_ioctl+0xb0/0xec >>> [ 103.421401] invoke_syscall+0x60/0x11c >>> [ 103.422000] el0_svc_common+0xb4/0xe8 >>> [ 103.422590] do_el0_svc+0x24/0x30 >>> [ 103.423131] el0_svc+0x3c/0x70 >>> [ 103.423640] el0t_64_sync_handler+0x68/0xbc >>> [ 103.424288] el0t_64_sync+0x1a8/0x1ac >>> >>> Signed-off-by: Xiwei Wang<xiwei.wang1(a)unisoc.com> >>> Signed-off-by: Aijun Sun<aijun.sun(a)unisoc.com> >>> Signed-off-by: Zhaoyang Huang<zhaoyang.huang(a)unisoc.com> >>> --- >>> drivers/dma-buf/heaps/cma_heap.c | 2 +- >>> 1 file changed, 1 insertion(+), 1 deletion(-) >>> >>> diff --git a/drivers/dma-buf/heaps/cma_heap.c b/drivers/dma-buf/heaps/cma_heap.c >>> index c384004b918e..b301fb63f16b 100644 >>> --- a/drivers/dma-buf/heaps/cma_heap.c >>> +++ b/drivers/dma-buf/heaps/cma_heap.c >>> @@ -168,7 +168,7 @@ static vm_fault_t cma_heap_vm_fault(struct vm_fault *vmf) >>> if (vmf->pgoff > buffer->pagecount) >>> return VM_FAULT_SIGBUS; >>> >>> - return vmf_insert_pfn(vma, vmf->address, page_to_pfn(buffer->pages[vmf->pgoff])); >>> + return vmf_insert_page(vma, vmf->address, buffer->pages[vmf->pgoff]); >>> } >>> >>> static const struct vm_operations_struct dma_heap_vm_ops = {

1 year

1
0
0 0

Re: [RFC PATCH 01/12] dma-buf: Introduce dma_buf_get_pfn_unlocked() kAPI

by Christoph Hellwig

On Wed, Jan 15, 2025 at 09:55:29AM +0100, Simona Vetter wrote: > I think for 90% of exporters pfn would fit, but there's some really funny > ones where you cannot get a cpu pfn by design. So we need to keep the > pfn-less interfaces around. But ideally for the pfn-capable exporters we'd > have helpers/common code that just implements all the other interfaces. There is no way to have dma address without a PFN in Linux right now. How would you generate them? That implies you have an IOMMU that can generate IOVAs for something that doesn't have a physical address at all. Or do you mean some that don't have pages associated with them, and thus have pfn_valid fail on them? They still have a PFN, just not one that is valid to use in most of the Linux MM.

1 year

2
3
0 0

Re: [RFC PATCH 08/12] vfio/pci: Create host unaccessible dma-buf for private device

by Jason Gunthorpe

On Wed, Jan 15, 2025 at 11:57:05PM +1100, Alexey Kardashevskiy wrote: > On 15/1/25 00:35, Jason Gunthorpe wrote: > > On Tue, Jun 18, 2024 at 07:28:43AM +0800, Xu Yilun wrote: > > > > > > is needed so the secure world can prepare anything it needs prior to > > > > starting the VM. > > > > > > OK. From Dan's patchset there are some touch point for vendor tsm > > > drivers to do secure world preparation. e.g. pci_tsm_ops::probe(). > > > > > > Maybe we could move to Dan's thread for discussion. > > > > > > https://lore.kernel.org/linux-coco/173343739517.1074769.1313478654854592548… > > > > I think Dan's series is different, any uapi from that series should > > not be used in the VMM case. We need proper vfio APIs for the VMM to > > use. I would expect VFIO to be calling some of that infrastructure. > > Something like this experiment? > > https://github.com/aik/linux/commit/ce052512fb8784e19745d4cb222e23cabc57792e Yeah, maybe, though I don't know which of vfio/iommufd/kvm should be hosting those APIs, the above does seem to be a reasonable direction. When the various fds are closed I would expect the kernel to unbind and restore the device back. Jason

1 year, 1 month

1
0
0 0

Re: [RFC PATCH] driver: dma-buf: use vmf_insert_page for cma_heap_vm_fault

by Christian König

Am 15.01.25 um 07:18 schrieb zhaoyang.huang: > From: Zhaoyang Huang <zhaoyang.huang(a)unisoc.com> > > When using dma-buf as memory pool for VMM. The vmf_insert_pfn will > apply PTE_SPECIAL on pte which have vm_normal_page report bad_pte and > return NULL. This commit would like to suggest to replace > vmf_insert_pfn by vmf_insert_page. Setting PTE_SPECIAL is completely intentional here to prevent get_user_pages() from working on DMA-buf mappings. So absolutely clear NAK to this patch here. What exactly are you trying to do? Regards, Christian. > > [ 103.402787] kvm [5276]: gfn(ipa)=0x80000 hva=0x7d4a400000 write_fault=0 > [ 103.403822] BUG: Bad page map in process crosvm_vcpu0 pte:168000140000f43 pmd:8000000c1ca0003 > [ 103.405144] addr:0000007d4a400000 vm_flags:040400fb anon_vma:0000000000000000 mapping:ffffff8085163df0 index:0 > [ 103.406536] file:dmabuf fault:cma_heap_vm_fault [cma_heap] mmap:dma_buf_mmap_internal read_folio:0x0 > [ 103.407877] CPU: 3 PID: 5276 Comm: crosvm_vcpu0 Tainted: G W OE 6.6.46-android15-8-g8bab72b63c20-dirty-4k #1 1e474a12dac4553a3ebba3a911f3b744176a5d2d > [ 103.409818] Hardware name: Unisoc UMS9632-base Board (DT) > [ 103.410613] Call trace: > [ 103.411038] dump_backtrace+0xf4/0x140 > [ 103.411641] show_stack+0x20/0x30 > [ 103.412184] dump_stack_lvl+0x60/0x84 > [ 103.412766] dump_stack+0x18/0x24 > [ 103.413304] print_bad_pte+0x1b8/0x1cc > [ 103.413909] vm_normal_page+0xc8/0xd0 > [ 103.414491] follow_page_pte+0xb0/0x304 > [ 103.415096] follow_page_mask+0x108/0x240 > [ 103.415721] __get_user_pages+0x168/0x4ac > [ 103.416342] __gup_longterm_locked+0x15c/0x864 > [ 103.417023] pin_user_pages+0x70/0xcc > [ 103.417609] pkvm_mem_abort+0xf8/0x5c0 > [ 103.418207] kvm_handle_guest_abort+0x3e0/0x3e4 > [ 103.418906] handle_exit+0xac/0x33c > [ 103.419472] kvm_arch_vcpu_ioctl_run+0x48c/0x8d8 > [ 103.420176] kvm_vcpu_ioctl+0x504/0x5bc > [ 103.420785] __arm64_sys_ioctl+0xb0/0xec > [ 103.421401] invoke_syscall+0x60/0x11c > [ 103.422000] el0_svc_common+0xb4/0xe8 > [ 103.422590] do_el0_svc+0x24/0x30 > [ 103.423131] el0_svc+0x3c/0x70 > [ 103.423640] el0t_64_sync_handler+0x68/0xbc > [ 103.424288] el0t_64_sync+0x1a8/0x1ac > > Signed-off-by: Xiwei Wang <xiwei.wang1(a)unisoc.com> > Signed-off-by: Aijun Sun <aijun.sun(a)unisoc.com> > Signed-off-by: Zhaoyang Huang <zhaoyang.huang(a)unisoc.com> > --- > drivers/dma-buf/heaps/cma_heap.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/dma-buf/heaps/cma_heap.c b/drivers/dma-buf/heaps/cma_heap.c > index c384004b918e..b301fb63f16b 100644 > --- a/drivers/dma-buf/heaps/cma_heap.c > +++ b/drivers/dma-buf/heaps/cma_heap.c > @@ -168,7 +168,7 @@ static vm_fault_t cma_heap_vm_fault(struct vm_fault *vmf) > if (vmf->pgoff > buffer->pagecount) > return VM_FAULT_SIGBUS; > > - return vmf_insert_pfn(vma, vmf->address, page_to_pfn(buffer->pages[vmf->pgoff])); > + return vmf_insert_page(vma, vmf->address, buffer->pages[vmf->pgoff]); > } > > static const struct vm_operations_struct dma_heap_vm_ops = {

1 year, 1 month

1
0
0 0

Re: [RFC PATCH 01/12] dma-buf: Introduce dma_buf_get_pfn_unlocked() kAPI

by Christian König

Am 15.01.25 um 09:55 schrieb Simona Vetter: >>> If we add something >>> new, we need clear rules and not just "here's the kvm code that uses it". >>> That's how we've done dma-buf at first, and it was a terrible mess of >>> mismatched expecations. >> Yes, that would be wrong. It should be self defined within dmabuf and >> kvm should adopt to it, move semantics and all. > Ack. > > I feel like we have a plan here. I think I have to object a bit on that. > Summary from my side: > > - Sort out pin vs revocable vs dynamic/moveable semantics, make sure > importers have no surprises. > > - Adopt whatever new dma-api datastructures pops out of the dma-api > reworks. > > - Add pfn based memory access as yet another optional access method, with > helpers so that exporters who support this get all the others for free. > > I don't see a strict ordering between these, imo should be driven by > actual users of the dma-buf api. > > Already done: > > - dmem cgroup so that we can resource control device pinnings just landed > in drm-next for next merge window. So that part is imo sorted and we can > charge ahead with pinning into device memory without all the concerns > we've had years ago when discussing that for p2p dma-buf support. > > But there might be some work so that we can p2p pin without requiring > dynamic attachments only, I haven't checked whether that needs > adjustment in dma-buf.c code or just in exporters. > > Anything missing? Well as far as I can see this use case is not a good fit for the DMA-buf interfaces in the first place. DMA-buf deals with devices and buffer exchange. What's necessary here instead is to give an importing VM full access on some memory for their specific use case. That full access includes CPU and DMA mappings, modifying caching attributes, potentially setting encryption keys for specific ranges etc.... etc... In other words we have a lot of things the importer here should be able to do which we don't want most of the DMA-buf importers to do. The semantics for things like pin vs revocable vs dynamic/moveable seems similar, but that's basically it. As far as I know the TEE subsystem also represents their allocations as file descriptors. If I'm not completely mistaken this use case most likely fit's better there. > I feel like this is small enough that m-l archives is good enough. For > some of the bigger projects we do in graphics we sometimes create entries > in our kerneldoc with wip design consensus and things like that. But > feels like overkill here. > >> My general desire is to move all of RDMA's MR process away from >> scatterlist and work using only the new DMA API. This will save *huge* >> amounts of memory in common workloads and be the basis for non-struct >> page DMA support, including P2P. > Yeah a more memory efficient structure than the scatterlist would be > really nice. That would even benefit the very special dma-buf exporters > where you cannot get a pfn and only the dma_addr_t, altough most of those > (all maybe even?) have contig buffers, so your scatterlist has only one > entry. But it would definitely be nice from a design pov. Completely agree on that part. Scatterlist have a some design flaws, especially mixing the input and out parameters of the DMA API into the same structure. Additional to that DMA addresses are basically missing which bus they belong to and details how the access should be made (e.g. snoop vs no-snoop etc...). > Aside: A way to more efficiently create compressed scatterlists would be > neat too, because a lot of drivers hand-roll that and it's a bit brittle > and kinda silly to duplicate. With compressed I mean just a single entry > for a contig range, in practice thanks to huge pages/folios and allocators > trying to hand out contig ranges if there's plenty of memory that saves a > lot of memory too. But currently it's a bit a pain to construct these > efficiently, mostly it's just a two-pass approach and then trying to free > surplus memory or krealloc to fit. Also I don't have good ideas here, but > dma-api folks might have some from looking at too many things that create > scatterlists. I mailed with Christoph about that a while back as well and we both agreed that it would probably be a good idea to start defining a data structure to better encapsulate DMA addresses. It's just that nobody had time for that yet and/or I wasn't looped in in the final discussion about it. Regards, Christian. > -Sima

1 year, 1 month

1
0
0 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Linaro-mm-sig