Hello,
I am trying to get feedback / ideas (specially from the people in --to) before I reach out to LKML.
Problem statement:
Share memory (dma-buf/dma-heap fd) over VSock (over virtio-msg-ffa).
Current status:
I have implemented the first version [1]/[2]. The system-heap memory is successfully shared with OPTEE, test setup available here [3].
Workflow:
Userspace:
open("/dev/dma_heap/system", O_RDWR) + ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc) + sendmsg(sock, &msg, 0) // msg: SCM_VSOCK_SHMEM / VSOCK_SHMEM_TYPE_FFA / VSOCK_SHMEM_SUBOP_OFFER
Kernel:
vsock_sendmsg_shmem(vsk, msg) | vsk->transport->map_dma_buf(dmabuf) | virtio_transport_map_dma_buf(dmabuf) | dma_buf_attach(dmabuf, dev); // Vsock's underlying device - virtio-msg device + dma_buf_map_attachment_unlocked(dbuf->attach, DMA_BIDIRECTIONAL); | system_heap_map_dma_buf(attach, dir); | dma_map_sgtable(attachment->dev, table, dir, attrs=0); | virtio_msg_dma_map_sg_dev(dev, sgl, nents, dir, attrs); // FFA dma-ops | ffa_dev->ops->mem_ops->memory_share(&args); + vmfdev->send(vmfdev, request, response, &vmfdev->idata);
The memory is shared with FFA SHARE type. This doesn't require a custom heap and works with any DMA heap/buf, system heap is just an example here.
FFA supports LEND-ing the memory as well and Google needs it. And this is where I am _blocked_ right now and need help.
We need to support both SHARE (both sides can access memory) and LEND (Linux doesn't access it any further - unmapped) mechanisms and I am not sure how to move forward.
- We can pass the share-type from userspace and that can reach up to vsock / dma-buf (with minor API change to dma_buf_map_attachment_unlocked()) / system-heap.
- But after that the standard dma_map_sgtable() call takes only one configurable parameter, `attributes` (which is set to 0 for now by system heap). There is nothing else which I can use to pass the flag to FFA specific DMA OPs.
- I thought maybe I can create a new DMA attribute: `DMA_ATTR_MEM_LEND`, which can be passed in the `attr` field and that will work I guess.
- But I am not sure if it would be acceptable upstream to add a new attribute for this.
- And so I am looking for some ideas.