Hello,
I have some questions / feedback on the Xen side of virtio-msg.
AFAIU virtio-msg defines a protocol in order to deal with discovery (optional) and configuration of the PV devices. But things are undefined regarding what is a "memory address".
In Xen memory model with grants, each guest has its own memory space. The frontend shares memory pages with the backend through grants pages identified by grant references.
So in a design based on grants, virtio addresses can't (or at least shouldn't) be guest physical address; but needs to be something derived on grants. A earlier design [1] was forging a address with the grant reference, but I feel it's not great, as it forces "map+unmap" cycles for temporary buffers thus has the same performance problem as Xen PV drivers (without persistent grants) where map+unmap cycles is a performance problem.
And would make the address space very fragmented and in often limited to 4KB buffers.
One idea that is already used by Xen displif (PV Display) is to have a "gref directory" and describe the address space on that. The gref directory is roughly a array of grant references (shared pages infos) that could describe a address space starting at 0, where each page is defined by a grant reference of the directory. That way, the backend can freely keep all or a part or the address space persistently mapped (or eventually map it all at once); and the address space is also contiguous which would help with >4KB buffers.
Any thoughts ?
[1] https://static.sched.com/hosted_files/xen2021/bf/Thursday_2021-Xen-Summit-vi...
-- Teddy Astie | Vates XCP-ng Developer
XCP-ng & Xen Orchestra - Vates solutions
web: https://vates.tech