On 21.06.24 11:25, Quentin Perret wrote:
On Friday 21 Jun 2024 at 10:02:08 (+0200), David Hildenbrand wrote:
Thanks for the information. IMHO we really should try to find a common ground here, and FOLL_EXCLUSIVE is likely not it :)
That's OK, IMO at least :-).
Thanks for reviving this discussion with your patch set!
pKVM is interested in in-place conversion, I believe there are valid use cases for in-place conversion for TDX and friends as well (as discussed, I think that might be a clean way to get huge/gigantic page support in).
This implies the option to:
- Have shared+private memory in guest_memfd
- Be able to mmap shared parts
- Be able to convert shared<->private in place
and later in my interest
- Have huge/gigantic page support in guest_memfd with the option of converting individual subpages
We might not want to make use of that model for all of CC -- as you state, sometimes the destructive approach might be better performance wise -- but having that option doesn't sound crazy to me (and maybe would solve real issues as well).
Cool.
After all, the common requirement here is that "private" pages are not mapped/pinned/accessible.
Sure, there might be cases like "pKVM can handle access to private pages in user page mappings", "AMD-SNP will not crash the host if writing to private pages" but there are not factors that really make a difference for a common solution.
Sure, there isn't much value in differentiating on these things. One might argue that we could save one mmap() on the private->shared conversion path by keeping all of guest_memfd mapped in userspace including private memory, but that's most probably not worth the effort of re-designing the whole thing just for that, so let's forget that.
In a world where we can mmap() the whole (sparse "shared") thing, and dynamically map/unmap the shared parts only it would be saving a page fault on private->shared conversion, correct.
But that's sounds more like a CC-specific optimization for frequent conversions, which we should just ignore initially.
The ability to handle stage-2 faults in the kernel has implications in other places however. It means we don't need to punch holes in the kernel linear map when donating memory to a guest for example, even with 'crazy' access patterns like load_unaligned_zeropad(). So that's good.
private memory: not mapped, not pinned shared memory: maybe mapped, maybe pinned granularity of conversion: single pages
Anything I am missing?
That looks good to me. And as discussed in previous threads, we have the ambition of getting page-migration to work, including for private memory, mostly to get kcompactd to work better when pVMs are running. Android makes extensive use of compaction, and pVMs currently stick out like a sore thumb.
Yes, I think migration for compaction has to be supported at some point (at least for small pages that can be either private or shared, not a mixture), and I suspect we should be able to integrate it with core-mm in a not-too-horrible fashion. For example, we do have a non-lru page migration infrastructure in place already if the LRU-based one is not a good fit.
Memory swapping and all other currently-strictly LRU-based mechanisms should be out of scope for now: as Sean says, we don't want to go down that path.
We can trivially implement a hypercall to have pKVM swap a private page with another without the guest having to know. The difficulty is obviously to hook that in Linux, and I've personally not looked into it properly, so that is clearly longer term. We don't want to take anybody by surprise if there is a need for some added complexity in guest_memfd to support this use-case though. I don't expect folks on the receiving end of that to agree to it blindly without knowing _what_ this complexity is FWIW. But at least our intentions are clear :-)
Agreed.