On Dec 18, 2021, at 4:35 PM, Linus Torvalds torvalds@linux-foundation.org wrote:
(I have only ever seen the kernel side of uffd, not the actual user side, so I'm not sure about the use patterns).
I use it in a very fine granularity, and I suspect QEMU and CRIU do so too.
That said, your suggestion of a shadow sw page table bit thing would also work. And it would solve some problems we have in core areas (notably "page_special()" which right now has that ARCH_HAS_PTE_SPECIAL thing).
It would make it really easy to have that "this page table entry is pinned" flag too.
I found my old messy code for the software-PTE thing.
I see that eventually I decided to hold a pointer to the “extra PTEs” of each page in the PMD-page-struct. [ I also implemented the 2-adjacent pages approach but this code is long gone. ]
My rationale was that:
1. It does not bound you to have the same size for PTE and “extra-PTE” 2. The PMD-page struct is anyhow hot (since you acquired the PTL) 3. Allocating “extra-PTE” dynamically does not require to rewire the page-tables, which requires a TLB flush.
I think there is a place to hold a pointer in the PMD-page-struct (_pt_pad_1, we just need to keep the lowest bit clear so the kernel won’t mistaken it to be a compound page).
I still don’t know what exactly you have in mind for making use out of it for the COW issue. Keeping a pin-count (which requires internal API changes for unpin_user_page() and friends?) or having “was ever pinned” sticky bit? And then changing page_needs_cow_for_dma() to look at the PTE so copy_present_pte() would break the COW eagerly?
Anyhow, I can clean it up and send (although it is rather simple and I ignored many thing, such as THP, remap, etc), but I am not sure I have the time now to fully address the COW problem. I will wait for Monday for David’s response.