On Tue, Sep 11, 2018 at 04:30:02PM -0400, Zi Yan wrote:
I want to understand the Anon THP part of the problem more clearly. For Anon THPs, you said, PTEs for the page will always come after mlocked PMD. I just wonder that if a process forks a child1 which forks its own child2 and the child1 mlocks a subpage causing split_pmd_page() and migrates its PTE-mapped THP, will the kernel see the sequence of PMD-mapped THP, PTE-mapped THP, and PMD-mapped THP while walking VMAs? Will the second PMD-mapped THP reset the mlock on the page?
VM_LOCKED is not inheritable. child2 will not have the VMA mlocked and will not try to mlock the page. If child2 will do mlock() manually it will cause CoW and the process will ge new pages in the new VMA.
In addition, I also discover that PageDoubleMap is not set for double mapped Anon THPs after migration, the following patch fixes it. Do you want me to send it separately or you can merge it with your patch?
I think we can live without DoubleMap for anon-THP: rmap walking order semantics and the fact that page can be shared only over fork() should be enough to get it under control. DoubleMap comes with overhead and I would like to avoid it where possible.