On Thu, Aug 25, 2022 at 10:42:41AM +1000, Alistair Popple wrote:
Peter Xu peterx@redhat.com writes:
On Wed, Aug 24, 2022 at 04:25:44PM -0400, Peter Xu wrote:
On Wed, Aug 24, 2022 at 11:56:25AM +1000, Alistair Popple wrote:
Still I don't know whether there'll be any side effect of having stall tlbs in !present ptes because I'm not familiar enough with the private dev swap migration code. But I think having them will be safe, even if redundant.
What side-effect were you thinking of? I don't see any issue with not TLB flushing stale device-private TLBs prior to the migration because they're not accessible anyway and shouldn't be in any TLB.
Sorry to be misleading, I never meant we must add them. As I said it's just that I don't know the code well so I don't know whether it's safe to not have it.
IIUC it's about whether having stall system-ram stall tlb in other processor would matter or not here. E.g. some none pte that this code collected (boosted both "cpages" and "npages" for a none pte) could have stall tlb in other cores that makes the page writable there.
For this one, let me give a more detailed example.
Thanks, I would have been completely lost about what you were talking about without this :-)
It's about whether below could happen:
thread 1 thread 2 thread 3 -------- -------- -------- write to page P (data=P1) (cached TLB writable)zap_pte_range() pgtable lock clear pte for page P pgtable unlock ... migrate_vma_collect pte none, npages++, cpages++ allocate device page copy data (with P1) map pte as device swap write to page P again (data updated from P1->P2) flush tlb
Then at last from processor side P should have data P2 but actually from device memory it's P1. Data corrupt.
In the above scenario migrate_vma_collect_pmd() will observe pte_none. This will mark the src_pfn[] array as needing a new zero page which will be installed by migrate_vma_pages()->migrate_vma_insert_page().
So there is no data to be copied hence there can't be any data corruption. Remember these are private anonymous pages, so any zap_pte_range() indicates the data is no longer needed (eg. MADV_DONTNEED).
My bad to have provided an example but invalid. :)
So if the trylock in the function is the only way to migrate this page, then I agree stall tlb is fine.
When I said I'm not familiar with the code, it's majorly about one thing I never figured out myself, in that migrate_vma_collect_pmd() has this optimization to trylock on the page, collect if it succeeded:
/*
- Optimize for the common case where page is only mapped once
- in one process. If we can lock the page, then we can safely
- set up a special migration page table entry now.
*/ if (trylock_page(page)) { ... } else { put_page(page); mpfn = 0; }
But it's kind of against a pure "optimization" in that if trylock failed, we'll clear the mpfn so the src[i] will be zero at last. Then will we directly give up on this page, or will we try to lock_page() again somewhere?
That comment is out dated. We used to try locking the page again but that was removed by ab09243aa95a ("mm/migrate.c: remove MIGRATE_PFN_LOCKED"). See https://lkml.kernel.org/r/20211025041608.289017-1-apopple@nvidia.com
Will post a clean-up for it.
That'll help, thanks.