The patch titled Subject: hugetlbfs: fix races and page leaks during migration has been added to the -mm tree. Its filename is huegtlbfs-fix-races-and-page-leaks-during-migration.patch
This patch should soon appear at http://ozlabs.org/~akpm/mmots/broken-out/huegtlbfs-fix-races-and-page-leaks-... and later at http://ozlabs.org/~akpm/mmotm/broken-out/huegtlbfs-fix-races-and-page-leaks-...
Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next and is updated there every 3-4 working days
------------------------------------------------------ From: Mike Kravetz mike.kravetz@oracle.com Subject: hugetlbfs: fix races and page leaks during migration
hugetlb pages should only be migrated if they are 'active'. The routines set/clear_page_huge_active() modify the active state of hugetlb pages. When a new hugetlb page is allocated at fault time, set_page_huge_active is called before the page is locked. Therefore, another thread could race and migrate the page while it is being added to page table by the fault code. This race is somewhat hard to trigger, but can be seen by strategically adding udelay to simulate worst case scheduling behavior. Depending on 'how' the code races, various BUG()s could be triggered.
To address this issue, simply delay the set_page_huge_active call until after the page is successfully added to the page table.
Hugetlb pages can also be leaked at migration time if the pages are associated with a file in an explicitly mounted hugetlbfs filesystem. For example, a test program which hole punches, faults and migrates pages in such a file (1G in size) will eventually fail because it can not allocate a page. Reported counts and usage at time of failure:
node0 537 free_hugepages 1024 nr_hugepages 0 surplus_hugepages node1 1000 free_hugepages 1024 nr_hugepages 0 surplus_hugepages
Filesystem Size Used Avail Use% Mounted on nodev 4.0G 4.0G 0 100% /var/opt/hugepool
Note that the filesystem shows 4G of pages used, while actual usage is 511 pages (just under 1G). Failed trying to allocate page 512.
If a hugetlb page is associated with an explicitly mounted filesystem, this information in contained in the page_private field. At migration time, this information is not preserved. To fix, simply transfer page_private from old to new page at migration time if necessary.
Link: http://lkml.kernel.org/r/20190212221400.3512-1-mike.kravetz@oracle.com Fixes: bcc54222309c ("mm: hugetlb: introduce page_huge_active") Signed-off-by: Mike Kravetz mike.kravetz@oracle.com Cc: Michal Hocko mhocko@kernel.org Cc: Naoya Horiguchi n-horiguchi@ah.jp.nec.com Cc: Andrea Arcangeli aarcange@redhat.com Cc: "Kirill A . Shutemov" kirill.shutemov@linux.intel.com Cc: Mel Gorman mgorman@techsingularity.net Cc: Davidlohr Bueso dave@stgolabs.net Cc: stable@vger.kernel.org Signed-off-by: Andrew Morton akpm@linux-foundation.org ---
fs/hugetlbfs/inode.c | 12 ++++++++++++ mm/hugetlb.c | 9 ++++++--- 2 files changed, 18 insertions(+), 3 deletions(-)
--- a/fs/hugetlbfs/inode.c~huegtlbfs-fix-races-and-page-leaks-during-migration +++ a/fs/hugetlbfs/inode.c @@ -859,6 +859,18 @@ static int hugetlbfs_migrate_page(struct rc = migrate_huge_page_move_mapping(mapping, newpage, page); if (rc != MIGRATEPAGE_SUCCESS) return rc; + + /* + * page_private is subpool pointer in hugetlb pages. Transfer to + * new page. PagePrivate is not associated with page_private for + * hugetlb pages and can not be set here as only page_huge_active + * pages can be migrated. + */ + if (page_private(page)) { + set_page_private(newpage, page_private(page)); + set_page_private(page, 0); + } + if (mode != MIGRATE_SYNC_NO_COPY) migrate_page_copy(newpage, page); else --- a/mm/hugetlb.c~huegtlbfs-fix-races-and-page-leaks-during-migration +++ a/mm/hugetlb.c @@ -3624,7 +3624,6 @@ retry_avoidcopy: copy_user_huge_page(new_page, old_page, address, vma, pages_per_huge_page(h)); __SetPageUptodate(new_page); - set_page_huge_active(new_page);
mmu_notifier_range_init(&range, mm, haddr, haddr + huge_page_size(h)); mmu_notifier_invalidate_range_start(&range); @@ -3645,6 +3644,7 @@ retry_avoidcopy: make_huge_pte(vma, new_page, 1)); page_remove_rmap(old_page, true); hugepage_add_new_anon_rmap(new_page, vma, haddr); + set_page_huge_active(new_page); /* Make the old page be freed below */ new_page = old_page; } @@ -3790,7 +3790,6 @@ retry: } clear_huge_page(page, address, pages_per_huge_page(h)); __SetPageUptodate(page); - set_page_huge_active(page);
if (vma->vm_flags & VM_MAYSHARE) { int err = huge_add_to_page_cache(page, mapping, idx); @@ -3861,6 +3860,10 @@ retry: }
spin_unlock(ptl); + + /* May already be set if not newly allocated page */ + set_page_huge_active(page); + unlock_page(page); out: return ret; @@ -4095,7 +4098,6 @@ int hugetlb_mcopy_atomic_pte(struct mm_s * the set_pte_at() write. */ __SetPageUptodate(page); - set_page_huge_active(page);
mapping = dst_vma->vm_file->f_mapping; idx = vma_hugecache_offset(h, dst_vma, dst_addr); @@ -4163,6 +4165,7 @@ int hugetlb_mcopy_atomic_pte(struct mm_s update_mmu_cache(dst_vma, dst_addr, dst_pte);
spin_unlock(ptl); + set_page_huge_active(page); if (vm_shared) unlock_page(page); ret = 0; _
Patches currently in -mm which might be from mike.kravetz@oracle.com are
huegtlbfs-fix-races-and-page-leaks-during-migration.patch