August 2019 - Linux-stable-mirror

[patch 11/18] mm/usercopy: use memory range to be accessed for wraparound check

by akpm＠linux-foundation.org

From: "Isaac J. Manjarres" <isaacm(a)codeaurora.org> Subject: mm/usercopy: use memory range to be accessed for wraparound check Currently, when checking to see if accessing n bytes starting at address "ptr" will cause a wraparound in the memory addresses, the check in check_bogus_address() adds an extra byte, which is incorrect, as the range of addresses that will be accessed is [ptr, ptr + (n - 1)]. This can lead to incorrectly detecting a wraparound in the memory address, when trying to read 4 KB from memory that is mapped to the the last possible page in the virtual address space, when in fact, accessing that range of memory would not cause a wraparound to occur. Use the memory range that will actually be accessed when considering if accessing a certain amount of bytes will cause the memory address to wrap around. Link: http://lkml.kernel.org/r/1564509253-23287-1-git-send-email-isaacm@codeauror… Fixes: f5509cc18daa ("mm: Hardened usercopy") Signed-off-by: Prasad Sodagudi <psodagud(a)codeaurora.org> Signed-off-by: Isaac J. Manjarres <isaacm(a)codeaurora.org> Co-developed-by: Prasad Sodagudi <psodagud(a)codeaurora.org> Reviewed-by: William Kucharski <william.kucharski(a)oracle.com> Acked-by: Kees Cook <keescook(a)chromium.org> Cc: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Cc: Trilok Soni <tsoni(a)codeaurora.org> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> --- mm/usercopy.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/usercopy.c~mm-usercopy-use-memory-range-to-be-accessed-for-wraparound-check +++ a/mm/usercopy.c @@ -147,7 +147,7 @@ static inline void check_bogus_address(c bool to_user) { /* Reject if object wraps past end of memory. */ - if (ptr + n < ptr) + if (ptr + (n - 1) < ptr) usercopy_abort("wrapped address", NULL, to_user, 0, ptr + n); /* Reject if NULL or ZERO-allocation. */ _

5 years, 4 months

1
0
0 0

[patch 08/18] mm/memcontrol.c: fix use after free in mem_cgroup_iter()

by akpm＠linux-foundation.org

From: Miles Chen <miles.chen(a)mediatek.com> Subject: mm/memcontrol.c: fix use after free in mem_cgroup_iter() This patch is sent to report an use after free in mem_cgroup_iter() after merging commit be2657752e9e ("mm: memcg: fix use after free in mem_cgroup_iter()"). I work with android kernel tree (4.9 & 4.14), and commit be2657752e9e ("mm: memcg: fix use after free in mem_cgroup_iter()") has been merged to the trees. However, I can still observe use after free issues addressed in the commit be2657752e9e. (on low-end devices, a few times this month) backtrace: css_tryget <- crash here mem_cgroup_iter shrink_node shrink_zones do_try_to_free_pages try_to_free_pages __perform_reclaim __alloc_pages_direct_reclaim __alloc_pages_slowpath __alloc_pages_nodemask To debug, I poisoned mem_cgroup before freeing it: static void __mem_cgroup_free(struct mem_cgroup *memcg) for_each_node(node) free_mem_cgroup_per_node_info(memcg, node); free_percpu(memcg->stat); + /* poison memcg before freeing it */ + memset(memcg, 0x78, sizeof(struct mem_cgroup)); kfree(memcg); } The coredump shows the position=0xdbbc2a00 is freed. (gdb) p/x ((struct mem_cgroup_per_node *)0xe5009e00)->iter[8] $13 = {position = 0xdbbc2a00, generation = 0x2efd} 0xdbbc2a00: 0xdbbc2e00 0x00000000 0xdbbc2800 0x00000100 0xdbbc2a10: 0x00000200 0x78787878 0x00026218 0x00000000 0xdbbc2a20: 0xdcad6000 0x00000001 0x78787800 0x00000000 0xdbbc2a30: 0x78780000 0x00000000 0x0068fb84 0x78787878 0xdbbc2a40: 0x78787878 0x78787878 0x78787878 0xe3fa5cc0 0xdbbc2a50: 0x78787878 0x78787878 0x00000000 0x00000000 0xdbbc2a60: 0x00000000 0x00000000 0x00000000 0x00000000 0xdbbc2a70: 0x00000000 0x00000000 0x00000000 0x00000000 0xdbbc2a80: 0x00000000 0x00000000 0x00000000 0x00000000 0xdbbc2a90: 0x00000001 0x00000000 0x00000000 0x00100000 0xdbbc2aa0: 0x00000001 0xdbbc2ac8 0x00000000 0x00000000 0xdbbc2ab0: 0x00000000 0x00000000 0x00000000 0x00000000 0xdbbc2ac0: 0x00000000 0x00000000 0xe5b02618 0x00001000 0xdbbc2ad0: 0x00000000 0x78787878 0x78787878 0x78787878 0xdbbc2ae0: 0x78787878 0x78787878 0x78787878 0x78787878 0xdbbc2af0: 0x78787878 0x78787878 0x78787878 0x78787878 0xdbbc2b00: 0x78787878 0x78787878 0x78787878 0x78787878 0xdbbc2b10: 0x78787878 0x78787878 0x78787878 0x78787878 0xdbbc2b20: 0x78787878 0x78787878 0x78787878 0x78787878 0xdbbc2b30: 0x78787878 0x78787878 0x78787878 0x78787878 0xdbbc2b40: 0x78787878 0x78787878 0x78787878 0x78787878 0xdbbc2b50: 0x78787878 0x78787878 0x78787878 0x78787878 0xdbbc2b60: 0x78787878 0x78787878 0x78787878 0x78787878 0xdbbc2b70: 0x78787878 0x78787878 0x78787878 0x78787878 0xdbbc2b80: 0x78787878 0x78787878 0x00000000 0x78787878 0xdbbc2b90: 0x78787878 0x78787878 0x78787878 0x78787878 0xdbbc2ba0: 0x78787878 0x78787878 0x78787878 0x78787878 In the reclaim path, try_to_free_pages() does not setup sc.target_mem_cgroup and sc is passed to do_try_to_free_pages(), ..., shrink_node(). In mem_cgroup_iter(), root is set to root_mem_cgroup because sc->target_mem_cgroup is NULL. It is possible to assign a memcg to root_mem_cgroup.nodeinfo.iter in mem_cgroup_iter(). try_to_free_pages struct scan_control sc = {...}, target_mem_cgroup is 0x0; do_try_to_free_pages shrink_zones shrink_node mem_cgroup *root = sc->target_mem_cgroup; memcg = mem_cgroup_iter(root, NULL, &reclaim); mem_cgroup_iter() if (!root) root = root_mem_cgroup; ... css = css_next_descendant_pre(css, &root->css); memcg = mem_cgroup_from_css(css); cmpxchg(&iter->position, pos, memcg); My device uses memcg non-hierarchical mode. When we release a memcg: invalidate_reclaim_iterators() reaches only dead_memcg and its parents. If non-hierarchical mode is used, invalidate_reclaim_iterators() never reaches root_mem_cgroup. static void invalidate_reclaim_iterators(struct mem_cgroup *dead_memcg) { struct mem_cgroup *memcg = dead_memcg; for (; memcg; memcg = parent_mem_cgroup(memcg) ... } So the use after free scenario looks like: CPU1 CPU2 try_to_free_pages do_try_to_free_pages shrink_zones shrink_node mem_cgroup_iter() if (!root) root = root_mem_cgroup; ... css = css_next_descendant_pre(css, &root->css); memcg = mem_cgroup_from_css(css); cmpxchg(&iter->position, pos, memcg); invalidate_reclaim_iterators(memcg); ... __mem_cgroup_free() kfree(memcg); try_to_free_pages do_try_to_free_pages shrink_zones shrink_node mem_cgroup_iter() if (!root) root = root_mem_cgroup; ... mz = mem_cgroup_nodeinfo(root, reclaim->pgdat->node_id); iter = &mz->iter[reclaim->priority]; pos = READ_ONCE(iter->position); css_tryget(&pos->css) <- use after free To avoid this, we should also invalidate root_mem_cgroup.nodeinfo.iter in invalidate_reclaim_iterators(). [cai(a)lca.pw: fix -Wparentheses compilation warning] Link: http://lkml.kernel.org/r/1564580753-17531-1-git-send-email-cai@lca.pw Link: http://lkml.kernel.org/r/20190730015729.4406-1-miles.chen@mediatek.com Fixes: 5ac8fb31ad2e ("mm: memcontrol: convert reclaim iterator to simple css refcounting") Signed-off-by: Miles Chen <miles.chen(a)mediatek.com> Signed-off-by: Qian Cai <cai(a)lca.pw> Acked-by: Michal Hocko <mhocko(a)suse.com> Cc: Johannes Weiner <hannes(a)cmpxchg.org> Cc: Vladimir Davydov <vdavydov.dev(a)gmail.com> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> --- mm/memcontrol.c | 39 +++++++++++++++++++++++++++++---------- 1 file changed, 29 insertions(+), 10 deletions(-) --- a/mm/memcontrol.c~mm-memcontrol-fix-use-after-free-in-mem_cgroup_iter +++ a/mm/memcontrol.c @@ -1130,26 +1130,45 @@ void mem_cgroup_iter_break(struct mem_cg css_put(&prev->css); } -static void invalidate_reclaim_iterators(struct mem_cgroup *dead_memcg) +static void __invalidate_reclaim_iterators(struct mem_cgroup *from, + struct mem_cgroup *dead_memcg) { - struct mem_cgroup *memcg = dead_memcg; struct mem_cgroup_reclaim_iter *iter; struct mem_cgroup_per_node *mz; int nid; int i; - for (; memcg; memcg = parent_mem_cgroup(memcg)) { - for_each_node(nid) { - mz = mem_cgroup_nodeinfo(memcg, nid); - for (i = 0; i <= DEF_PRIORITY; i++) { - iter = &mz->iter[i]; - cmpxchg(&iter->position, - dead_memcg, NULL); - } + for_each_node(nid) { + mz = mem_cgroup_nodeinfo(from, nid); + for (i = 0; i <= DEF_PRIORITY; i++) { + iter = &mz->iter[i]; + cmpxchg(&iter->position, + dead_memcg, NULL); } } } +static void invalidate_reclaim_iterators(struct mem_cgroup *dead_memcg) +{ + struct mem_cgroup *memcg = dead_memcg; + struct mem_cgroup *last; + + do { + __invalidate_reclaim_iterators(memcg, dead_memcg); + last = memcg; + } while ((memcg = parent_mem_cgroup(memcg))); + + /* + * When cgruop1 non-hierarchy mode is used, + * parent_mem_cgroup() does not walk all the way up to the + * cgroup root (root_mem_cgroup). So we have to handle + * dead_memcg from cgroup root separately. + */ + if (last != root_mem_cgroup) + __invalidate_reclaim_iterators(root_mem_cgroup, + dead_memcg); +} + /** * mem_cgroup_scan_tasks - iterate over tasks of a memory cgroup hierarchy * @memcg: hierarchy root _

5 years, 4 months

1
0
0 0

[patch 07/18] mm/z3fold.c: fix z3fold_destroy_pool() race condition

by akpm＠linux-foundation.org

From: Henry Burns <henryburns(a)google.com> Subject: mm/z3fold.c: fix z3fold_destroy_pool() race condition The constraint from the zpool use of z3fold_destroy_pool() is there are no outstanding handles to memory (so no active allocations), but it is possible for there to be outstanding work on either of the two wqs in the pool. Calling z3fold_deregister_migration() before the workqueues are drained means that there can be allocated pages referencing a freed inode, causing any thread in compaction to be able to trip over the bad pointer in PageMovable(). Link: http://lkml.kernel.org/r/20190726224810.79660-2-henryburns@google.com Fixes: 1f862989b04a ("mm/z3fold.c: support page migration") Signed-off-by: Henry Burns <henryburns(a)google.com> Reviewed-by: Shakeel Butt <shakeelb(a)google.com> Reviewed-by: Jonathan Adams <jwadams(a)google.com> Cc: Vitaly Vul <vitaly.vul(a)sony.com> Cc: Vitaly Wool <vitalywool(a)gmail.com> Cc: David Howells <dhowells(a)redhat.com> Cc: Thomas Gleixner <tglx(a)linutronix.de> Cc: Al Viro <viro(a)zeniv.linux.org.uk> Cc: Henry Burns <henrywolfeburns(a)gmail.com> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> --- mm/z3fold.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) --- a/mm/z3fold.c~mm-z3foldc-fix-z3fold_destroy_pool-race-condition +++ a/mm/z3fold.c @@ -817,16 +817,19 @@ out: static void z3fold_destroy_pool(struct z3fold_pool *pool) { kmem_cache_destroy(pool->c_handle); - z3fold_unregister_migration(pool); /* * We need to destroy pool->compact_wq before pool->release_wq, * as any pending work on pool->compact_wq will call * queue_work(pool->release_wq, &pool->work). + * + * There are still outstanding pages until both workqueues are drained, + * so we cannot unregister migration until then. */ destroy_workqueue(pool->compact_wq); destroy_workqueue(pool->release_wq); + z3fold_unregister_migration(pool); kfree(pool); } _

5 years, 4 months

1
0
0 0

[patch 06/18] mm/z3fold.c: fix z3fold_destroy_pool() ordering

by akpm＠linux-foundation.org

From: Henry Burns <henryburns(a)google.com> Subject: mm/z3fold.c: fix z3fold_destroy_pool() ordering The constraint from the zpool use of z3fold_destroy_pool() is there are no outstanding handles to memory (so no active allocations), but it is possible for there to be outstanding work on either of the two wqs in the pool. If there is work queued on pool->compact_workqueue when it is called, z3fold_destroy_pool() will do: z3fold_destroy_pool() destroy_workqueue(pool->release_wq) destroy_workqueue(pool->compact_wq) drain_workqueue(pool->compact_wq) do_compact_page(zhdr) kref_put(&zhdr->refcount) __release_z3fold_page(zhdr, ...) queue_work_on(pool->release_wq, &pool->work) *BOOM* So compact_wq needs to be destroyed before release_wq. Link: http://lkml.kernel.org/r/20190726224810.79660-1-henryburns@google.com Fixes: 5d03a6613957 ("mm/z3fold.c: use kref to prevent page free/compact race") Signed-off-by: Henry Burns <henryburns(a)google.com> Reviewed-by: Shakeel Butt <shakeelb(a)google.com> Reviewed-by: Jonathan Adams <jwadams(a)google.com> Cc: Vitaly Vul <vitaly.vul(a)sony.com> Cc: Vitaly Wool <vitalywool(a)gmail.com> Cc: David Howells <dhowells(a)redhat.com> Cc: Thomas Gleixner <tglx(a)linutronix.de> Cc: Al Viro <viro(a)zeniv.linux.org.uk Cc: Henry Burns <henrywolfeburns(a)gmail.com> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> --- mm/z3fold.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) --- a/mm/z3fold.c~mm-z3foldc-fix-z3fold_destroy_pool-ordering +++ a/mm/z3fold.c @@ -818,8 +818,15 @@ static void z3fold_destroy_pool(struct z { kmem_cache_destroy(pool->c_handle); z3fold_unregister_migration(pool); - destroy_workqueue(pool->release_wq); + + /* + * We need to destroy pool->compact_wq before pool->release_wq, + * as any pending work on pool->compact_wq will call + * queue_work(pool->release_wq, &pool->work). + */ + destroy_workqueue(pool->compact_wq); + destroy_workqueue(pool->release_wq); kfree(pool); } _

5 years, 4 months

1
0
0 0

[patch 05/18] mm: mempolicy: handle vma with unmovable pages mapped correctly in mbind

by akpm＠linux-foundation.org

From: Yang Shi <yang.shi(a)linux.alibaba.com> Subject: mm: mempolicy: handle vma with unmovable pages mapped correctly in mbind When running syzkaller internally, we ran into the below bug on 4.9.x kernel: kernel BUG at mm/huge_memory.c:2124! invalid opcode: 0000 [#1] SMP KASAN Dumping ftrace buffer: (ftrace buffer empty) Modules linked in: CPU: 0 PID: 1518 Comm: syz-executor107 Not tainted 4.9.168+ #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 0.5.1 01/01/2011 task: ffff880067b34900 task.stack: ffff880068998000 RIP: 0010:[<ffffffff81895d6b>] [<ffffffff81895d6b>] split_huge_page_to_list+0x8fb/0x1030 mm/huge_memory.c:2124 RSP: 0018:ffff88006899f980 EFLAGS: 00010286 RAX: 0000000000000000 RBX: ffffea00018f1700 RCX: 0000000000000000 RDX: 1ffffd400031e2e7 RSI: 0000000000000001 RDI: ffffea00018f1738 RBP: ffff88006899f9e8 R08: 0000000000000001 R09: 0000000000000000 R10: 0000000000000000 R11: fffffbfff0d8b13e R12: ffffea00018f1400 R13: ffffea00018f1400 R14: ffffea00018f1720 R15: ffffea00018f1401 FS: 00007fa333996740(0000) GS:ffff88006c600000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000020000040 CR3: 0000000066b9c000 CR4: 00000000000606f0 Stack: 0000000000000246 ffff880067b34900 0000000000000000 ffff88007ffdc000 0000000000000000 ffff88006899f9e8 ffffffff812b4015 ffff880064c64e18 ffffea00018f1401 dffffc0000000000 ffffea00018f1700 0000000020ffd000 Call Trace: [<ffffffff818490f1>] split_huge_page include/linux/huge_mm.h:100 [inline] [<ffffffff818490f1>] queue_pages_pte_range+0x7e1/0x1480 mm/mempolicy.c:538 [<ffffffff817ed0da>] walk_pmd_range mm/pagewalk.c:50 [inline] [<ffffffff817ed0da>] walk_pud_range mm/pagewalk.c:90 [inline] [<ffffffff817ed0da>] walk_pgd_range mm/pagewalk.c:116 [inline] [<ffffffff817ed0da>] __walk_page_range+0x44a/0xdb0 mm/pagewalk.c:208 [<ffffffff817edb94>] walk_page_range+0x154/0x370 mm/pagewalk.c:285 [<ffffffff81844515>] queue_pages_range+0x115/0x150 mm/mempolicy.c:694 [<ffffffff8184f493>] do_mbind mm/mempolicy.c:1241 [inline] [<ffffffff8184f493>] SYSC_mbind+0x3c3/0x1030 mm/mempolicy.c:1370 [<ffffffff81850146>] SyS_mbind+0x46/0x60 mm/mempolicy.c:1352 [<ffffffff810097e2>] do_syscall_64+0x1d2/0x600 arch/x86/entry/common.c:282 [<ffffffff82ff6f93>] entry_SYSCALL_64_after_swapgs+0x5d/0xdb Code: c7 80 1c 02 00 e8 26 0a 76 01 <0f> 0b 48 c7 c7 40 46 45 84 e8 4c RIP [<ffffffff81895d6b>] split_huge_page_to_list+0x8fb/0x1030 mm/huge_memory.c:2124 RSP <ffff88006899f980> with the below test: ---8<--- uint64_t r[1] = {0xffffffffffffffff}; int main(void) { syscall(__NR_mmap, 0x20000000, 0x1000000, 3, 0x32, -1, 0); intptr_t res = 0; res = syscall(__NR_socket, 0x11, 3, 0x300); if (res != -1) r[0] = res; *(uint32_t*)0x20000040 = 0x10000; *(uint32_t*)0x20000044 = 1; *(uint32_t*)0x20000048 = 0xc520; *(uint32_t*)0x2000004c = 1; syscall(__NR_setsockopt, r[0], 0x107, 0xd, 0x20000040, 0x10); syscall(__NR_mmap, 0x20fed000, 0x10000, 0, 0x8811, r[0], 0); *(uint64_t*)0x20000340 = 2; syscall(__NR_mbind, 0x20ff9000, 0x4000, 0x4002, 0x20000340, 0x45d4, 3); return 0; } ---8<--- Actually the test does: mmap(0x20000000, 16777216, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x20000000 socket(AF_PACKET, SOCK_RAW, 768) = 3 setsockopt(3, SOL_PACKET, PACKET_TX_RING, {block_size=65536, block_nr=1, frame_size=50464, frame_nr=1}, 16) = 0 mmap(0x20fed000, 65536, PROT_NONE, MAP_SHARED|MAP_FIXED|MAP_POPULATE|MAP_DENYWRITE, 3, 0) = 0x20fed000 mbind(..., MPOL_MF_STRICT|MPOL_MF_MOVE) = 0 The setsockopt() would allocate compound pages (16 pages in this test) for packet tx ring, then the mmap() would call packet_mmap() to map the pages into the user address space specified by the mmap() call. When calling mbind(), it would scan the vma to queue the pages for migration to the new node. It would split any huge page since 4.9 doesn't support THP migration, however, the packet tx ring compound pages are not THP and even not movable. So, the above bug is triggered. However, the later kernel is not hit by this issue due to d44d363f65780f2ac2 ("mm: don't assume anonymous pages have SwapBacked flag"), which just removes the PageSwapBacked check for a different reason. But, there is a deeper issue. According to the semantic of mbind(), it should return -EIO if MPOL_MF_MOVE or MPOL_MF_MOVE_ALL was specified and MPOL_MF_STRICT was also specified, but the kernel was unable to move all existing pages in the range. The tx ring of the packet socket is definitely not movable, however, mbind() returns success for this case. Although the most socket file associates with non-movable pages, but XDP may have movable pages from gup. So, it sounds not fine to just check the underlying file type of vma in vma_migratable(). Change migrate_page_add() to check if the page is movable or not, if it is unmovable, just return -EIO. But do not abort pte walk immediately, since there may be pages off LRU temporarily. We should migrate other pages if MPOL_MF_MOVE* is specified. Set has_unmovable flag if some paged could not be not moved, then return -EIO for mbind() eventually. With this change the above test would return -EIO as expected. [yang.shi(a)linux.alibaba.com: fix review comments from Vlastimil] Link: http://lkml.kernel.org/r/1563556862-54056-3-git-send-email-yang.shi@linux.a… Link: http://lkml.kernel.org/r/1561162809-59140-3-git-send-email-yang.shi@linux.a… Signed-off-by: Yang Shi <yang.shi(a)linux.alibaba.com> Reviewed-by: Vlastimil Babka <vbabka(a)suse.cz> Cc: Michal Hocko <mhocko(a)suse.com> Cc: Mel Gorman <mgorman(a)techsingularity.net> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> --- mm/mempolicy.c | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-) --- a/mm/mempolicy.c~mm-mempolicy-handle-vma-with-unmovable-pages-mapped-correctly-in-mbind +++ a/mm/mempolicy.c @@ -403,7 +403,7 @@ static const struct mempolicy_operations }, }; -static void migrate_page_add(struct page *page, struct list_head *pagelist, +static int migrate_page_add(struct page *page, struct list_head *pagelist, unsigned long flags); struct queue_pages { @@ -463,12 +463,11 @@ static int queue_pages_pmd(pmd_t *pmd, s flags = qp->flags; /* go to thp migration */ if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) { - if (!vma_migratable(walk->vma)) { + if (!vma_migratable(walk->vma) || + migrate_page_add(page, qp->pagelist, flags)) { ret = 1; goto unlock; } - - migrate_page_add(page, qp->pagelist, flags); } else ret = -EIO; unlock: @@ -532,7 +531,14 @@ static int queue_pages_pte_range(pmd_t * has_unmovable = true; break; } - migrate_page_add(page, qp->pagelist, flags); + + /* + * Do not abort immediately since there may be + * temporary off LRU pages in the range. Still + * need migrate other LRU pages. + */ + if (migrate_page_add(page, qp->pagelist, flags)) + has_unmovable = true; } else break; } @@ -961,7 +967,7 @@ static long do_get_mempolicy(int *policy /* * page migration, thp tail pages can be passed. */ -static void migrate_page_add(struct page *page, struct list_head *pagelist, +static int migrate_page_add(struct page *page, struct list_head *pagelist, unsigned long flags) { struct page *head = compound_head(page); @@ -974,8 +980,19 @@ static void migrate_page_add(struct page mod_node_page_state(page_pgdat(head), NR_ISOLATED_ANON + page_is_file_cache(head), hpage_nr_pages(head)); + } else if (flags & MPOL_MF_STRICT) { + /* + * Non-movable page may reach here. And, there may be + * temporary off LRU pages or non-LRU movable pages. + * Treat them as unmovable pages since they can't be + * isolated, so they can't be moved at the moment. It + * should return -EIO for this case too. + */ + return -EIO; } } + + return 0; } /* page allocation callback for NUMA node migration */ @@ -1178,9 +1195,10 @@ static struct page *new_page(struct page } #else -static void migrate_page_add(struct page *page, struct list_head *pagelist, +static int migrate_page_add(struct page *page, struct list_head *pagelist, unsigned long flags) { + return -EIO; } int do_migrate_pages(struct mm_struct *mm, const nodemask_t *from, _

5 years, 4 months

1
0
0 0

[patch 04/18] mm: mempolicy: make the behavior consistent when MPOL_MF_MOVE* and MPOL_MF_STRICT were specified

by akpm＠linux-foundation.org

From: Yang Shi <yang.shi(a)linux.alibaba.com> Subject: mm: mempolicy: make the behavior consistent when MPOL_MF_MOVE* and MPOL_MF_STRICT were specified When both MPOL_MF_MOVE* and MPOL_MF_STRICT was specified, mbind() should try best to migrate misplaced pages, if some of the pages could not be migrated, then return -EIO. There are three different sub-cases: 1. vma is not migratable 2. vma is migratable, but there are unmovable pages 3. vma is migratable, pages are movable, but migrate_pages() fails If #1 happens, kernel would just abort immediately, then return -EIO, after a7f40cfe3b7ada ("mm: mempolicy: make mbind() return -EIO when MPOL_MF_STRICT is specified"). If #3 happens, kernel would set policy and migrate pages with best-effort, but won't rollback the migrated pages and reset the policy back. Before that commit, they behaves in the same way. It'd better to keep their behavior consistent. But, rolling back the migrated pages and resetting the policy back sounds not feasible, so just make #1 behave as same as #3. Userspace will know that not everything was successfully migrated (via -EIO), and can take whatever steps it deems necessary - attempt rollback, determine which exact page(s) are violating the policy, etc. Make queue_pages_range() return 1 to indicate there are unmovable pages or vma is not migratable. The #2 is not handled correctly in the current kernel, the following patch will fix it. [yang.shi(a)linux.alibaba.com: fix review comments from Vlastimil] Link: http://lkml.kernel.org/r/1563556862-54056-2-git-send-email-yang.shi@linux.a… Link: http://lkml.kernel.org/r/1561162809-59140-2-git-send-email-yang.shi@linux.a… Signed-off-by: Yang Shi <yang.shi(a)linux.alibaba.com> Reviewed-by: Vlastimil Babka <vbabka(a)suse.cz> Cc: Michal Hocko <mhocko(a)suse.com> Cc: Mel Gorman <mgorman(a)techsingularity.net> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> --- mm/mempolicy.c | 68 +++++++++++++++++++++++++++++++++-------------- 1 file changed, 48 insertions(+), 20 deletions(-) --- a/mm/mempolicy.c~mm-mempolicy-make-the-behavior-consistent-when-mpol_mf_move-and-mpol_mf_strict-were-specified +++ a/mm/mempolicy.c @@ -429,11 +429,14 @@ static inline bool queue_pages_required( } /* - * queue_pages_pmd() has three possible return values: - * 1 - pages are placed on the right node or queued successfully. - * 0 - THP was split. - * -EIO - is migration entry or MPOL_MF_STRICT was specified and an existing - * page was already on a node that does not follow the policy. + * queue_pages_pmd() has four possible return values: + * 0 - pages are placed on the right node or queued successfully. + * 1 - there is unmovable page, and MPOL_MF_MOVE* & MPOL_MF_STRICT were + * specified. + * 2 - THP was split. + * -EIO - is migration entry or only MPOL_MF_STRICT was specified and an + * existing page was already on a node that does not follow the + * policy. */ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, unsigned long end, struct mm_walk *walk) @@ -451,19 +454,17 @@ static int queue_pages_pmd(pmd_t *pmd, s if (is_huge_zero_page(page)) { spin_unlock(ptl); __split_huge_pmd(walk->vma, pmd, addr, false, NULL); + ret = 2; goto out; } - if (!queue_pages_required(page, qp)) { - ret = 1; + if (!queue_pages_required(page, qp)) goto unlock; - } - ret = 1; flags = qp->flags; /* go to thp migration */ if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) { if (!vma_migratable(walk->vma)) { - ret = -EIO; + ret = 1; goto unlock; } @@ -479,6 +480,13 @@ out: /* * Scan through pages checking if pages follow certain conditions, * and move them to the pagelist if they do. + * + * queue_pages_pte_range() has three possible return values: + * 0 - pages are placed on the right node or queued successfully. + * 1 - there is unmovable page, and MPOL_MF_MOVE* & MPOL_MF_STRICT were + * specified. + * -EIO - only MPOL_MF_STRICT was specified and an existing page was already + * on a node that does not follow the policy. */ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, struct mm_walk *walk) @@ -488,17 +496,17 @@ static int queue_pages_pte_range(pmd_t * struct queue_pages *qp = walk->private; unsigned long flags = qp->flags; int ret; + bool has_unmovable = false; pte_t *pte; spinlock_t *ptl; ptl = pmd_trans_huge_lock(pmd, vma); if (ptl) { ret = queue_pages_pmd(pmd, ptl, addr, end, walk); - if (ret > 0) - return 0; - else if (ret < 0) + if (ret != 2) return ret; } + /* THP was split, fall through to pte walk */ if (pmd_trans_unstable(pmd)) return 0; @@ -519,14 +527,21 @@ static int queue_pages_pte_range(pmd_t * if (!queue_pages_required(page, qp)) continue; if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) { - if (!vma_migratable(vma)) + /* MPOL_MF_STRICT must be specified if we get here */ + if (!vma_migratable(vma)) { + has_unmovable = true; break; + } migrate_page_add(page, qp->pagelist, flags); } else break; } pte_unmap_unlock(pte - 1, ptl); cond_resched(); + + if (has_unmovable) + return 1; + return addr != end ? -EIO : 0; } @@ -639,7 +654,13 @@ static int queue_pages_test_walk(unsigne * * If pages found in a given range are on a set of nodes (determined by * @nodes and @flags,) it's isolated and queued to the pagelist which is - * passed via @private.) + * passed via @private. + * + * queue_pages_range() has three possible return values: + * 1 - there is unmovable page, but MPOL_MF_MOVE* & MPOL_MF_STRICT were + * specified. + * 0 - queue pages successfully or no misplaced page. + * -EIO - there is misplaced page and only MPOL_MF_STRICT was specified. */ static int queue_pages_range(struct mm_struct *mm, unsigned long start, unsigned long end, @@ -1182,6 +1203,7 @@ static long do_mbind(unsigned long start struct mempolicy *new; unsigned long end; int err; + int ret; LIST_HEAD(pagelist); if (flags & ~(unsigned long)MPOL_MF_VALID) @@ -1243,10 +1265,15 @@ static long do_mbind(unsigned long start if (err) goto mpol_out; - err = queue_pages_range(mm, start, end, nmask, + ret = queue_pages_range(mm, start, end, nmask, flags | MPOL_MF_INVERT, &pagelist); - if (!err) - err = mbind_range(mm, start, end, new); + + if (ret < 0) { + err = -EIO; + goto up_out; + } + + err = mbind_range(mm, start, end, new); if (!err) { int nr_failed = 0; @@ -1259,13 +1286,14 @@ static long do_mbind(unsigned long start putback_movable_pages(&pagelist); } - if (nr_failed && (flags & MPOL_MF_STRICT)) + if ((ret > 0) || (nr_failed && (flags & MPOL_MF_STRICT))) err = -EIO; } else putback_movable_pages(&pagelist); +up_out: up_write(&mm->mmap_sem); - mpol_out: +mpol_out: mpol_put(new); return err; } _

5 years, 4 months

1
0
0 0

[patch 03/18] mm/hmm: fix bad subpage pointer in try_to_unmap_one

by akpm＠linux-foundation.org

From: Ralph Campbell <rcampbell(a)nvidia.com> Subject: mm/hmm: fix bad subpage pointer in try_to_unmap_one When migrating an anonymous private page to a ZONE_DEVICE private page, the source page->mapping and page->index fields are copied to the destination ZONE_DEVICE struct page and the page_mapcount() is increased. This is so rmap_walk() can be used to unmap and migrate the page back to system memory. However, try_to_unmap_one() computes the subpage pointer from a swap pte which computes an invalid page pointer and a kernel panic results such as: BUG: unable to handle page fault for address: ffffea1fffffffc8 Currently, only single pages can be migrated to device private memory so no subpage computation is needed and it can be set to "page". [rcampbell(a)nvidia.com: add comment] Link: http://lkml.kernel.org/r/20190724232700.23327-4-rcampbell@nvidia.com Link: http://lkml.kernel.org/r/20190719192955.30462-4-rcampbell@nvidia.com Fixes: a5430dda8a3a1c ("mm/migrate: support un-addressable ZONE_DEVICE page in migration") Signed-off-by: Ralph Campbell <rcampbell(a)nvidia.com> Cc: "Jérôme Glisse" <jglisse(a)redhat.com> Cc: "Kirill A. Shutemov" <kirill.shutemov(a)linux.intel.com> Cc: Mike Kravetz <mike.kravetz(a)oracle.com> Cc: Christoph Hellwig <hch(a)lst.de> Cc: Jason Gunthorpe <jgg(a)mellanox.com> Cc: John Hubbard <jhubbard(a)nvidia.com> Cc: Andrea Arcangeli <aarcange(a)redhat.com> Cc: Andrey Ryabinin <aryabinin(a)virtuozzo.com> Cc: Christoph Lameter <cl(a)linux.com> Cc: Dan Williams <dan.j.williams(a)intel.com> Cc: Dave Hansen <dave.hansen(a)linux.intel.com> Cc: Ira Weiny <ira.weiny(a)intel.com> Cc: Jan Kara <jack(a)suse.cz> Cc: Lai Jiangshan <jiangshanlai(a)gmail.com> Cc: Logan Gunthorpe <logang(a)deltatee.com> Cc: Martin Schwidefsky <schwidefsky(a)de.ibm.com> Cc: Matthew Wilcox <willy(a)infradead.org> Cc: Mel Gorman <mgorman(a)techsingularity.net> Cc: Michal Hocko <mhocko(a)suse.com> Cc: Pekka Enberg <penberg(a)kernel.org> Cc: Randy Dunlap <rdunlap(a)infradead.org> Cc: Vlastimil Babka <vbabka(a)suse.cz> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> --- mm/rmap.c | 8 ++++++++ 1 file changed, 8 insertions(+) --- a/mm/rmap.c~mm-hmm-fix-bad-subpage-pointer-in-try_to_unmap_one +++ a/mm/rmap.c @@ -1475,7 +1475,15 @@ static bool try_to_unmap_one(struct page /* * No need to invalidate here it will synchronize on * against the special swap migration pte. + * + * The assignment to subpage above was computed from a + * swap PTE which results in an invalid pointer. + * Since only PAGE_SIZE pages can currently be + * migrated, just set it to page. This will need to be + * changed when hugepage migrations to device private + * memory are supported. */ + subpage = page; goto discard; } _

5 years, 4 months

1
0
0 0

[patch 02/18] mm/hmm: fix ZONE_DEVICE anon page mapping reuse

by akpm＠linux-foundation.org

From: Ralph Campbell <rcampbell(a)nvidia.com> Subject: mm/hmm: fix ZONE_DEVICE anon page mapping reuse When a ZONE_DEVICE private page is freed, the page->mapping field can be set. If this page is reused as an anonymous page, the previous value can prevent the page from being inserted into the CPU's anon rmap table. For example, when migrating a pte_none() page to device memory: migrate_vma(ops, vma, start, end, src, dst, private) migrate_vma_collect() src[] = MIGRATE_PFN_MIGRATE migrate_vma_prepare() /* no page to lock or isolate so OK */ migrate_vma_unmap() /* no page to unmap so OK */ ops->alloc_and_copy() /* driver allocates ZONE_DEVICE page for dst[] */ migrate_vma_pages() migrate_vma_insert_page() page_add_new_anon_rmap() __page_set_anon_rmap() /* This check sees the page's stale mapping field */ if (PageAnon(page)) return /* page->mapping is not updated */ The result is that the migration appears to succeed but a subsequent CPU fault will be unable to migrate the page back to system memory or worse. Clear the page->mapping field when freeing the ZONE_DEVICE page so stale pointer data doesn't affect future page use. Link: http://lkml.kernel.org/r/20190719192955.30462-3-rcampbell@nvidia.com Fixes: b7a523109fb5c9d2d6dd ("mm: don't clear ->mapping in hmm_devmem_free") Signed-off-by: Ralph Campbell <rcampbell(a)nvidia.com> Reviewed-by: John Hubbard <jhubbard(a)nvidia.com> Reviewed-by: Christoph Hellwig <hch(a)lst.de> Cc: Dan Williams <dan.j.williams(a)intel.com> Cc: Jason Gunthorpe <jgg(a)mellanox.com> Cc: Logan Gunthorpe <logang(a)deltatee.com> Cc: Ira Weiny <ira.weiny(a)intel.com> Cc: Matthew Wilcox <willy(a)infradead.org> Cc: Mel Gorman <mgorman(a)techsingularity.net> Cc: Jan Kara <jack(a)suse.cz> Cc: "Kirill A. Shutemov" <kirill.shutemov(a)linux.intel.com> Cc: Michal Hocko <mhocko(a)suse.com> Cc: Andrea Arcangeli <aarcange(a)redhat.com> Cc: Mike Kravetz <mike.kravetz(a)oracle.com> Cc: "Jérôme Glisse" <jglisse(a)redhat.com> Cc: Andrey Ryabinin <aryabinin(a)virtuozzo.com> Cc: Christoph Lameter <cl(a)linux.com> Cc: Dave Hansen <dave.hansen(a)linux.intel.com> Cc: Lai Jiangshan <jiangshanlai(a)gmail.com> Cc: Martin Schwidefsky <schwidefsky(a)de.ibm.com> Cc: Pekka Enberg <penberg(a)kernel.org> Cc: Randy Dunlap <rdunlap(a)infradead.org> Cc: Vlastimil Babka <vbabka(a)suse.cz> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> --- mm/memremap.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) --- a/mm/memremap.c~mm-hmm-fix-zone_device-anon-page-mapping-reuse +++ a/mm/memremap.c @@ -403,6 +403,30 @@ void __put_devmap_managed_page(struct pa mem_cgroup_uncharge(page); + /* + * When a device_private page is freed, the page->mapping field + * may still contain a (stale) mapping value. For example, the + * lower bits of page->mapping may still identify the page as + * an anonymous page. Ultimately, this entire field is just + * stale and wrong, and it will cause errors if not cleared. + * One example is: + * + * migrate_vma_pages() + * migrate_vma_insert_page() + * page_add_new_anon_rmap() + * __page_set_anon_rmap() + * ...checks page->mapping, via PageAnon(page) call, + * and incorrectly concludes that the page is an + * anonymous page. Therefore, it incorrectly, + * silently fails to set up the new anon rmap. + * + * For other types of ZONE_DEVICE pages, migration is either + * handled differently or not done at all, so there is no need + * to clear page->mapping. + */ + if (is_device_private_page(page)) + page->mapping = NULL; + page->pgmap->ops->page_free(page); } else if (!count) __put_page(page); _

5 years, 4 months

1
0
0 0

Re: [PATCH] mm: memcontrol: flush percpu vmevents before releasing memcg

by Roman Gushchin

On Tue, Aug 13, 2019 at 02:31:17PM -0700, Andrew Morton wrote: > On Mon, 12 Aug 2019 16:37:54 -0700 Roman Gushchin <guro(a)fb.com> wrote: > > > Similar to vmstats, percpu caching of local vmevents leads to an > > accumulation of errors on non-leaf levels. This happens because > > some leftovers may remain in percpu caches, so that they are > > never propagated up by the cgroup tree and just disappear into > > nonexistence with on releasing of the memory cgroup. > > > > To fix this issue let's accumulate and propagate percpu vmevents > > values before releasing the memory cgroup similar to what we're > > doing with vmstats. > > > > Since on cpu hotplug we do flush percpu vmstats anyway, we can > > iterate only over online cpus. > > > > Fixes: 42a300353577 ("mm: memcontrol: fix recursive statistics correctness & scalabilty") > > No cc:stable? > Here too, cc:stable is definitely missing. Adding now. Thanks!

5 years, 4 months

1
0
0 0

+ mm-page_alloc-move_freepages-should-not-examine-struct-page-of-reserved-memory.patch added to -mm tree

by akpm＠linux-foundation.org

The patch titled Subject: mm, page_alloc: move_freepages should not examine struct page of reserved memory has been added to the -mm tree. Its filename is mm-page_alloc-move_freepages-should-not-examine-struct-page-of-reserved-memory.patch This patch should soon appear at http://ozlabs.org/~akpm/mmots/broken-out/mm-page_alloc-move_freepages-shoul… and later at http://ozlabs.org/~akpm/mmotm/broken-out/mm-page_alloc-move_freepages-shoul… Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next and is updated there every 3-4 working days ------------------------------------------------------ From: David Rientjes <rientjes(a)google.com> Subject: mm, page_alloc: move_freepages should not examine struct page of reserved memory After commit 907ec5fca3dc ("mm: zero remaining unavailable struct pages"), struct page of reserved memory is zeroed. This causes page->flags to be 0 and fixes issues related to reading /proc/kpageflags, for example, of reserved memory. The VM_BUG_ON() in move_freepages_block(), however, assumes that page_zone() is meaningful even for reserved memory. That assumption is no longer true after the aforementioned commit. There's no reason why move_freepages_block() should be testing the legitimacy of page_zone() for reserved memory; its scope is limited only to pages on the zone's freelist. Note that pfn_valid() can be true for reserved memory: there is a backing struct page. The check for page_to_nid(page) is also buggy but reserved memory normally only appears on node 0 so the zeroing doesn't affect this. Move the debug checks to after verifying PageBuddy is true. This isolates the scope of the checks to only be for buddy pages which are on the zone's freelist which move_freepages_block() is operating on. In this case, an incorrect node or zone is a bug worthy of being warned about (and the examination of struct page is acceptable bcause this memory is not reserved). Why does move_freepages_block() gets called on reserved memory? It's simply math after finding a valid free page from the per-zone free area to use as fallback. We find the beginning and end of the pageblock of the valid page and that can bring us into memory that was reserved per the e820. pfn_valid() is still true (it's backed by a struct page), but since it's zero'd we shouldn't make any inferences here about comparing its node or zone. The current node check just happens to succeed most of the time by luck because reserved memory typically appears on node 0. The fix here is to validate that we actually have buddy pages before testing if there's any type of zone or node strangeness going on. Link: http://lkml.kernel.org/r/alpine.DEB.2.21.1908122036560.10779@chino.kir.corp… Fixes: 907ec5fca3dc ("mm: zero remaining unavailable struct pages") Signed-off-by: David Rientjes <rientjes(a)google.com> Cc: Naoya Horiguchi <n-horiguchi(a)ah.jp.nec.com> Cc: Masayoshi Mizuma <m.mizuma(a)jp.fujitsu.com> Cc: Oscar Salvador <osalvador(a)suse.de> Cc: Pavel Tatashin <pavel.tatashin(a)microsoft.com> Cc: Vlastimil Babka <vbabka(a)suse.cz> Cc: Naoya Horiguchi <n-horiguchi(a)ah.jp.nec.com> Cc: Mel Gorman <mgorman(a)techsingularity.net> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> --- mm/page_alloc.c | 19 ++++--------------- 1 file changed, 4 insertions(+), 15 deletions(-) --- a/mm/page_alloc.c~mm-page_alloc-move_freepages-should-not-examine-struct-page-of-reserved-memory +++ a/mm/page_alloc.c @@ -2238,27 +2238,12 @@ static int move_freepages(struct zone *z unsigned int order; int pages_moved = 0; -#ifndef CONFIG_HOLES_IN_ZONE - /* - * page_zone is not safe to call in this context when - * CONFIG_HOLES_IN_ZONE is set. This bug check is probably redundant - * anyway as we check zone boundaries in move_freepages_block(). - * Remove at a later date when no bug reports exist related to - * grouping pages by mobility - */ - VM_BUG_ON(pfn_valid(page_to_pfn(start_page)) && - pfn_valid(page_to_pfn(end_page)) && - page_zone(start_page) != page_zone(end_page)); -#endif for (page = start_page; page <= end_page;) { if (!pfn_valid_within(page_to_pfn(page))) { page++; continue; } - /* Make sure we are not inadvertently changing nodes */ - VM_BUG_ON_PAGE(page_to_nid(page) != zone_to_nid(zone), page); - if (!PageBuddy(page)) { /* * We assume that pages that could be isolated for @@ -2273,6 +2258,10 @@ static int move_freepages(struct zone *z continue; } + /* Make sure we are not inadvertently changing nodes */ + VM_BUG_ON_PAGE(page_to_nid(page) != zone_to_nid(zone), page); + VM_BUG_ON_PAGE(page_zone(page) != zone, page); + order = page_order(page); move_to_free_area(page, &zone->free_area[order], migratetype); page += 1 << order; _ Patches currently in -mm which might be from rientjes(a)google.com are mm-page_alloc-move_freepages-should-not-examine-struct-page-of-reserved-memory.patch

5 years, 4 months

1
0
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-stable-mirror August 2019