- Linux-kselftest-mirror - lists.linaro.org

[PATCH net-next v2 0/2] selftests/ptp: Add support for new timestamp IOCTLs

by Alex Maftei

PTP_SYS_OFFSET_EXTENDED was added in November 2018 in 361800876f80 (" ptp: add PTP_SYS_OFFSET_EXTENDED ioctl") and PTP_SYS_OFFSET_PRECISE was added in February 2016 in 719f1aa4a671 ("ptp: Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping") The PTP selftest code is lacking support for these two IOCTLS. This short series of patches adds support for them. Changes in v2: - Fixed rebase issues (v1 somehow ended up with patch 1 being from the first manual split of my changes and patch 2 being from rebase 2 out of 3) - Rebased on top of net-next Alex Maftei (2): selftests/ptp: Add -x option for testing PTP_SYS_OFFSET_EXTENDED selftests/ptp: Add -X option for testing PTP_SYS_OFFSET_PRECISE tools/testing/selftests/ptp/testptp.c | 73 ++++++++++++++++++++++++++- 1 file changed, 71 insertions(+), 2 deletions(-) -- 2.25.1

2 years, 1 month

3
4
0 0

[PATCH v10 0/6] Add IO page table replacement support

by Nicolin Chen

[ This series depends on the VFIO device cdev series ] Changelog v10: * Added Reviewed-by from Jason * Replaced the iommufd_ref_to_user call with refcount_inc * Added a wrapper iommufd_access_change_ioas_id and used it in the iommufd_access_attach() and iommufd_access_replace() APIs v9: https://lore.kernel.org/linux-iommu/cover.1690440730.git.nicolinc@nvidia.co… * Rebased on top of Jason's iommufd for-next tree * Added Reviewed-by from Jason and Alex * Reworked the replace API patches * Added a new patch allowing passing in to iopt_remove_access * Added a new patch of a helper function following Jason's design, mainly by blocking any concurrent detach/replace and keeping the refcount_dec at the end of the function * Added a call of the new helper in iommufd_access_destroy_object() to reduce race condition * Simplified the replace API patch v8: https://lore.kernel.org/all/cover.1690226015.git.nicolinc@nvidia.com/ * Rebased on top of Jason's iommufd_hwpt series and then cdev v15 series: https://lore.kernel.org/all/0-v8-6659224517ea+532-iommufd_alloc_jgg@nvidia.… https://lore.kernel.org/kvm/20230718135551.6592-1-yi.l.liu@intel.com/ * Changed the order of detach() and attach() in replace(), to fix a bug v7: https://lore.kernel.org/all/cover.1683593831.git.nicolinc@nvidia.com/ * Rebased on top of v6.4-rc1 and cdev v11 candidate * Fixed a wrong file in replace() API patch * Added Kevin's "Reviewed-by" to replace() API patch v6: https://lore.kernel.org/all/cover.1679939952.git.nicolinc@nvidia.com/ * Rebased on top of cdev v8 series https://lore.kernel.org/kvm/20230327094047.47215-1-yi.l.liu@intel.com/ * Added "Reviewed-by" from Kevin to PATCH-4 * Squashed access->ioas updating lines into iommufd_access_change_pt(), and changed function return type accordingly for simplification. v5: https://lore.kernel.org/all/cover.1679559476.git.nicolinc@nvidia.com/ * Kept the cmd->id in the iommufd_test_create_access() so the access can be created with an ioas by default. Then, renamed the previous ioctl IOMMU_TEST_OP_ACCESS_SET_IOAS to IOMMU_TEST_OP_ACCESS_REPLACE_IOAS, so it would be used to replace an access->ioas pointer. * Added iommufd_access_replace() API after the introductions of the other two APIs iommufd_access_attach() and iommufd_access_detach(). * Since vdev->iommufd_attached is also set in emulated pathway too, call iommufd_access_update(), similar to the physical pathway. v4: https://lore.kernel.org/all/cover.1678284812.git.nicolinc@nvidia.com/ * Rebased on top of Jason's series adding replace() and hwpt_alloc() https://lore.kernel.org/all/0-v2-51b9896e7862+8a8c-iommufd_alloc_jgg@nvidia… * Rebased on top of cdev series v6 https://lore.kernel.org/kvm/20230308132903.465159-1-yi.l.liu@intel.com/ * Dropped the patch that's moved to cdev series. * Added unmap function pointer sanity before calling it. * Added "Reviewed-by" from Kevin and Yi. * Added back the VFIO change updating the ATTACH uAPI. v3: https://lore.kernel.org/all/cover.1677288789.git.nicolinc@nvidia.com/ * Rebased on top of Jason's iommufd_hwpt branch: https://lore.kernel.org/all/0-v2-406f7ac07936+6a-iommufd_hwpt_jgg@nvidia.co… * Dropped patches from this series accordingly. There were a couple of VFIO patches that will be submitted after the VFIO cdev series. Also, renamed the series to be "emulated". * Moved dma_unmap sanity patch to the first in the series. * Moved dma_unmap sanity to cover both VFIO and IOMMUFD pathways. * Added Kevin's "Reviewed-by" to two of the patches. * Fixed a NULL pointer bug in vfio_iommufd_emulated_bind(). * Moved unmap() call to the common place in iommufd_access_set_ioas(). v2: https://lore.kernel.org/all/cover.1675802050.git.nicolinc@nvidia.com/ * Rebased on top of vfio_device cdev v2 series. * Update the kdoc and commit message of iommu_group_replace_domain(). * Dropped revert-to-core-domain part in iommu_group_replace_domain(). * Dropped !ops->dma_unmap check in vfio_iommufd_emulated_attach_ioas(). * Added missing rc value in vfio_iommufd_emulated_attach_ioas() from the iommufd_access_set_ioas() call. * Added a new patch in vfio_main to deny vfio_pin/unpin_pages() calls if vdev->ops->dma_unmap is not implemented. * Added a __iommmufd_device_detach helper and let the replace routine do a partial detach(). * Added restriction on auto_domains to use the replace feature. * Added the patch "iommufd/device: Make hwpt_list list_add/del symmetric" from the has_group removal series. v1: https://lore.kernel.org/all/cover.1675320212.git.nicolinc@nvidia.com/ Hi all, The existing IOMMU APIs provide a pair of functions: iommu_attach_group() for callers to attach a device from the default_domain (NULL if not being supported) to a given iommu domain, and iommu_detach_group() for callers to detach a device from a given domain to the default_domain. Internally, the detach_dev op is deprecated for the newer drivers with default_domain. This means that those drivers likely can switch an attaching domain to another one, without stagging the device at a blocking or default domain, for use cases such as: 1) vPASID mode, when a guest wants to replace a single pasid (PASID=0) table with a larger table (PASID=N) 2) Nesting mode, when switching the attaching device from an S2 domain to an S1 domain, or when switching between relevant S1 domains. This series is rebased on top of Jason Gunthorpe's series that introduces iommu_group_replace_domain API and IOMMUFD infrastructure for the IOMMUFD "physical" devices. The IOMMUFD "emulated" deivces will need some extra steps to replace the access->ioas object and its iopt pointer. You can also find this series on Github: https://github.com/nicolinc/iommufd/commits/iommu_group_replace_domain-v10 Thank you Nicolin Chen Nicolin Chen (6): vfio: Do not allow !ops->dma_unmap in vfio_pin/unpin_pages() iommufd: Allow passing in iopt_access_list_id to iopt_remove_access() iommufd: Add iommufd_access_change_ioas(_id) helpers iommufd: Add iommufd_access_replace() API iommufd/selftest: Add IOMMU_TEST_OP_ACCESS_REPLACE_IOAS coverage vfio: Support IO page table replacement drivers/iommu/iommufd/device.c | 132 ++++++++++++------ drivers/iommu/iommufd/io_pagetable.c | 6 +- drivers/iommu/iommufd/iommufd_private.h | 3 +- drivers/iommu/iommufd/iommufd_test.h | 4 + drivers/iommu/iommufd/selftest.c | 19 +++ drivers/vfio/iommufd.c | 11 +- drivers/vfio/vfio_main.c | 4 + include/linux/iommufd.h | 1 + include/uapi/linux/vfio.h | 6 + tools/testing/selftests/iommu/iommufd.c | 29 +++- tools/testing/selftests/iommu/iommufd_utils.h | 19 +++ 11 files changed, 184 insertions(+), 50 deletions(-) -- 2.41.0

2 years, 1 month

2
12
0 0

[PATCH v7 0/4] RISC-V: mm: Make SV48 the default address space

by Charlie Jenkins

Make sv48 the default address space for mmap as some applications currently depend on this assumption. Users can now select a desired address space using a non-zero hint address to mmap. Previously, requesting the default address space from mmap by passing zero as the hint address would result in using the largest address space possible. Some applications depend on empty bits in the virtual address space, like Go and Java, so this patch provides more flexibility for application developers. -Charlie --- v7: - Changing RLIMIT_STACK inside of an executing program does not trigger arch_pick_mmap_layout(), so rewrite tests to change RLIMIT_STACK from a script before executing tests. RLIMIT_STACK of infinity forces bottomup mmap allocation. - Make arch_get_mmap_base macro more readible by extracting out the rnd calculation. - Use MMAP_MIN_VA_BITS in TASK_UNMAPPED_BASE to support case when mmap attempts to allocate address smaller than DEFAULT_MAP_WINDOW. - Fix incorrect wording in documentation. v6: - Rebase onto the correct base v5: - Minor wording change in documentation - Change some parenthesis in arch_get_mmap_ macros - Added case for addr==0 in arch_get_mmap_ because without this, programs would crash if RLIMIT_STACK was modified before executing the program. This was tested using the libhugetlbfs tests. v4: - Split testcases/document patch into test cases, in-code documentation, and formal documentation patches - Modified the mmap_base macro to be more legible and better represent memory layout - Fixed documentation to better reflect the implmentation - Renamed DEFAULT_VA_BITS to MMAP_VA_BITS - Added additional test case for rlimit changes --- Charlie Jenkins (4): RISC-V: mm: Restrict address space for sv39,sv48,sv57 RISC-V: mm: Add tests for RISC-V mm RISC-V: mm: Update pgtable comment documentation RISC-V: mm: Document mmap changes Documentation/riscv/vm-layout.rst | 22 +++++++ arch/riscv/include/asm/elf.h | 2 +- arch/riscv/include/asm/pgtable.h | 21 ++++-- arch/riscv/include/asm/processor.h | 47 ++++++++++++-- tools/testing/selftests/riscv/Makefile | 2 +- tools/testing/selftests/riscv/mm/.gitignore | 2 + tools/testing/selftests/riscv/mm/Makefile | 15 +++++ .../riscv/mm/testcases/mmap_bottomup.c | 35 ++++++++++ .../riscv/mm/testcases/mmap_default.c | 35 ++++++++++ .../selftests/riscv/mm/testcases/mmap_test.h | 64 +++++++++++++++++++ .../selftests/riscv/mm/testcases/run_mmap.sh | 12 ++++ 11 files changed, 244 insertions(+), 13 deletions(-) create mode 100644 tools/testing/selftests/riscv/mm/.gitignore create mode 100644 tools/testing/selftests/riscv/mm/Makefile create mode 100644 tools/testing/selftests/riscv/mm/testcases/mmap_bottomup.c create mode 100644 tools/testing/selftests/riscv/mm/testcases/mmap_default.c create mode 100644 tools/testing/selftests/riscv/mm/testcases/mmap_test.h create mode 100755 tools/testing/selftests/riscv/mm/testcases/run_mmap.sh -- 2.41.0

2 years, 1 month

2
6
0 0

[PATCH v9 0/6] cover-letter: Add IO page table replacement support

by Nicolin Chen

[ This series depends on the VFIO device cdev series ] Changelog v9: * Rebased on top of Jason's iommufd for-next tree * Added Reviewed-by from Jason and Alex * Reworked the replace API patches * Added a new patch allowing passing in to iopt_remove_access * Added a new patch of a helper function following Jason's design, mainly by blocking any concurrent detach/replace and keeping the refcount_dec at the end of the function * Added a call of the new helper in iommufd_access_destroy_object() to reduce race condition * Simplified the replace API patch v8: https://lore.kernel.org/all/cover.1690226015.git.nicolinc@nvidia.com/ * Rebased on top of Jason's iommufd_hwpt series and then cdev v15 series: https://lore.kernel.org/all/0-v8-6659224517ea+532-iommufd_alloc_jgg@nvidia.… https://lore.kernel.org/kvm/20230718135551.6592-1-yi.l.liu@intel.com/ * Changed the order of detach() and attach() in replace(), to fix a bug v7: https://lore.kernel.org/all/cover.1683593831.git.nicolinc@nvidia.com/ * Rebased on top of v6.4-rc1 and cdev v11 candidate * Fixed a wrong file in replace() API patch * Added Kevin's "Reviewed-by" to replace() API patch v6: https://lore.kernel.org/all/cover.1679939952.git.nicolinc@nvidia.com/ * Rebased on top of cdev v8 series https://lore.kernel.org/kvm/20230327094047.47215-1-yi.l.liu@intel.com/ * Added "Reviewed-by" from Kevin to PATCH-4 * Squashed access->ioas updating lines into iommufd_access_change_pt(), and changed function return type accordingly for simplification. v5: https://lore.kernel.org/all/cover.1679559476.git.nicolinc@nvidia.com/ * Kept the cmd->id in the iommufd_test_create_access() so the access can be created with an ioas by default. Then, renamed the previous ioctl IOMMU_TEST_OP_ACCESS_SET_IOAS to IOMMU_TEST_OP_ACCESS_REPLACE_IOAS, so it would be used to replace an access->ioas pointer. * Added iommufd_access_replace() API after the introductions of the other two APIs iommufd_access_attach() and iommufd_access_detach(). * Since vdev->iommufd_attached is also set in emulated pathway too, call iommufd_access_update(), similar to the physical pathway. v4: https://lore.kernel.org/all/cover.1678284812.git.nicolinc@nvidia.com/ * Rebased on top of Jason's series adding replace() and hwpt_alloc() https://lore.kernel.org/all/0-v2-51b9896e7862+8a8c-iommufd_alloc_jgg@nvidia… * Rebased on top of cdev series v6 https://lore.kernel.org/kvm/20230308132903.465159-1-yi.l.liu@intel.com/ * Dropped the patch that's moved to cdev series. * Added unmap function pointer sanity before calling it. * Added "Reviewed-by" from Kevin and Yi. * Added back the VFIO change updating the ATTACH uAPI. v3: https://lore.kernel.org/all/cover.1677288789.git.nicolinc@nvidia.com/ * Rebased on top of Jason's iommufd_hwpt branch: https://lore.kernel.org/all/0-v2-406f7ac07936+6a-iommufd_hwpt_jgg@nvidia.co… * Dropped patches from this series accordingly. There were a couple of VFIO patches that will be submitted after the VFIO cdev series. Also, renamed the series to be "emulated". * Moved dma_unmap sanity patch to the first in the series. * Moved dma_unmap sanity to cover both VFIO and IOMMUFD pathways. * Added Kevin's "Reviewed-by" to two of the patches. * Fixed a NULL pointer bug in vfio_iommufd_emulated_bind(). * Moved unmap() call to the common place in iommufd_access_set_ioas(). v2: https://lore.kernel.org/all/cover.1675802050.git.nicolinc@nvidia.com/ * Rebased on top of vfio_device cdev v2 series. * Update the kdoc and commit message of iommu_group_replace_domain(). * Dropped revert-to-core-domain part in iommu_group_replace_domain(). * Dropped !ops->dma_unmap check in vfio_iommufd_emulated_attach_ioas(). * Added missing rc value in vfio_iommufd_emulated_attach_ioas() from the iommufd_access_set_ioas() call. * Added a new patch in vfio_main to deny vfio_pin/unpin_pages() calls if vdev->ops->dma_unmap is not implemented. * Added a __iommmufd_device_detach helper and let the replace routine do a partial detach(). * Added restriction on auto_domains to use the replace feature. * Added the patch "iommufd/device: Make hwpt_list list_add/del symmetric" from the has_group removal series. v1: https://lore.kernel.org/all/cover.1675320212.git.nicolinc@nvidia.com/ Hi all, The existing IOMMU APIs provide a pair of functions: iommu_attach_group() for callers to attach a device from the default_domain (NULL if not being supported) to a given iommu domain, and iommu_detach_group() for callers to detach a device from a given domain to the default_domain. Internally, the detach_dev op is deprecated for the newer drivers with default_domain. This means that those drivers likely can switch an attaching domain to another one, without stagging the device at a blocking or default domain, for use cases such as: 1) vPASID mode, when a guest wants to replace a single pasid (PASID=0) table with a larger table (PASID=N) 2) Nesting mode, when switching the attaching device from an S2 domain to an S1 domain, or when switching between relevant S1 domains. This series is rebased on top of Jason Gunthorpe's series that introduces iommu_group_replace_domain API and IOMMUFD infrastructure for the IOMMUFD "physical" devices. The IOMMUFD "emulated" deivces will need some extra steps to replace the access->ioas object and its iopt pointer. You can also find this series on Github: https://github.com/nicolinc/iommufd/commits/iommu_group_replace_domain-v9 Thank you Nicolin Chen Nicolin Chen (6): vfio: Do not allow !ops->dma_unmap in vfio_pin/unpin_pages() iommufd: Allow passing in iopt_access_list_id to iopt_remove_access() iommufd: Add iommufd_access_change_ioas helper iommufd: Add iommufd_access_replace() API iommufd/selftest: Add IOMMU_TEST_OP_ACCESS_REPLACE_IOAS coverage vfio: Support IO page table replacement drivers/iommu/iommufd/device.c | 123 ++++++++++++------ drivers/iommu/iommufd/io_pagetable.c | 6 +- drivers/iommu/iommufd/iommufd_private.h | 3 +- drivers/iommu/iommufd/iommufd_test.h | 4 + drivers/iommu/iommufd/selftest.c | 19 +++ drivers/vfio/iommufd.c | 11 +- drivers/vfio/vfio_main.c | 4 + include/linux/iommufd.h | 1 + include/uapi/linux/vfio.h | 6 + tools/testing/selftests/iommu/iommufd.c | 29 ++++- tools/testing/selftests/iommu/iommufd_utils.h | 19 +++ 11 files changed, 175 insertions(+), 50 deletions(-) -- 2.41.0

2 years, 1 month

2
10
0 0

Re: fs/proc/task_mmu: Implement IOCTL for efficient page table scanning

by Muhammad Usama Anjum

On 7/21/23 12:28 AM, Michał Mirosław wrote: > This is a massaged version of patch by Muhammad Usama Anjum [1] > to illustrate my review comments and hopefully push the implementation > efforts closer to conclusion. The changes are: Thank you so much for this effort. I also want to reach conclusion. I'll agree with all the changes which don't affect me. But some requirements aren't being fulfilled with this current design. > > 1. the API: > > a. return ranges as {begin, end} instead of {begin + len}; Agreed. > b. rename match "flags" to 'page categories' everywhere - this makes > it easier to differentiate the ioctl()s categorisation of pages > from struct page flags; I've no problem with it. #define PAGE_IS_WPASYNC (1 << 0) #define PAGE_IS_WRITTEN (1 << 1) You have another new flag PAGE_IS_WPASYNC. But there is no application of PAGE_IS_WPASYNC. We must not add a flag which don't have any user. > c. change {required + excluded} to {inverted + required}. This was > rejected before, but I'd like to illustrate the difference. > Old interface can be translated to the new by: > categories_inverted = excluded_mask > categories_mask = required_mask | excluded_mask > categories_anyof_mask = anyof_mask > The new way allows filtering by: A & (B | !C) > categories_inverted = C > categories_mask = A > categories_anyof_mask = B | C I'm still unable to get the idea of inverted masks. IIRC Andei had also not supported/accepted this masking scheme. But I'll be okay with it if he supports this masking. > d. change the ioctl to be a SCAN with optional WP. Addressing the > original use-case, GetWriteWatch() can be implemented as: As I've mentioned several times previously (without the name of ResetWriteWatch()) that we need exclusive WP without GET. This could be implemented with UFFDIO_WRITEPROTECT. But when we use UFFDIO_WRITEPROTECT, we hit some special case and performance is very slow. So with UFFD WP expert, Peter Xu we have decided to put exclusive WP in this IOCTL for implementation of ResetWriteWatch(). A lot of simplification of the patch is made possible because of not keeping exclusive WP. (You have also written some quality code, more better.) > > memset(&args, 0, sizeof(args)); > args.start = lpBaseAddress; > args.end = lpBaseAddress + dwRegionSize; > args.max_pages = *lpdwCount; > *lpdwGranularity = PAGE_SIZE; > args.flags = PM_SCAN_CHECK_WPASYNC; > if (dwFlags & WRITE_WATCH_FLAG_RESET) > args.flags |= PM_SCAN_WP_MATCHED; > args.categories_mask = PAGE_IS_WRITTEN; > args.return_mask = PAGE_IS_WRITTEN; > args.vec = tmp_buffer; > args.vec_len = tmp_buffer_len; > do { > n = ioctl(..., &args); > if (n < 0) > return error(n); > for (page_region *p = ...) { > write page addresses; > } > if (output_full) > return ...; > } while (args.start != args.end); > return ...; > > For the CRIU's usecase of live migration this would be similar, > but without using max_pages limit. The other CRIU usecase is to > efficiently dump sparse memory mappings - those could use the > full range of page category filtering as needed in a checkpoint > phase.a > e. allow no-op calls I've no issue with it. > > 2. the implementation: > a. gather the page-categorising and write-protecting code in one place; Agreed. > b. optimization: add whole-vma skipping for WP usecase; I don't know who can benefit from it. Do you have any user in mind? When the user come of this optimization, this can be added later. > c. extracted output limiting code to pagemap_scan_output(); If user passes half THP, current code wouldn't split huge page and WP the whole THP. We would loose the dirty state of other half huge page. This is bug. consoliding the output limiting code looks optimal, but we'll need to same limiting code to detect if full THP hasn't been passed in case of THP and HugeTLB. > d. extracted range coalescing to pagemap_scan_push_range(); My old pagemap_scan_output has become pagemap_scan_push_range(). > e. extracted THP entry handling to pagemap_scan_thp_entry(); Good. But I didn't found value in seperating it just like other historic pagemap code. > f. added a shortcut for non-WP hugetlb scan; avoids conditional > locking; Yeah, some if conditions have been removed. But overall did couple of calls to is_interesting and scan_output functions instead of just one. > g. extracted scan buffer handling code out of do_pagemap_scan(); > h. rework output code to always try to write pending ranges; if EFAULT > is generated it always overwrites the original error code; > (the case of SIGKILL is needlessly trying to write the output > now, but this should be rare case and ignoring it makes the code > not needing a goto) > > PTAL and comment/fix. Compile tested with allyesconfig. For the (unlikely) > case that the patch is good as is: > > Signed-off-by: Muhammad Usama Anjum <usama.anjum(a)collabora.com> > Signed-off-by: Michał Mirosław <mirq-linux(a)rere.qmqm.pl> > > [1] https://lore.kernel.org/lkml/20230713101415.108875-3-usama.anjum@collabora.… > "[PATCH v25 2/5] fs/proc/task_mmu: Implement IOCTL to get and > optionally clear info about PTEs" > > Best Regards > Michał Mirosław > > Signed-off-by: Michał Mirosław <mirq-linux(a)rere.qmqm.pl> > --- > fs/proc/task_mmu.c | 649 ++++++++++++++++++++++++++++++++++++++++ > include/linux/hugetlb.h | 1 + > include/uapi/linux/fs.h | 56 ++++ > mm/hugetlb.c | 2 +- > 4 files changed, 707 insertions(+), 1 deletion(-) > > diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c > index c1e6531cb02a..7d624bafecf9 100644 > --- a/fs/proc/task_mmu.c > +++ b/fs/proc/task_mmu.c > @@ -19,6 +19,8 @@ > #include <linux/shmem_fs.h> > #include <linux/uaccess.h> > #include <linux/pkeys.h> > +#include <linux/minmax.h> > +#include <linux/overflow.h> > > #include <asm/elf.h> > #include <asm/tlb.h> > @@ -1749,11 +1751,658 @@ static int pagemap_release(struct inode *inode, struct file *file) > return 0; > } > > +#define PM_SCAN_CATEGORIES (PAGE_IS_WPASYNC | PAGE_IS_WRITTEN | \ > + PAGE_IS_FILE | PAGE_IS_PRESENT | \ > + PAGE_IS_SWAPPED | PAGE_IS_PFNZERO) > +#define PM_SCAN_FLAGS (PM_SCAN_WP_MATCHING | PM_SCAN_CHECK_WPASYNC) > + > +struct pagemap_scan_private { > + struct pm_scan_arg arg; > + unsigned long cur_vma_category; > + struct page_region *vec_buf, cur_buf; > + unsigned long vec_buf_len, vec_buf_index, found_pages, end_addr; > + struct page_region __user* vec_out; > +}; > + > +static unsigned long pagemap_page_category(struct vm_area_struct *vma, unsigned long addr, pte_t pte) > +{ > + unsigned long categories = 0; > + > + if (pte_present(pte)) { > + struct page *page = vm_normal_page(vma, addr, pte); > + > + categories |= PAGE_IS_PRESENT; > + if (!pte_uffd_wp(pte)) > + categories |= PAGE_IS_WRITTEN; > + if (page && !PageAnon(page)) > + categories |= PAGE_IS_FILE; > + if (is_zero_pfn(pte_pfn(pte))) > + categories |= PAGE_IS_PFNZERO; > + } else if (is_swap_pte(pte)) { > + swp_entry_t swp = pte_to_swp_entry(pte); > + > + categories |= PAGE_IS_SWAPPED; > + if (!pte_swp_uffd_wp_any(pte)) > + categories |= PAGE_IS_WRITTEN; > + if (is_pfn_swap_entry(swp) && !PageAnon(pfn_swap_entry_to_page(swp))) > + categories |= PAGE_IS_FILE; > + } > + > + return categories; > +} > + > +static void make_uffd_wp_pte(struct vm_area_struct *vma, > + unsigned long addr, pte_t *pte) > +{ > + pte_t ptent = ptep_get(pte); > + > + if (pte_present(ptent)) { > + pte_t old_pte; > + > + old_pte = ptep_modify_prot_start(vma, addr, pte); > + ptent = pte_mkuffd_wp(ptent); > + ptep_modify_prot_commit(vma, addr, pte, old_pte, ptent); > + } else if (is_swap_pte(ptent)) { > + ptent = pte_swp_mkuffd_wp(ptent); > + set_pte_at(vma->vm_mm, addr, pte, ptent); > + } else { > + set_pte_at(vma->vm_mm, addr, pte, > + make_pte_marker(PTE_MARKER_UFFD_WP)); > + } > +} > + > +#ifdef CONFIG_TRANSPARENT_HUGEPAGE > +static unsigned long pagemap_thp_category(pmd_t pmd) > +{ > + unsigned long categories = 0; > + > + if (pmd_present(pmd)) { > + categories |= PAGE_IS_PRESENT; > + if (!pmd_uffd_wp(pmd)) > + categories |= PAGE_IS_WRITTEN; > + if (is_zero_pfn(pmd_pfn(pmd))) > + categories |= PAGE_IS_PFNZERO; > + } else if (is_swap_pmd(pmd)) { > + categories |= PAGE_IS_SWAPPED; > + if (!pmd_swp_uffd_wp(pmd)) > + categories |= PAGE_IS_WRITTEN; > + } > + > + return categories; > +} > + > +static void make_uffd_wp_pmd(struct vm_area_struct *vma, > + unsigned long addr, pmd_t *pmdp) > +{ > + pmd_t old, pmd = *pmdp; > + > + if (pmd_present(pmd)) { > + old = pmdp_invalidate_ad(vma, addr, pmdp); > + pmd = pmd_mkuffd_wp(old); > + set_pmd_at(vma->vm_mm, addr, pmdp, pmd); > + } else if (is_migration_entry(pmd_to_swp_entry(pmd))) { > + pmd = pmd_swp_mkuffd_wp(pmd); > + set_pmd_at(vma->vm_mm, addr, pmdp, pmd); > + } > +} > +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ > + > +#ifdef CONFIG_HUGETLB_PAGE > +static unsigned long pagemap_hugetlb_category(pte_t pte) > +{ > + unsigned long categories = 0; > + > + if (pte_present(pte)) { > + categories |= PAGE_IS_PRESENT; > + if (!huge_pte_uffd_wp(pte)) > + categories |= PAGE_IS_WRITTEN; > + if (!PageAnon(pte_page(pte))) > + categories |= PAGE_IS_FILE; > + if (is_zero_pfn(pte_pfn(pte))) > + categories |= PAGE_IS_PFNZERO; > + } else if (is_swap_pte(pte)) { > + categories |= PAGE_IS_SWAPPED; > + if (!pte_swp_uffd_wp_any(pte)) > + categories |= PAGE_IS_WRITTEN; > + } > + > + return categories; > +} > + > +static void make_uffd_wp_huge_pte(struct vm_area_struct *vma, > + unsigned long addr, pte_t *ptep, > + pte_t ptent) > +{ > + if (is_hugetlb_entry_hwpoisoned(ptent) || is_pte_marker(ptent)) > + return; > + > + if (is_hugetlb_entry_migration(ptent)) > + set_huge_pte_at(vma->vm_mm, addr, ptep, > + pte_swp_mkuffd_wp(ptent)); > + else if (!huge_pte_none(ptent)) > + huge_ptep_modify_prot_commit(vma, addr, ptep, ptent, > + huge_pte_mkuffd_wp(ptent)); > + else > + set_huge_pte_at(vma->vm_mm, addr, ptep, > + make_pte_marker(PTE_MARKER_UFFD_WP)); > +} > +#endif /* CONFIG_HUGETLB_PAGE */ > + > +static bool pagemap_scan_is_interesting_page(unsigned long categories, > + const struct pagemap_scan_private *p) > +{ > + categories ^= p->arg.category_inverted; > + if ((categories & p->arg.category_mask) != p->arg.category_mask) > + return false; > + if (p->arg.category_anyof_mask && !(categories & p->arg.category_anyof_mask)) > + return false; > + > + return true; > +} > + > +static bool pagemap_scan_is_interesting_vma(unsigned long categories, > + const struct pagemap_scan_private *p) > +{ > + unsigned long required = p->arg.category_mask & PAGE_IS_WPASYNC; > + > + categories ^= p->arg.category_inverted; > + if ((categories & required) != required) > + return false; > + return true; > +} > + > +static int pagemap_scan_test_walk(unsigned long start, unsigned long end, > + struct mm_walk *walk) > +{ > + struct pagemap_scan_private *p = walk->private; > + struct vm_area_struct *vma = walk->vma; > + unsigned long vma_category = 0; > + > + if (userfaultfd_wp_async(vma) && userfaultfd_wp_use_markers(vma)) > + vma_category |= PAGE_IS_WPASYNC; > + else if (p->arg.flags & PM_SCAN_CHECK_WPASYNC) > + return -EPERM; > + > + if (vma->vm_flags & VM_PFNMAP) > + return 1; > + > + if (!pagemap_scan_is_interesting_vma(vma_category, p)) > + return 1; > + > + p->cur_vma_category = vma_category; > + return 0; > +} > + > +static bool pagemap_scan_push_range(unsigned long categories, > + struct pagemap_scan_private *p, > + unsigned long addr, unsigned long end) > +{ > + struct page_region *cur_buf = &p->cur_buf; > + > + /* > + * When there is no output buffer provided at all, the sentinel values > + * won't match here. There is no other way for `cur_buf->end` to be > + * non-zero other than it being non-empty. > + */ > + if (addr == cur_buf->end && categories == cur_buf->categories) { > + cur_buf->end = end; > + return true; > + } > + > + if (cur_buf->end) { > + if (p->vec_buf_index >= p->vec_buf_len) > + return false; > + > + memcpy(&p->vec_buf[p->vec_buf_index], cur_buf, > + sizeof(*p->vec_buf)); > + ++p->vec_buf_index; > + } > + > + cur_buf->start = addr; > + cur_buf->end = end; > + cur_buf->categories = categories; > + return true; > +} > + > +static void pagemap_scan_backout_range(struct pagemap_scan_private *p, > + unsigned long addr, unsigned long end) > +{ > + struct page_region *cur_buf = &p->cur_buf; > + > + if (cur_buf->start != addr) { > + cur_buf->end = addr; > + } else { > + cur_buf->start = cur_buf->end = 0; > + } > + > + p->end_addr = 0; > +} > + > +static int pagemap_scan_output(unsigned long categories, > + struct pagemap_scan_private *p, > + unsigned long addr, unsigned long *end) > +{ > + unsigned long n_pages, total_pages; > + int ret = 0; > + > + if (!p->arg.return_mask) > + return 0; > + > + categories &= p->arg.return_mask; > + > + n_pages = (*end - addr) / PAGE_SIZE; > + if (check_add_overflow(p->found_pages, n_pages, &total_pages) || total_pages > p->arg.max_pages) { > + size_t n_too_much = total_pages - p->arg.max_pages; > + *end -= n_too_much * PAGE_SIZE; > + n_pages -= n_too_much; > + ret = -ENOSPC; > + } > + > + if (!pagemap_scan_push_range(categories, p, addr, *end)) { > + *end = addr; > + n_pages = 0; > + ret = -ENOSPC; > + } > + > + p->found_pages += n_pages; > + if (ret) > + p->end_addr = *end; > + return ret; > +} > + > +static int pagemap_scan_thp_entry(pmd_t *pmd, unsigned long start, > + unsigned long end, struct mm_walk *walk) > +{ > +#ifdef CONFIG_TRANSPARENT_HUGEPAGE > + struct pagemap_scan_private *p = walk->private; > + struct vm_area_struct *vma = walk->vma; > + unsigned long categories; > + spinlock_t *ptl; > + int ret = 0; > + > + ptl = pmd_trans_huge_lock(pmd, vma); > + if (!ptl) > + return -ENOENT; > + > + categories = p->cur_vma_category | pagemap_thp_category(*pmd); > + > + if (!pagemap_scan_is_interesting_page(categories, p)) > + goto out_unlock; > + > + ret = pagemap_scan_output(categories, p, start, &end); > + if (start == end) > + goto out_unlock; > + > + if (~p->arg.flags & PM_SCAN_WP_MATCHING) > + goto out_unlock; > + if (~categories & PAGE_IS_WRITTEN) > + goto out_unlock; > + > + /* > + * Break huge page into small pages if the WP operation > + * need to be performed is on a portion of the huge page. > + */ > + if (ret == -ENOSPC) { > + spin_unlock(ptl); > + split_huge_pmd(vma, pmd, start); > + pagemap_scan_backout_range(p, start, end); > + return -ENOENT; > + } > + > + make_uffd_wp_pmd(vma, start, pmd); > + flush_tlb_range(vma, start, end); > +out_unlock: > + spin_unlock(ptl); > + return ret; > +#else /* !CONFIG_TRANSPARENT_HUGEPAGE */ > + return -ENOENT; > +#endif > +} > + > +static int pagemap_scan_pmd_entry(pmd_t *pmd, unsigned long start, > + unsigned long end, struct mm_walk *walk) > +{ > + struct pagemap_scan_private *p = walk->private; > + struct vm_area_struct *vma = walk->vma; > + pte_t *pte, *start_pte; > + unsigned long addr; > + bool flush = false; > + spinlock_t *ptl; > + int ret; > + > + arch_enter_lazy_mmu_mode(); > + > + ret = pagemap_scan_thp_entry(pmd, start, end, walk); > + if (ret != -ENOENT) { > + arch_leave_lazy_mmu_mode(); > + return ret; > + } > + > + start_pte = pte = pte_offset_map_lock(vma->vm_mm, pmd, start, &ptl); > + if (!pte) { > + arch_leave_lazy_mmu_mode(); > + walk->action = ACTION_AGAIN; > + return 0; > + } > + > + for (addr = start; addr != end; pte++, addr += PAGE_SIZE) { > + unsigned long categories = p->cur_vma_category | > + pagemap_page_category(vma, addr, ptep_get(pte)); > + unsigned long next = addr + PAGE_SIZE; > + > + if (!pagemap_scan_is_interesting_page(categories, p)) > + continue; > + > + ret = pagemap_scan_output(categories, p, addr, &next); > + if (next == addr) > + break; > + > + if (~p->arg.flags & PM_SCAN_WP_MATCHING) > + continue; > + if (~categories & PAGE_IS_WRITTEN) > + continue; > + > + make_uffd_wp_pte(vma, addr, pte); > + if (!flush) { > + start = addr; > + flush = true; > + } > + } > + > + if (flush) > + flush_tlb_range(vma, start, addr); > + > + pte_unmap_unlock(start_pte, ptl); > + arch_leave_lazy_mmu_mode(); > + > + cond_resched(); > + return ret; > +} > + > +#ifdef CONFIG_HUGETLB_PAGE > +static int pagemap_scan_hugetlb_entry(pte_t *ptep, unsigned long hmask, > + unsigned long start, unsigned long end, > + struct mm_walk *walk) > +{ > + unsigned long n_pages = (end - start)/PAGE_SIZE; > + struct pagemap_scan_private *p = walk->private; > + struct vm_area_struct *vma = walk->vma; > + unsigned long categories; > + spinlock_t *ptl; > + int ret = 0; > + pte_t pte; > + > + if (~p->arg.flags & PM_SCAN_WP_MATCHING) { > + /* Go the short route when not write-protecting pages. */ > + > + pte = huge_ptep_get(ptep); > + categories = p->cur_vma_category | pagemap_hugetlb_category(pte); > + > + if (!pagemap_scan_is_interesting_page(categories, p)) > + return 0; > + > + return pagemap_scan_output(categories, p, start, &end); > + } > + > + if (n_pages < HPAGE_SIZE/PAGE_SIZE) { > + /* > + * Partial hugetlb page clear isn't supported > + */ > + p->end_addr = start; > + return -EINVAL; > + } > + > + i_mmap_lock_write(vma->vm_file->f_mapping); > + ptl = huge_pte_lock(hstate_vma(vma), vma->vm_mm, ptep); > + > + pte = huge_ptep_get(ptep); > + categories = p->cur_vma_category | pagemap_hugetlb_category(pte); > + > + if (!pagemap_scan_is_interesting_page(categories, p)) > + goto out_unlock; > + > + ret = pagemap_scan_output(categories, p, start, &end); > + if (start == end) > + goto out_unlock; > + > + if (categories & PAGE_IS_WRITTEN) { > + make_uffd_wp_huge_pte(vma, start, ptep, pte); > + flush_hugetlb_tlb_range(vma, start, end); > + } > + > +out_unlock: > + spin_unlock(ptl); > + i_mmap_unlock_write(vma->vm_file->f_mapping); > + > + return ret; > +} > +#else > +#define pagemap_scan_hugetlb_entry NULL > +#endif > + > +static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end, > + int depth, struct mm_walk *walk) > +{ > + struct pagemap_scan_private *p = walk->private; > + struct vm_area_struct *vma = walk->vma; > + int ret; > + > + if (!vma || !pagemap_scan_is_interesting_page(p->cur_vma_category, p)) > + return 0; > + > + ret = pagemap_scan_output(p->cur_vma_category, p, addr, &end); > + if (addr == end) > + return ret; > + > + if (~p->arg.flags & PM_SCAN_WP_MATCHING) > + return ret; > + > + int err = uffd_wp_range(vma, addr, end - addr, true); > + if (err < 0) > + ret = err; > + > + return ret; > +} > + > +static const struct mm_walk_ops pagemap_scan_ops = { > + .test_walk = pagemap_scan_test_walk, > + .pmd_entry = pagemap_scan_pmd_entry, > + .pte_hole = pagemap_scan_pte_hole, > + .hugetlb_entry = pagemap_scan_hugetlb_entry, > +}; > + > +static int pagemap_scan_get_args(struct pm_scan_arg *arg, > + unsigned long uarg) > +{ > + unsigned long start, end, vec; > + > + if (copy_from_user(arg, (void __user *)uarg, sizeof(*arg))) > + return -EFAULT; > + > + if (arg->size != sizeof(struct pm_scan_arg)) > + return -EINVAL; > + > + /* Validate requested features */ > + if (arg->flags & ~PM_SCAN_FLAGS) > + return -EINVAL; > + if ((arg->category_inverted | arg->category_mask | > + arg->category_anyof_mask | arg->return_mask) & ~PM_SCAN_CATEGORIES) > + return -EINVAL; > + > + start = untagged_addr((unsigned long)arg->start); > + end = untagged_addr((unsigned long)arg->end); > + vec = untagged_addr((unsigned long)arg->vec); > + > + /* Validate memory pointers */ > + if (!IS_ALIGNED(start, PAGE_SIZE)) > + return -EINVAL; > + if (!access_ok((void __user *)start, end - start)) > + return -EFAULT; > + if (arg->vec_len && !vec) > + return -EFAULT; > + if (!access_ok((void __user *)vec, > + arg->vec_len * sizeof(struct page_region))) > + return -EFAULT; > + > + /* Fixup default values */ > + if (!arg->max_pages) > + arg->max_pages = ULONG_MAX; > + > + return 0; > +} > + > +static int pagemap_scan_writeback_args(struct pm_scan_arg *arg, > + unsigned long uargl) > +{ > + struct pm_scan_arg __user *uarg = (void __user *)uargl; > + > + if (copy_to_user(&uarg->start, &arg->start, sizeof(arg->start))) > + return -EFAULT; > + > + return 0; > +} > + > +static int pagemap_scan_init_bounce_buffer(struct pagemap_scan_private *p) > +{ > + if (!p->arg.vec_len) { > + /* > + * An arbitrary non-page-aligned sentinel value for > + * pagemap_scan_push_range(). > + */ > + p->cur_buf.start = p->cur_buf.end = ULLONG_MAX; > + return 0; > + } > + > + /* > + * Allocate a smaller buffer to get output from inside the page > + * walk functions and walk the range in PAGEMAP_WALK_SIZE chunks. > + * The last range is always stored in p.cur_buf to allow coalescing > + * consecutive ranges that have the same categories returned across > + * walk_page_range() calls. > + */ > + p->vec_buf_len = min_t(size_t, PAGEMAP_WALK_SIZE >> PAGE_SHIFT, > + p->arg.vec_len - 1); > + p->vec_buf = kmalloc_array(p->vec_buf_len, sizeof(*p->vec_buf), > + GFP_KERNEL); > + if (!p->vec_buf) > + return -ENOMEM; > + > + p->vec_out = (void __user *)p->arg.vec; > + > + return 0; > +} > + > +static int pagemap_scan_flush_buffer(struct pagemap_scan_private *p) > +{ > + const struct page_region *buf = p->vec_buf; > + int n = (int)p->vec_buf_index; > + > + if (!n) > + return 0; > + > + if (copy_to_user(p->vec_out, buf, n * sizeof(*buf))) > + return -EFAULT; > + > + p->arg.vec_len -= n; > + p->vec_out += n; > + > + p->vec_buf_index = 0; > + p->vec_buf_len = min_t(size_t, p->vec_buf_len, p->arg.vec_len - 1); > + > + return n; > +} > + > +static long do_pagemap_scan(struct mm_struct *mm, unsigned long uarg) > +{ > + unsigned long walk_start, walk_end; > + struct mmu_notifier_range range; > + struct pagemap_scan_private p; > + size_t n_ranges_out = 0; > + int ret; > + > + memset(&p, 0, sizeof(p)); > + ret = pagemap_scan_get_args(&p.arg, uarg); > + if (ret) > + return ret; > + > + ret = pagemap_scan_init_bounce_buffer(&p); > + if (ret) > + return ret; > + > + /* Protection change for the range is going to happen. */ > + if (p.arg.flags & PM_SCAN_WP_MATCHING) { > + mmu_notifier_range_init(&range, MMU_NOTIFY_PROTECTION_VMA, 0, > + mm, p.arg.start, p.arg.end); > + mmu_notifier_invalidate_range_start(&range); > + } > + > + walk_start = walk_end = p.arg.start; > + for (; walk_end != p.arg.end; walk_start = walk_end) { > + int n_out; > + walk_end = min_t(unsigned long, > + (walk_start + PAGEMAP_WALK_SIZE) & PAGEMAP_WALK_MASK, > + p.arg.end); > + > + ret = mmap_read_lock_killable(mm); > + if (ret) > + break; > + ret = walk_page_range(mm, walk_start, walk_end, > + &pagemap_scan_ops, &p); > + mmap_read_unlock(mm); > + > + n_out = pagemap_scan_flush_buffer(&p); > + if (n_out < 0) > + ret = n_out; > + else > + n_ranges_out += n_out; > + > + if (ret) > + break; > + } > + > + if (p.cur_buf.start != p.cur_buf.end) { > + if (copy_to_user(p.vec_out, &p.cur_buf, sizeof(p.cur_buf))) > + ret = -EFAULT; > + else > + ++n_ranges_out; > + } > + > + /* ENOSPC signifies early stop (buffer full) from the walk. */ > + if (!ret || ret == -ENOSPC) > + ret = n_ranges_out; > + > + p.arg.start = p.end_addr ? p.end_addr : walk_start; > + if (pagemap_scan_writeback_args(&p.arg, uarg)) > + ret = -EFAULT; > + > + if (p.arg.flags & PM_SCAN_WP_MATCHING) > + mmu_notifier_invalidate_range_end(&range); > + > + kfree(p.vec_buf); > + return ret; > +} > + > +static long do_pagemap_cmd(struct file *file, unsigned int cmd, > + unsigned long arg) > +{ > + struct mm_struct *mm = file->private_data; > + > + switch (cmd) { > + case PAGEMAP_SCAN: > + return do_pagemap_scan(mm, arg); > + > + default: > + return -EINVAL; > + } > +} > + > const struct file_operations proc_pagemap_operations = { > .llseek = mem_lseek, /* borrow this */ > .read = pagemap_read, > .open = pagemap_open, > .release = pagemap_release, > + .unlocked_ioctl = do_pagemap_cmd, > + .compat_ioctl = do_pagemap_cmd, > }; > #endif /* CONFIG_PROC_PAGE_MONITOR */ > > diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h > index 9f4bac3df59e..4f5fed605538 100644 > --- a/include/linux/hugetlb.h > +++ b/include/linux/hugetlb.h > @@ -259,6 +259,7 @@ long hugetlb_change_protection(struct vm_area_struct *vma, > unsigned long cp_flags); > > bool is_hugetlb_entry_migration(pte_t pte); > +bool is_hugetlb_entry_hwpoisoned(pte_t pte); > void hugetlb_unshare_all_pmds(struct vm_area_struct *vma); > > #else /* !CONFIG_HUGETLB_PAGE */ > diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h > index b7b56871029c..db39442befcb 100644 > --- a/include/uapi/linux/fs.h > +++ b/include/uapi/linux/fs.h > @@ -305,4 +305,60 @@ typedef int __bitwise __kernel_rwf_t; > #define RWF_SUPPORTED (RWF_HIPRI | RWF_DSYNC | RWF_SYNC | RWF_NOWAIT |\ > RWF_APPEND) > > +/* Pagemap ioctl */ > +#define PAGEMAP_SCAN _IOWR('f', 16, struct pm_scan_arg) > + > +/* Bits are set in flags of the page_region and masks in pm_scan_args */ > +#define PAGE_IS_WPASYNC (1 << 0) > +#define PAGE_IS_WRITTEN (1 << 1) > +#define PAGE_IS_FILE (1 << 2) > +#define PAGE_IS_PRESENT (1 << 3) > +#define PAGE_IS_SWAPPED (1 << 4) > +#define PAGE_IS_PFNZERO (1 << 5) > + > +/* > + * struct page_region - Page region with flags > + * @start: Start of the region > + * @end: End of the region (exclusive) > + * @categories: PAGE_IS_* category bitmask for the region > + */ > +struct page_region { > + __u64 start; > + __u64 end; > + __u64 categories; > +}; > + > +/* Flags for PAGEMAP_SCAN ioctl */ > +#define PM_SCAN_WP_MATCHING (1 << 0) /* Write protect the pages matched. */ > +#define PM_SCAN_CHECK_WPASYNC (1 << 1) /* Abort the scan when a non-WP-enabled page is found. */ > + > +/* > + * struct pm_scan_arg - Pagemap ioctl argument > + * @size: Size of the structure > + * @flags: Flags for the IOCTL > + * @start: Starting address of the region > + * (Ending address of the walk is also returned in it) > + * @end: Ending address of the region > + * @vec: Address of page_region struct array for output > + * @vec_len: Length of the page_region struct array > + * @max_pages: Optional limit for number of returned pages (0 = disabled) > + * @category_inverted: PAGE_IS_* categories which values match if 0 instead of 1 > + * @category_mask: Skip pages for which any category doesn't match > + * @category_anyof_mask: Skip pages for which no category matches > + * @return_mask: PAGE_IS_* categories that are to be reported in `page_region`s returned > + */ > +struct pm_scan_arg { > + __u64 size; > + __u64 flags; > + __u64 start; > + __u64 end; > + __u64 vec; > + __u64 vec_len; > + __u64 max_pages; > + __u64 category_inverted; > + __u64 category_mask; > + __u64 category_anyof_mask; > + __u64 return_mask; > +}; > + > #endif /* _UAPI_LINUX_FS_H */ > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > index 751968eff92d..4bf8cc4aba2e 100644 > --- a/mm/hugetlb.c > +++ b/mm/hugetlb.c > @@ -5008,7 +5008,7 @@ bool is_hugetlb_entry_migration(pte_t pte) > return false; > } > > -static bool is_hugetlb_entry_hwpoisoned(pte_t pte) > +bool is_hugetlb_entry_hwpoisoned(pte_t pte) > { > swp_entry_t swp; > -- BR, Muhammad Usama Anjum

2 years, 1 month

5
25
0 0

[PATCH v3] kselftest/arm64: Exit streaming mode after collecting signal context

by Mark Brown

When we collect a signal context with one of the SME modes enabled we will have enabled that mode behind the compiler and libc's back so they may issue some instructions not valid in streaming mode, causing spurious failures. For the code prior to issuing the BRK to trigger signal handling we need to stay in streaming mode if we were already there since that's a part of the signal context the caller is trying to collect. Unfortunately this code includes a memset() which is likely to be heavily optimised and is likely to use FP instructions incompatible with streaming mode. We can avoid this happening by open coding the memset(), inserting a volatile assembly statement to avoid the compiler recognising what's being done and doing something in optimisation. This code is not performance critical so the inefficiency should not be an issue. After collecting the context we can simply exit streaming mode, avoiding these issues. Use a full SMSTOP for safety to prevent any issues appearing with ZA. Reported-by: Will Deacon <will(a)kernel.org> Signed-off-by: Mark Brown <broonie(a)kernel.org> --- Changes in v3: - Open code OPTIMISER_HIDE_VAR() instead of the memory clobber. - Link to v2: https://lore.kernel.org/r/20230712-arm64-signal-memcpy-fix-v2-1-494f7025caf… Changes in v2: - Rebase onto v6.5-rc1. - Link to v1: https://lore.kernel.org/r/20230628-arm64-signal-memcpy-fix-v1-1-db3e0300829… --- .../selftests/arm64/signal/test_signals_utils.h | 25 +++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/arm64/signal/test_signals_utils.h b/tools/testing/selftests/arm64/signal/test_signals_utils.h index 222093f51b67..c7f5627171dd 100644 --- a/tools/testing/selftests/arm64/signal/test_signals_utils.h +++ b/tools/testing/selftests/arm64/signal/test_signals_utils.h @@ -60,13 +60,25 @@ static __always_inline bool get_current_context(struct tdescr *td, size_t dest_sz) { static volatile bool seen_already; + int i; + char *uc = (char *)dest_uc; assert(td && dest_uc); /* it's a genuine invocation..reinit */ seen_already = 0; td->live_uc_valid = 0; td->live_sz = dest_sz; - memset(dest_uc, 0x00, td->live_sz); + + /* + * This is a memset() but we don't want the compiler to + * optimise it into either instructions or a library call + * which might be incompatible with streaming mode. + */ + for (i = 0; i < td->live_sz; i++) { + uc[i] = 0; + __asm__ ("" : "=r" (uc[i]) : "0" (uc[i])); + } + td->live_uc = dest_uc; /* * Grab ucontext_t triggering a SIGTRAP. @@ -103,6 +115,17 @@ static __always_inline bool get_current_context(struct tdescr *td, : : "memory"); + /* + * If we were grabbing a streaming mode context then we may + * have entered streaming mode behind the system's back and + * libc or compiler generated code might decide to do + * something invalid in streaming mode, or potentially even + * the state of ZA. Issue a SMSTOP to exit both now we have + * grabbed the state. + */ + if (td->feats_supported & FEAT_SME) + asm volatile("msr S0_3_C4_C6_3, xzr"); + /* * If we get here with seen_already==1 it implies the td->live_uc * context has been used to get back here....this probably means --- base-commit: 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5 change-id: 20230628-arm64-signal-memcpy-fix-7de3b3c8fa10 Best regards, -- Mark Brown <broonie(a)kernel.org>

2 years, 1 month

2
3
0 0

[PATCH v2 0/3] arm64/fpsimd: Fix use after free in SME when changing SVE VL

by Mark Brown

This series fixes an issue which David Spickett found where if we change the SVE VL while SME is in use we can end up attempting to save state to an unallocated buffer and adds testing coverage for that plus a bit more coverage of VL changes, just for paranioa. Signed-off-by: Mark Brown <broonie(a)kernel.org> --- Changes in v2: - Always reallocate the SVE state. - Rebase onto v6.5-rc2. - Link to v1: https://lore.kernel.org/r/20230713-arm64-fix-sve-sme-vl-change-v1-0-129dd86… --- Mark Brown (3): arm64/fpsimd: Ensure SME storage is allocated after SVE VL changes kselftest/arm64: Add a test case for SVE VL changes with SME active kselftest/arm64: Validate that changing one VL type does not affect another arch/arm64/kernel/fpsimd.c | 33 +++++-- tools/testing/selftests/arm64/fp/vec-syscfg.c | 127 +++++++++++++++++++++++++- 2 files changed, 148 insertions(+), 12 deletions(-) --- base-commit: 06785562d1b99ff6dc1cd0af54be5e3ff999dc02 change-id: 20230713-arm64-fix-sve-sme-vl-change-60eb1fa6a707 Best regards, -- Mark Brown <broonie(a)kernel.org>

2 years, 1 month

2
5
0 0

[PATCH net-next 0/4] selftests: openvswitch: add flow programming cases

by Aaron Conole

The openvswitch selftests currently contain a few cases for managing the datapath, which includes creating datapath instances, adding interfaces, and doing some basic feature / upcall tests. This is useful to validate the control path. Add the ability to program some of the more common flows with actions. This can be improved overtime to include regression testing, etc. Aaron Conole (4): selftests: openvswitch: add an initial flow programming case selftests: openvswitch: add a test for ipv4 forwarding selftests: openvswitch: add basic ct test case parsing selftests: openvswitch: add ct-nat test case with ipv4 .../selftests/net/openvswitch/openvswitch.sh | 223 ++++++++ .../selftests/net/openvswitch/ovs-dpctl.py | 507 ++++++++++++++++++ 2 files changed, 730 insertions(+) -- 2.40.1

2 years, 1 month

3
13
0 0

[RFC PATCH 00/10] Device Memory TCP

by Mina Almasry

* TL;DR: Device memory TCP (devmem TCP) is a proposal for transferring data to and/or from device memory efficiently, without bouncing the data to a host memory buffer. * Problem: A large amount of data transfers have device memory as the source and/or destination. Accelerators drastically increased the volume of such transfers. Some examples include: - ML accelerators transferring large amounts of training data from storage into GPU/TPU memory. In some cases ML training setup time can be as long as 50% of TPU compute time, improving data transfer throughput & efficiency can help improving GPU/TPU utilization. - Distributed training, where ML accelerators, such as GPUs on different hosts, exchange data among them. - Distributed raw block storage applications transfer large amounts of data with remote SSDs, much of this data does not require host processing. Today, the majority of the Device-to-Device data transfers the network are implemented as the following low level operations: Device-to-Host copy, Host-to-Host network transfer, and Host-to-Device copy. The implementation is suboptimal, especially for bulk data transfers, and can put significant strains on system resources, such as host memory bandwidth, PCIe bandwidth, etc. One important reason behind the current state is the kernel’s lack of semantics to express device to network transfers. * Proposal: In this patch series we attempt to optimize this use case by implementing socket APIs that enable the user to: 1. send device memory across the network directly, and 2. receive incoming network packets directly into device memory. Packet _payloads_ go directly from the NIC to device memory for receive and from device memory to NIC for transmit. Packet _headers_ go to/from host memory and are processed by the TCP/IP stack normally. The NIC _must_ support header split to achieve this. Advantages: - Alleviate host memory bandwidth pressure, compared to existing network-transfer + device-copy semantics. - Alleviate PCIe BW pressure, by limiting data transfer to the lowest level of the PCIe tree, compared to traditional path which sends data through the root complex. With this proposal we're able to reach ~96.6% line rate speeds with data sent and received directly from/to device memory. * Patch overview: ** Part 1: struct paged device memory Currently the standard for device memory sharing is DMABUF, which doesn't generate struct pages. On the other hand, networking stack (skbs, drivers, and page pool) operate on pages. We have 2 options: 1. Generate struct pages for dmabuf device memory, or, 2. Modify the networking stack to understand a new memory type. This proposal implements option #1. We implement a small framework to generate struct pages for an sg_table returned from dma_buf_map_attachment(). The support added here should be generic and easily extended to other use cases interested in struct paged device memory. We use this framework to generate pages that can be used in the networking stack. ** Part 2: recvmsg() & sendmsg() APIs We define user APIs for the user to send and receive these dmabuf pages. ** part 3: support for unreadable skb frags Dmabuf pages are not accessible by the host; we implement changes throughput the networking stack to correctly handle skbs with unreadable frags. ** part 4: page pool support We piggy back on Jakub's page pool memory providers idea: https://github.com/kuba-moo/linux/tree/pp-providers It allows the page pool to define a memory provider that provides the page allocation and freeing. It helps abstract most of the device memory TCP changes from the driver. This is not strictly necessary, the driver can choose to allocate dmabuf pages and use them directly without going through the page pool (if acceptable to their maintainers). Not included with this RFC is the GVE devmem TCP support, just to simplify the review. Code available here if desired: https://github.com/mina/linux/tree/tcpdevmem This RFC is built on top of v6.4-rc7 with Jakub's pp-providers changes cherry-picked. * NIC dependencies: 1. (strict) Devmem TCP require the NIC to support header split, i.e. the capability to split incoming packets into a header + payload and to put each into a separate buffer. Devmem TCP works by using dmabuf pages for the packet payload, and host memory for the packet headers. 2. (optional) Devmem TCP works better with flow steering support & RSS support, i.e. the NIC's ability to steer flows into certain rx queues. This allows the sysadmin to enable devmem TCP on a subset of the rx queues, and steer devmem TCP traffic onto these queues and non devmem TCP elsewhere. The NIC I have access to with these properties is the GVE with DQO support running in Google Cloud, but any NIC that supports these features would suffice. I may be able to help reviewers bring up devmem TCP on their NICs. * Testing: The series includes a udmabuf kselftest that show a simple use case of devmem TCP and validates the entire data path end to end without a dependency on a specific dmabuf provider. Not included in this series is our devmem TCP benchmark, which transfers data to/from GPU dmabufs directly. With this implementation & benchmark we're able to reach ~96.6% line rate speeds with 4 GPU/NIC pairs running bi-direction traffic, with all the packet payloads going straight to the GPU memory (no host buffer bounce). ** Test Setup Kernel: v6.4-rc7, with this RFC and Jakub's memory provider API cherry-picked locally. Hardware: Google Cloud A3 VMs. NIC: GVE with header split & RSS & flow steering support. Benchmark: custom devmem TCP benchmark not yet open sourced. Mina Almasry (10): dma-buf: add support for paged attachment mappings dma-buf: add support for NET_RX pages dma-buf: add support for NET_TX pages net: add support for skbs with unreadable frags tcp: implement recvmsg() RX path for devmem TCP net: add SO_DEVMEM_DONTNEED setsockopt to release RX pages tcp: implement sendmsg() TX path for for devmem tcp selftests: add ncdevmem, netcat for devmem TCP memory-provider: updates core provider API for devmem TCP memory-provider: add dmabuf devmem provider drivers/dma-buf/dma-buf.c | 444 ++++++++++++++++ include/linux/dma-buf.h | 142 +++++ include/linux/netdevice.h | 1 + include/linux/skbuff.h | 34 +- include/linux/socket.h | 1 + include/net/page_pool.h | 21 + include/net/sock.h | 4 + include/net/tcp.h | 6 +- include/uapi/asm-generic/socket.h | 6 + include/uapi/linux/dma-buf.h | 12 + include/uapi/linux/uio.h | 10 + net/core/datagram.c | 3 + net/core/page_pool.c | 111 +++- net/core/skbuff.c | 81 ++- net/core/sock.c | 47 ++ net/ipv4/tcp.c | 262 +++++++++- net/ipv4/tcp_input.c | 13 +- net/ipv4/tcp_ipv4.c | 8 + net/ipv4/tcp_output.c | 5 +- net/packet/af_packet.c | 4 +- tools/testing/selftests/net/.gitignore | 1 + tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/ncdevmem.c | 693 +++++++++++++++++++++++++ 23 files changed, 1868 insertions(+), 42 deletions(-) create mode 100644 tools/testing/selftests/net/ncdevmem.c -- 2.41.0.390.g38632f3daf-goog

2 years, 1 month

8
29
0 0

[PATCH v5 00/10] tracing: introducing eventfs

by Ajay Kaher

Events Tracing infrastructure contains lot of files, directories (internally in terms of inodes, dentries). And ends up by consuming memory in MBs. We can have multiple events of Events Tracing, which further requires more memory. Instead of creating inodes/dentries, eventfs could keep meta-data and skip the creation of inodes/dentries. As and when require, eventfs will create the inodes/dentries only for required files/directories. Also eventfs would delete the inodes/dentries once no more requires but preserve the meta data. Tracing events took ~9MB, with this approach it took ~4.5MB for ~10K files/dir. Diff from v4: Patch 02: moved from v4 08/10 added fs/tracefs/internal.h Patch 03: moved from v4 02/10 removed fs/tracefs/internal.h Patch 04: moved from v4 03/10 moved out changes of fs/tracefs/internal.h Patch 05: moved from v4 04/10 renamed eventfs_add_top_file() -> eventfs_add_events_file() Patch 06: moved from v4 07/10 implemented create_dentry() helper function added create_file(), create_dir() stub function Patch 07: moved from v4 06/10 Patch 08: moved from v4 05/10 improved eventfs remove functionality Patch 09: removed unwanted if conditions Patch 10: added available_filter_functions check Diff from v3: Patch 3,4,5,7,9: removed all the eventfs_rwsem code and replaced it with an srcu lock for the readers, and a mutex to synchronize the writers of the list. Patch 2: moved 'tracefs_inode' and 'get_tracefs()' to v4 03/10 Patch 3: moved the struct eventfs_file and eventfs_inode into event_inode.c as it really should not be exposed to all users. Patch 5: added a recursion check to eventfs_remove_rec() as it is really dangerous to have unchecked recursion in the kernel (we do have a fixed size stack). have the free use srcu callbacks. After the srcu grace periods are done, it adds the eventfs_file onto a llist (lockless link list) and wakes up a work queue. Then the work queue does the freeing (this needs to be done in task/workqueue context, as srcu callbacks are done in softirq context). Patch 6: renamed: eventfs_create_file() -> create_file() eventfs_create_dir() -> create_dir() Diff from v2: Patch 01: new patch:'Require all trace events to have a TRACE_SYSTEM' Patch 02: moved from v1 1/9 Patch 03: moved from v1 2/9 As suggested by Zheng Yejian, introduced eventfs_prepare_ef() helper function to add files or directories to eventfs fix WARNING reported by kernel test robot in v1 8/9 Patch 04: moved from v1 3/9 used eventfs_prepare_ef() to add files fix WARNING reported by kernel test robot in v1 8/9 Patch 05: moved from v1 4/9 fix compiling warning reported by kernel test robot in v1 4/9 Patch 06: moved from v1 5/9 Patch 07: moved from v1 6/9 Patch 08: moved from v1 7/9 Patch 09: moved from v1 8/9 rebased because of v3 01/10 Patch 10: moved from v1 9/9 Diff from v1: Patch 1: add header file Patch 2: resolved kernel test robot issues protecting eventfs lists using nested eventfs_rwsem Patch 3: protecting eventfs lists using nested eventfs_rwsem Patch 4: improve events cleanup code to fix crashes Patch 5: resolved kernel test robot issues removed d_instantiate_anon() calls Patch 6: resolved kernel test robot issues fix kprobe test in eventfs_root_lookup() protecting eventfs lists using nested eventfs_rwsem Patch 7: remove header file Patch 8: pass eventfs_rwsem as argument to eventfs functions called eventfs_remove_events_dir() instead of tracefs_remove() from event_trace_del_tracer() Patch 9: new patch to fix kprobe test case fs/tracefs/Makefile | 1 + fs/tracefs/event_inode.c | 795 ++++++++++++++++++ fs/tracefs/inode.c | 151 +++- fs/tracefs/internal.h | 26 + include/linux/trace_events.h | 1 + include/linux/tracefs.h | 30 + kernel/trace/trace.h | 2 +- kernel/trace/trace_events.c | 76 +- .../ftrace/test.d/kprobe/kprobe_args_char.tc | 9 +- .../test.d/kprobe/kprobe_args_string.tc | 9 +- 10 files changed, 1048 insertions(+), 52 deletions(-) create mode 100644 fs/tracefs/event_inode.c create mode 100644 fs/tracefs/internal.h -- 2.39.0

2 years, 1 month

2
16
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror