September 2023 - Linux-stable-mirror

[PATCH 6.1] block: fix use-after-free of q->q_usage_counter

by Saranya Muruganandam

From: Ming Lei <ming.lei(a)redhat.com> commit d36a9ea5e7766961e753ee38d4c331bbe6ef659b upstream. For blk-mq, queue release handler is usually called after blk_mq_freeze_queue_wait() returns. However, the q_usage_counter->release() handler may not be run yet at that time, so this can cause a use-after-free. Fix the issue by moving percpu_ref_exit() into blk_free_queue_rcu(). Since ->release() is called with rcu read lock held, it is agreed that the race should be covered in caller per discussion from the two links. Backport-notes: Not a clean cherry-pick since a lot has changed, however essentially the same fix. Reported-by: Zhang Wensheng <zhangwensheng(a)huaweicloud.com> Reported-by: Zhong Jinghua <zhongjinghua(a)huawei.com> Link: https://lore.kernel.org/linux-block/Y5prfOjyyjQKUrtH@T590/T/#u Link: https://lore.kernel.org/lkml/Y4%2FmzMd4evRg9yDi@fedora/ Cc: Hillf Danton <hdanton(a)sina.com> Cc: Yu Kuai <yukuai3(a)huawei.com> Cc: Dennis Zhou <dennis(a)kernel.org> Fixes: 2b0d3d3e4fcf ("percpu_ref: reduce memory footprint of percpu_ref in fast path") Signed-off-by: Ming Lei <ming.lei(a)redhat.com> Link: https://lore.kernel.org/r/20221215021629.74870-1-ming.lei@redhat.com Signed-off-by: Jens Axboe <axboe(a)kernel.dk> Signed-off-by: Saranya Muruganandam <saranyamohan(a)google.com> --- block/blk-sysfs.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index a582ea0da74f..a82bdec923b2 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -737,6 +737,7 @@ static void blk_free_queue_rcu(struct rcu_head *rcu_head) struct request_queue *q = container_of(rcu_head, struct request_queue, rcu_head); + percpu_ref_exit(&q->q_usage_counter); kmem_cache_free(blk_get_queue_kmem_cache(blk_queue_has_srcu(q)), q); } @@ -762,8 +763,6 @@ static void blk_release_queue(struct kobject *kobj) might_sleep(); - percpu_ref_exit(&q->q_usage_counter); - if (q->poll_stat) blk_stat_remove_callback(q, q->poll_cb); blk_stat_free_callback(q->poll_cb); -- 2.42.0.515.g380fc7ccd1-goog

1 year, 7 months

1
0
0 0

[PATCH -stable,5.10 00/17] Netfilter stable fixes for 5.10

by Pablo Neira Ayuso

Hi Greg, Sasha, The following list shows the backported patches, this batch is targeting at garbage collection (GC) / set timeout fixes that address possible UaF and memleaks. I am using original commit IDs for reference: 1) 212ed75dc5fb ("netfilter: nf_tables: integrate pipapo into commit protocol") 2) 24138933b97b ("netfilter: nf_tables: don't skip expired elements during walk") 3) 5f68718b34a5 ("netfilter: nf_tables: GC transaction API to avoid race with control plane") 4) f6c383b8c31a ("netfilter: nf_tables: adapt set backend to use GC transaction API") 5) c92db3030492 ("netfilter: nft_set_hash: mark set element as dead when deleting from packet path") 6) a2dd0233cbc4 ("netfilter: nf_tables: remove busy mark and gc batch API") 7) 7845914f45f0 ("netfilter: nf_tables: don't fail inserts if duplicate has expired") 8) 6a33d8b73dfa ("netfilter: nf_tables: fix GC transaction races with netns and netlink event exit path") 9) 02c6c24402bf ("netfilter: nf_tables: GC transaction race with netns dismantle") 10) 720344340fb9 ("netfilter: nf_tables: GC transaction race with abort path") 11) 8357bc946a2a ("netfilter: nf_tables: use correct lock to protect gc_list") 12) 8e51830e29e1 ("netfilter: nf_tables: defer gc run if previous batch is still pending") 13) 2ee52ae94baa ("netfilter: nft_set_rbtree: skip sync GC for new elements in this transaction") 14) 96b33300fba8 ("netfilter: nft_set_rbtree: use read spinlock to avoid datapath contention") 15) 6d365eabce3c ("netfilter: nft_set_pipapo: stop GC iteration if GC transaction allocation fails") 16) b079155faae9 ("netfilter: nft_set_hash: try later when GC hits EAGAIN on iteration") 17) cf5000a7787c ("netfilter: nf_tables: fix memleak when more than 255 elements expired") Please, apply. Thanks. Florian Westphal (4): netfilter: nf_tables: don't skip expired elements during walk netfilter: nf_tables: don't fail inserts if duplicate has expired netfilter: nf_tables: defer gc run if previous batch is still pending netfilter: nf_tables: fix memleak when more than 255 elements expired Pablo Neira Ayuso (13): netfilter: nf_tables: integrate pipapo into commit protocol netfilter: nf_tables: GC transaction API to avoid race with control plane netfilter: nf_tables: adapt set backend to use GC transaction API netfilter: nft_set_hash: mark set element as dead when deleting from packet path netfilter: nf_tables: remove busy mark and gc batch API netfilter: nf_tables: fix GC transaction races with netns and netlink event exit path netfilter: nf_tables: GC transaction race with netns dismantle netfilter: nf_tables: GC transaction race with abort path netfilter: nf_tables: use correct lock to protect gc_list netfilter: nft_set_rbtree: skip sync GC for new elements in this transaction netfilter: nft_set_rbtree: use read spinlock to avoid datapath contention netfilter: nft_set_pipapo: stop GC iteration if GC transaction allocation fails netfilter: nft_set_hash: try later when GC hits EAGAIN on iteration include/net/netfilter/nf_tables.h | 125 +++++------ net/netfilter/nf_tables_api.c | 341 +++++++++++++++++++++++++++--- net/netfilter/nft_set_hash.c | 87 +++++--- net/netfilter/nft_set_pipapo.c | 115 ++++++---- net/netfilter/nft_set_rbtree.c | 157 ++++++++------ 5 files changed, 589 insertions(+), 236 deletions(-) -- 2.30.2

1 year, 7 months

1
17
0 0

[PATCH 6.1 000/139] 6.1.55-rc1 review

by Greg Kroah-Hartman

This is the start of the stable review cycle for the 6.1.55 release. There are 139 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know. Responses should be made by Fri, 22 Sep 2023 11:28:09 +0000. Anything received after that time might be too late. The whole patch series can be found in one patch at: https://www.kernel.org/pub/linux/kernel/v6.x/stable-review/patch-6.1.55-rc1… or in the git tree and branch at: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-6.1.y and the diffstat can be found below. thanks, greg k-h ------------- Pseudo-Shortlog of commits: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Linux 6.1.55-rc1 Jamal Hadi Salim <jhs(a)mojatatu.com> net/sched: Retire rsvp classifier Christian König <christian.koenig(a)amd.com> drm/amdgpu: fix amdgpu_cs_p1_user_fence Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Revert "memcg: drop kmem.limit_in_bytes" Yifan Zhang <yifan1.zhang(a)amd.com> drm/amd/display: fix the white screen issue when >= 64GB DRAM Shida Zhang <zhangshida(a)kylinos.cn> ext4: fix rec_len verify error Damien Le Moal <dlemoal(a)kernel.org> scsi: pm8001: Setup IRQs on resume Junxiao Bi <junxiao.bi(a)oracle.com> scsi: megaraid_sas: Fix deadlock on firmware crashdump Szuying Chen <chensiying21(a)gmail.com> ata: libahci: clear pending interrupt status Niklas Cassel <niklas.cassel(a)wdc.com> ata: libata: disallow dev-initiated LPM transitions to unsupported states Tommy Huang <tommy_huang(a)aspeedtech.com> i2c: aspeed: Reset the i2c controller when timeout occurs Steven Rostedt (Google) <rostedt(a)goodmis.org> tracefs: Add missing lockdown check to tracefs_create_dir() Jeff Layton <jlayton(a)kernel.org> nfsd: fix change_info in NFSv4 RENAME replies Ondrej Mosnacek <omosnace(a)redhat.com> selinux: fix handling of empty opts in selinux_fs_context_submount() Steven Rostedt (Google) <rostedt(a)goodmis.org> tracing: Have option files inc the trace array ref count Steven Rostedt (Google) <rostedt(a)goodmis.org> tracing: Have current_trace inc the trace array ref count Steven Rostedt (Google) <rostedt(a)goodmis.org> tracing: Increase trace array ref count on enable and filter files Steven Rostedt (Google) <rostedt(a)goodmis.org> tracing: Have event inject files inc the trace array ref count Steven Rostedt (Google) <rostedt(a)goodmis.org> tracing: Have tracing_max_latency inc the trace array ref count Josef Bacik <josef(a)toxicpanda.com> btrfs: check for BTRFS_FS_ERROR in pending ordered assert Filipe Manana <fdmanana(a)suse.com> btrfs: release path before inode lookup during the ino lookup ioctl Qu Wenruo <wqu(a)suse.com> btrfs: fix a compilation error if DEBUG is defined in btree_dirty_folio Filipe Manana <fdmanana(a)suse.com> btrfs: fix lockdep splat and potential deadlock after failure running delayed items Jens Axboe <axboe(a)kernel.dk> dm: don't attempt to queue IO under RCU protection Hamza Mahfooz <hamza.mahfooz(a)amd.com> Revert "drm/amd: Disable S/G for APUs when 64GB or more host memory" Mariusz Tkaczyk <mariusz.tkaczyk(a)linux.intel.com> md: Put the right device in md_seq_next Keith Busch <kbusch(a)kernel.org> nvme: avoid bogus CRTO values Pavel Begunkov <asml.silence(a)gmail.com> io_uring/net: fix iter retargeting for selected buf Amir Goldstein <amir73il(a)gmail.com> ovl: fix incorrect fdput() on aio completion Amir Goldstein <amir73il(a)gmail.com> ovl: fix failed copyup of fileattr on a symlink Christian Brauner <brauner(a)kernel.org> attr: block mode changes of symlinks Trond Myklebust <trond.myklebust(a)hammerspace.com> Revert "SUNRPC: Fail faster on bad verifier" Nigel Croxon <ncroxon(a)redhat.com> md/raid1: fix error: ISO C90 forbids mixed declarations Arnd Bergmann <arnd(a)arndb.de> samples/hw_breakpoint: fix building without module unloading Song Liu <song(a)kernel.org> x86/purgatory: Remove LTO flags Kirill A. Shutemov <kirill.shutemov(a)linux.intel.com> x86/boot/compressed: Reserve more memory for page tables Lukas Wunner <lukas(a)wunner.de> panic: Reenable preemption in WARN slowpath Jinjie Ruan <ruanjinjie(a)huawei.com> scsi: lpfc: Fix the NULL vs IS_ERR() bug for debugfs_create_file() David Disseldorp <ddiss(a)suse.de> scsi: target: core: Fix target_cmd_counter leak Song Shuai <songshuaishuai(a)tinylab.org> riscv: kexec: Align the kexeced kernel entry Peter Zijlstra <peterz(a)infradead.org> x86/ibt: Suppress spurious ENDBR Masami Hiramatsu (Google) <mhiramat(a)kernel.org> selftests: tracing: Fix to unmount tracefs for recovering environment Jinjie Ruan <ruanjinjie(a)huawei.com> scsi: qla2xxx: Fix NULL vs IS_ERR() bug for debugfs_create_dir() Jinjie Ruan <ruanjinjie(a)huawei.com> drm: gm12u320: Fix the timeout usage for usb_bulk_msg() Varun Prakash <varun(a)chelsio.com> nvmet-tcp: pass iov_len instead of sg->length to bvec_set_page() Christoph Hellwig <hch(a)lst.de> nvmet: use bvec_set_page to initialize bvecs Christoph Hellwig <hch(a)lst.de> block: factor out a bvec_set_page helper Anand Jain <anand.jain(a)oracle.com> btrfs: compare the correct fsid/metadata_uuid in btrfs_validate_super Anand Jain <anand.jain(a)oracle.com> btrfs: add a helper to read the superblock metadata_uuid Tiezhu Yang <yangtiezhu(a)loongson.cn> MIPS: Use "grep -E" instead of "egrep" Ekansh Gupta <quic_ekangupt(a)quicinc.com> misc: fastrpc: Fix incorrect DMA mapping unmap request Dmitry Osipenko <dmitry.osipenko(a)collabora.com> misc: fastrpc: Prepare to dynamic dma-buf locking specification Dmitry Osipenko <dmitry.osipenko(a)collabora.com> dma-buf: Add unlocked variant of attachment-mapping functions John Ogness <john.ogness(a)linutronix.de> printk: Consolidate console deferred printing John Ogness <john.ogness(a)linutronix.de> printk: Keep non-panic-CPUs out of console lock Rob Clark <robdclark(a)chromium.org> interconnect: Fix locking for runpm vs reclaim Zhen Lei <thunder.leizhen(a)huawei.com> kobject: Add sanity check for kset->kobj.ktype in kset_register() Sakari Ailus <sakari.ailus(a)linux.intel.com> media: pci: ipu3-cio2: Initialise timing struct to avoid a compiler warning Xu Yang <xu.yang_2(a)nxp.com> usb: chipidea: add workaround for chipidea PEC bug Xu Yang <xu.yang_2(a)nxp.com> usb: ehci: add workaround for chipidea PORTSC.PEC bug Baoquan He <bhe(a)redhat.com> misc: open-dice: make OPEN_DICE depend on HAS_IOMEM Christophe Leroy <christophe.leroy(a)csgroup.eu> serial: cpm_uart: Avoid suspicious locking Konstantin Shelekhin <k.shelekhin(a)yadro.com> scsi: target: iscsi: Fix buffer overflow in lio_target_nacl_info_show() Chenyuan Mi <michenyuan(a)huawei.com> tools: iio: iio_generic_buffer: Fix some integer type and calculation Ma Ke <make_ruc2021(a)163.com> usb: gadget: fsl_qe_udc: validate endpoint index for ch9 udc Xiaolei Wang <xiaolei.wang(a)windriver.com> usb: cdns3: Put the cdns set active part outside the spin lock Hans Verkuil <hverkuil-cisco(a)xs4all.nl> media: pci: cx23885: replace BUG with error return Hans Verkuil <hverkuil-cisco(a)xs4all.nl> media: tuners: qt1010: replace BUG_ON with a regular error Justin Tee <justin.tee(a)broadcom.com> scsi: lpfc: Abort outstanding ELS cmds when mailbox timeout error is detected Zhang Shurong <zhang_shurong(a)foxmail.com> media: dvb-usb-v2: gl861: Fix null-ptr-deref in gl861_i2c_master_xfer Zhang Shurong <zhang_shurong(a)foxmail.com> media: az6007: Fix null-ptr-deref in az6007_i2c_xfer() Zhang Shurong <zhang_shurong(a)foxmail.com> media: anysee: fix null-ptr-deref in anysee_master_xfer Zhang Shurong <zhang_shurong(a)foxmail.com> media: af9005: Fix null-ptr-deref in af9005_i2c_xfer Zhang Shurong <zhang_shurong(a)foxmail.com> media: dw2102: Fix null-ptr-deref in dw2102_i2c_transfer() Zhang Shurong <zhang_shurong(a)foxmail.com> media: dvb-usb-v2: af9035: Fix null-ptr-deref in af9035_i2c_master_xfer Lu Hongfei <luhongfei(a)vivo.com> media: mdp3: Fix resource leaks in of_find_device_by_node Yong-Xuan Wang <yongxuan.wang(a)sifive.com> PCI: fu740: Set the number of MSI vectors Nirmal Patel <nirmal.patel(a)linux.intel.com> PCI: vmd: Disable bridge window for domain reset ruanjinjie <ruanjinjie(a)huawei.com> powerpc/pseries: fix possible memory leak in ibmebus_bus_init() Mårten Lindahl <marten.lindahl(a)axis.com> ARM: 9317/1: kexec: Make smp stop calls asynchronous Mark Brown <broonie(a)kernel.org> PCI: dwc: Provide deinit callback for i.MX Liu Shixin via Jfs-discussion <jfs-discussion(a)lists.sourceforge.net> jfs: fix invalid free of JFS_IP(ipimap)->i_imap in diUnmount Andrew Kanner <andrew.kanner(a)gmail.com> fs/jfs: prevent double-free in dbUnmount() after failed jfs_remount() Georg Ottinger <g.ottinger(a)gmx.at> ext2: fix datatype of block number in ext2_xattr_set2() Zhang Shurong <zhang_shurong(a)foxmail.com> md: raid1: fix potential OOB in raid1_remove_disk() Tony Lindgren <tony(a)atomide.com> bus: ti-sysc: Configure uart quirks for k3 SoC AngeloGioacchino Del Regno <angelogioacchino.delregno(a)collabora.com> drm/mediatek: dp: Change logging to dev for mtk_dp_aux_transfer() Tuo Li <islituo(a)gmail.com> drm/exynos: fix a possible null-pointer dereference due to data race in exynos_drm_crtc_atomic_disable() Leo Chen <sancchen(a)amd.com> drm/amd/display: Blocking invalid 420 modes on HDMI TMDS for DCN314 Leo Chen <sancchen(a)amd.com> drm/amd/display: Blocking invalid 420 modes on HDMI TMDS for DCN31 Austin Zheng <austin.zheng(a)amd.com> drm/amd/display: Use DTBCLK as refclk instead of DPREFCLK Pierre-Louis Bossart <pierre-louis.bossart(a)linux.intel.com> ALSA: hda: intel-dsp-cfg: add LunarLake support Uday M Bhat <uday.m.bhat(a)intel.com> ASoC: Intel: sof_sdw: Update BT offload config for soundwire config Pierre-Louis Bossart <pierre-louis.bossart(a)linux.intel.com> ASoC: SOF: topology: simplify code to prevent static analysis warnings Leo Ma <hanghong.ma(a)amd.com> drm/amd/display: Fix underflow issue on 175hz timing Rong Tao <rongtao(a)cestc.cn> samples/hw_breakpoint: Fix kernel BUG 'invalid opcode: 0000' Krzysztof Kozlowski <krzysztof.kozlowski(a)linaro.org> arm64: dts: qcom: sm8250-edo: correct ramoops pmsg-size Krzysztof Kozlowski <krzysztof.kozlowski(a)linaro.org> arm64: dts: qcom: sm8150-kumano: correct ramoops pmsg-size Krzysztof Kozlowski <krzysztof.kozlowski(a)linaro.org> arm64: dts: qcom: sm6350: correct ramoops pmsg-size Krzysztof Kozlowski <krzysztof.kozlowski(a)linaro.org> arm64: dts: qcom: sm6125-pdx201: correct ramoops pmsg-size Ralph Campbell <rcampbell(a)nvidia.com> drm/edid: Add quirk for OSVR HDK 2.0 Marek Vasut <marex(a)denx.de> drm/bridge: tc358762: Instruct DSI host to generate HSE packets Hao Luo <haoluo(a)google.com> libbpf: Free btf_vmlinux when closing bpf_object Johannes Berg <johannes.berg(a)intel.com> wifi: mac80211_hwsim: drop short frames Johannes Berg <johannes.berg(a)intel.com> wifi: mac80211: check for station first in client probe Johannes Berg <johannes.berg(a)intel.com> wifi: cfg80211: ocb: don't leave if not joined Johannes Berg <johannes.berg(a)intel.com> wifi: cfg80211: reject auth/assoc to AP with our address GONG, Ruiqi <gongruiqi1(a)huawei.com> netfilter: ebtables: fix fortify warnings in size_entry_mwt() Johannes Berg <johannes.berg(a)intel.com> wifi: mac80211: check S1G action frame size GONG, Ruiqi <gongruiqi1(a)huawei.com> alx: fix OOB-read compiler warning Giulio Benetti <giulio.benetti(a)benettiengineering.com> mmc: sdhci-esdhc-imx: improve ESDHC_FLAG_ERR010450 Alexander Steffen <Alexander.Steffen(a)infineon.com> tpm_tis: Resend command to recover from data transfer errors Eric Dumazet <edumazet(a)google.com> netlink: convert nlk->flags to atomic flags Ying Hsu <yinghsu(a)chromium.org> Bluetooth: Fix hci_suspend_sync crash Mark O'Donovan <shiftee(a)posteo.net> crypto: lib/mpi - avoid null pointer deref in mpi_cmp_ui() xu xin <xu.xin16(a)zte.com.cn> net/ipv4: return the real errno instead of -EINVAL Kuniyuki Iwashima <kuniyu(a)amazon.com> net: Use sockaddr_storage for getsockopt(SO_PEERNAME). John Watts <contact(a)jookia.org> can: sun4i_can: Add support for the Allwinner D1 John Watts <contact(a)jookia.org> can: sun4i_can: Add acceptance register quirk Dmitry Antipov <dmantipov(a)yandex.ru> wifi: wil6210: fix fortify warnings Matt Whitlock <kernel(a)mattwhitlock.name> mt76: mt7921: don't assume adequate headroom for SDIO headers Dmitry Antipov <dmantipov(a)yandex.ru> wifi: mwifiex: fix fortify warning Dongliang Mu <dzm91(a)hust.edu.cn> wifi: ath9k: fix printk specifier Dmitry Antipov <dmantipov(a)yandex.ru> wifi: ath9k: fix fortify warnings Wojciech Drewek <wojciech.drewek(a)intel.com> ice: Don't tx before switchdev is fully configured Azeem Shaikh <azeemshaikh38(a)gmail.com> crypto: lrw,xts - Replace strlcpy with strscpy Jiri Pirko <jiri(a)nvidia.com> devlink: remove reload failed checks in params get/set callbacks Zhangjin Wu <falcon(a)tinylab.org> selftests/nolibc: fix up kernel parameters support Mario Limonciello <mario.limonciello(a)amd.com> ACPI: x86: s2idle: Catch multiple ACPI_TYPE_PACKAGE objects Tomislav Novak <tnovak(a)meta.com> hw_breakpoint: fix single-stepping when using bpf_overflow_handler Xu Yang <xu.yang_2(a)nxp.com> perf/imx_ddr: speed up overflow frequency of cycle Yicong Yang <yangyicong(a)hisilicon.com> perf/smmuv3: Enable HiSilicon Erratum 162001900 quirk for HIP08/09 Jiri Slaby (SUSE) <jirislaby(a)kernel.org> ACPI: video: Add backlight=native DMI quirk for Lenovo Ideapad Z470 Paul E. McKenney <paulmck(a)kernel.org> scftorture: Forgive memory-allocation failure if KASAN Zqiang <qiang.zhang1211(a)gmail.com> rcuscale: Move rcu_scale_writer() schedule_timeout_uninterruptible() to _idle() Wander Lairson Costa <wander(a)redhat.com> kernel/fork: beware of __put_task_struct() calling context Abhishek Mainkar <abmainkar(a)nvidia.com> ACPICA: Add AML_NO_OPERAND_RESOLVE flag to Timer Will Shiu <Will.Shiu(a)mediatek.com> locks: fix KASAN: use-after-free in trace_event_raw_event_filelock_lock Qu Wenruo <wqu(a)suse.com> btrfs: output extra debug info if we failed to find an inline backref Fedor Pchelkin <pchelkin(a)ispras.ru> autofs: fix memory leak of waitqueues in autofs_catatonic_mode ------------- Diffstat: Documentation/admin-guide/cgroup-v1/memory.rst | 2 + Documentation/arm64/silicon-errata.rst | 3 + Makefile | 4 +- arch/arm/kernel/hw_breakpoint.c | 8 +- arch/arm/kernel/machine_kexec.c | 14 +- .../dts/qcom/sm6125-sony-xperia-seine-pdx201.dts | 2 +- arch/arm64/boot/dts/qcom/sm6350.dtsi | 2 +- .../boot/dts/qcom/sm8150-sony-xperia-kumano.dtsi | 2 +- .../boot/dts/qcom/sm8250-sony-xperia-edo.dtsi | 2 +- arch/arm64/kernel/hw_breakpoint.c | 4 +- arch/mips/Makefile | 2 +- arch/mips/vdso/Makefile | 2 +- arch/powerpc/platforms/pseries/ibmebus.c | 1 + arch/riscv/kernel/elf_kexec.c | 8 +- arch/x86/boot/compressed/ident_map_64.c | 8 + arch/x86/include/asm/boot.h | 45 +- arch/x86/include/asm/linkage.h | 8 + arch/x86/purgatory/Makefile | 4 + block/bio-integrity.c | 7 +- block/bio.c | 12 +- crypto/lrw.c | 6 +- crypto/xts.c | 6 +- drivers/acpi/acpica/psopcode.c | 2 +- drivers/acpi/arm64/iort.c | 5 +- drivers/acpi/video_detect.c | 9 + drivers/acpi/x86/s2idle.c | 6 + drivers/ata/ahci.c | 9 + drivers/ata/libahci.c | 35 +- drivers/ata/libata-sata.c | 19 +- drivers/bus/ti-sysc.c | 2 + drivers/char/tpm/tpm_tis_core.c | 13 +- drivers/dma-buf/dma-buf.c | 53 ++ drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 - drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 18 +- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 - drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 19 +- drivers/gpu/drm/amd/display/dc/dcn32/dcn32_dccg.c | 3 +- .../amd/display/dc/dml/dcn31/display_mode_vba_31.c | 4 +- .../display/dc/dml/dcn314/display_mode_vba_314.c | 4 +- .../dc/dml/dcn32/display_mode_vba_util_32.c | 7 +- drivers/gpu/drm/bridge/tc358762.c | 2 +- drivers/gpu/drm/drm_edid.c | 1 + drivers/gpu/drm/exynos/exynos_drm_crtc.c | 5 +- drivers/gpu/drm/mediatek/mtk_dp.c | 6 +- drivers/gpu/drm/tiny/gm12u320.c | 10 +- drivers/i2c/busses/i2c-aspeed.c | 7 +- drivers/interconnect/core.c | 8 +- drivers/md/dm.c | 23 +- drivers/md/md.c | 2 +- drivers/md/raid1.c | 3 + drivers/media/pci/cx23885/cx23885-video.c | 2 +- drivers/media/pci/intel/ipu3/ipu3-cio2-main.c | 2 +- .../media/platform/mediatek/mdp3/mtk-mdp3-comp.c | 2 + drivers/media/tuners/qt1010.c | 11 +- drivers/media/usb/dvb-usb-v2/af9035.c | 14 +- drivers/media/usb/dvb-usb-v2/anysee.c | 2 +- drivers/media/usb/dvb-usb-v2/az6007.c | 8 + drivers/media/usb/dvb-usb-v2/gl861.c | 2 +- drivers/media/usb/dvb-usb/af9005.c | 5 + drivers/media/usb/dvb-usb/dw2102.c | 24 + drivers/misc/Kconfig | 1 + drivers/misc/fastrpc.c | 12 +- drivers/mmc/host/sdhci-esdhc-imx.c | 7 +- drivers/net/can/Kconfig | 4 +- drivers/net/can/sun4i_can.c | 22 +- drivers/net/ethernet/atheros/alx/ethtool.c | 5 +- drivers/net/ethernet/intel/ice/ice_eswitch.c | 3 + drivers/net/wireless/ath/ath9k/ahb.c | 4 +- drivers/net/wireless/ath/ath9k/mac.h | 6 +- drivers/net/wireless/ath/ath9k/pci.c | 4 +- drivers/net/wireless/ath/ath9k/xmit.c | 4 +- drivers/net/wireless/ath/wil6210/txrx.c | 2 +- drivers/net/wireless/ath/wil6210/txrx.h | 6 +- drivers/net/wireless/ath/wil6210/txrx_edma.c | 2 +- drivers/net/wireless/ath/wil6210/txrx_edma.h | 6 +- drivers/net/wireless/mac80211_hwsim.c | 7 +- drivers/net/wireless/marvell/mwifiex/tdls.c | 9 +- drivers/net/wireless/mediatek/mt76/mt7921/mac.c | 4 + drivers/nvme/host/core.c | 54 +- drivers/nvme/target/io-cmd-file.c | 10 +- drivers/nvme/target/tcp.c | 5 +- drivers/pci/controller/dwc/pci-imx6.c | 1 + drivers/pci/controller/dwc/pcie-fu740.c | 1 + drivers/pci/controller/vmd.c | 19 +- drivers/perf/arm_smmuv3_pmu.c | 46 +- drivers/perf/fsl_imx8_ddr_perf.c | 21 + drivers/scsi/lpfc/lpfc.h | 1 + drivers/scsi/lpfc/lpfc_debugfs.c | 14 +- drivers/scsi/lpfc/lpfc_els.c | 25 +- drivers/scsi/lpfc/lpfc_init.c | 20 +- drivers/scsi/lpfc/lpfc_sli.c | 8 +- drivers/scsi/megaraid/megaraid_sas.h | 2 +- drivers/scsi/megaraid/megaraid_sas_base.c | 21 +- drivers/scsi/pm8001/pm8001_init.c | 51 +- drivers/scsi/qla2xxx/qla_dfs.c | 6 +- drivers/target/iscsi/iscsi_target_configfs.c | 54 +- drivers/target/target_core_transport.c | 1 + drivers/tty/serial/cpm_uart/cpm_uart_core.c | 13 +- drivers/usb/cdns3/cdns3-plat.c | 3 +- drivers/usb/cdns3/cdnsp-pci.c | 3 +- drivers/usb/cdns3/core.c | 15 +- drivers/usb/cdns3/core.h | 7 +- drivers/usb/chipidea/ci.h | 1 + drivers/usb/chipidea/ci_hdrc_imx.c | 4 +- drivers/usb/chipidea/core.c | 2 + drivers/usb/chipidea/host.c | 1 + drivers/usb/gadget/udc/fsl_qe_udc.c | 2 + drivers/usb/host/ehci-hcd.c | 8 +- drivers/usb/host/ehci-hub.c | 10 +- drivers/usb/host/ehci.h | 10 + fs/attr.c | 20 +- fs/autofs/waitq.c | 3 +- fs/btrfs/delayed-inode.c | 19 +- fs/btrfs/disk-io.c | 22 +- fs/btrfs/extent-tree.c | 5 + fs/btrfs/ioctl.c | 8 +- fs/btrfs/ordered-data.c | 2 +- fs/btrfs/volumes.c | 8 + fs/btrfs/volumes.h | 1 + fs/ext2/xattr.c | 4 +- fs/ext4/namei.c | 26 +- fs/jfs/jfs_dmap.c | 1 + fs/jfs/jfs_imap.c | 1 + fs/locks.c | 2 +- fs/nfsd/nfs4proc.c | 4 +- fs/overlayfs/copy_up.c | 3 +- fs/overlayfs/file.c | 9 +- fs/tracefs/inode.c | 3 + include/linux/acpi_iort.h | 1 + include/linux/bvec.h | 15 + include/linux/dma-buf.h | 6 + include/linux/instruction_pointer.h | 5 + include/linux/libata.h | 4 + include/linux/perf_event.h | 22 +- include/linux/sched/task.h | 28 +- include/linux/usb/chipidea.h | 1 + include/uapi/linux/netfilter_bridge/ebtables.h | 14 +- io_uring/net.c | 5 + kernel/fork.c | 8 + kernel/panic.c | 1 + kernel/printk/printk.c | 80 ++- kernel/printk/printk_safe.c | 9 +- kernel/rcu/rcuscale.c | 2 +- kernel/scftorture.c | 6 +- kernel/trace/trace.c | 68 +- kernel/trace/trace.h | 2 + kernel/trace/trace_events.c | 6 +- kernel/trace/trace_events_inject.c | 3 +- lib/kobject.c | 5 + lib/mpi/mpi-cmp.c | 8 +- mm/memcontrol.c | 10 + net/bluetooth/hci_core.c | 4 + net/bridge/netfilter/ebtables.c | 3 +- net/core/sock.c | 6 +- net/devlink/leftover.c | 4 +- net/ipv4/ip_output.c | 2 +- net/mac80211/cfg.c | 15 +- net/mac80211/rx.c | 4 + net/netlink/af_netlink.c | 90 +-- net/netlink/af_netlink.h | 22 +- net/netlink/diag.c | 10 +- net/sched/Kconfig | 28 - net/sched/Makefile | 2 - net/sched/cls_rsvp.c | 24 - net/sched/cls_rsvp.h | 764 --------------------- net/sched/cls_rsvp6.c | 24 - net/sunrpc/clnt.c | 2 +- net/wireless/mlme.c | 13 + net/wireless/ocb.c | 3 + samples/hw_breakpoint/data_breakpoint.c | 4 +- security/selinux/hooks.c | 10 +- sound/hda/intel-dsp-config.c | 8 + sound/soc/intel/boards/sof_sdw.c | 4 +- sound/soc/sof/topology.c | 5 +- tools/iio/iio_generic_buffer.c | 17 +- tools/lib/bpf/libbpf.c | 1 + tools/testing/selftests/ftrace/ftracetest | 8 + tools/testing/selftests/nolibc/nolibc-test.c | 33 +- .../tc-testing/tc-tests/filters/rsvp.json | 203 ------ 179 files changed, 1227 insertions(+), 1609 deletions(-)

1 year, 7 months

12
156
0 0

+ arm64-hugetlb-fix-set_huge_pte_at-to-work-with-all-swap-entries.patch added to mm-hotfixes-unstable branch

by Andrew Morton

The patch titled Subject: arm64: hugetlb: fix set_huge_pte_at() to work with all swap entries has been added to the -mm mm-hotfixes-unstable branch. Its filename is arm64-hugetlb-fix-set_huge_pte_at-to-work-with-all-swap-entries.patch This patch will shortly appear at https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patche… This patch will later appear in the mm-hotfixes-unstable branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next via the mm-everything branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm and is updated there every 2-3 working days ------------------------------------------------------ From: Ryan Roberts <ryan.roberts(a)arm.com> Subject: arm64: hugetlb: fix set_huge_pte_at() to work with all swap entries Date: Fri, 22 Sep 2023 12:58:04 +0100 When called with a swap entry that does not embed a PFN (e.g. PTE_MARKER_POISONED or PTE_MARKER_UFFD_WP), the previous implementation of set_huge_pte_at() would either cause a BUG() to fire (if CONFIG_DEBUG_VM is enabled) or cause a dereference of an invalid address and subsequent panic. arm64's huge pte implementation supports multiple huge page sizes, some of which are implemented in the page table with multiple contiguous entries. So set_huge_pte_at() needs to work out how big the logical pte is, so that it can also work out how many physical ptes (or pmds) need to be written. It previously did this by grabbing the folio out of the pte and querying its size. However, there are cases when the pte being set is actually a swap entry. But this also used to work fine, because for huge ptes, we only ever saw migration entries and hwpoison entries. And both of these types of swap entries have a PFN embedded, so the code would grab that and everything still worked out. But over time, more calls to set_huge_pte_at() have been added that set swap entry types that do not embed a PFN. And this causes the code to go bang. The triggering case is for the uffd poison test, commit 99aa77215ad0 ("selftests/mm: add uffd unit test for UFFDIO_POISON"), which causes a PTE_MARKER_POISONED swap entry to be set, coutesey of commit 8a13897fb0da ("mm: userfaultfd: support UFFDIO_POISON for hugetlbfs") - added in v6.5-rc7. Although review shows that there are other call sites that set PTE_MARKER_UFFD_WP (which also has no PFN), these don't trigger on arm64 because arm64 doesn't support UFFD WP. Arguably, the root cause is really due to commit 18f3962953e4 ("mm: hugetlb: kill set_huge_swap_pte_at()"), which aimed to simplify the interface to the core code by removing set_huge_swap_pte_at() (which took a page size parameter) and replacing it with calls to set_huge_pte_at() where the size was inferred from the folio, as descibed above. While that commit didn't break anything at the time, it did break the interface because it couldn't handle swap entries without PFNs. And since then new callers have come along which rely on this working. But given the brokeness is only observable after commit 8a13897fb0da ("mm: userfaultfd: support UFFDIO_POISON for hugetlbfs"), that one gets the Fixes tag. Now that we have modified the set_huge_pte_at() interface to pass the huge page size in the previous patch, we can trivially fix this issue. Link: https://lkml.kernel.org/r/20230922115804.2043771-3-ryan.roberts@arm.com Fixes: 8a13897fb0da ("mm: userfaultfd: support UFFDIO_POISON for hugetlbfs") Signed-off-by: Ryan Roberts <ryan.roberts(a)arm.com> Cc: Albert Ou <aou(a)eecs.berkeley.edu> Cc: Alexander Gordeev <agordeev(a)linux.ibm.com> Cc: Alexandre Ghiti <alex(a)ghiti.fr> Cc: Anshuman Khandual <anshuman.khandual(a)arm.com> Cc: Arnd Bergmann <arnd(a)arndb.de> Cc: Axel Rasmussen <axelrasmussen(a)google.com> Cc: Catalin Marinas <catalin.marinas(a)arm.com> Cc: Christian Borntraeger <borntraeger(a)linux.ibm.com> Cc: Christophe Leroy <christophe.leroy(a)csgroup.eu> Cc: Christoph Hellwig <hch(a)infradead.org> Cc: David S. Miller <davem(a)davemloft.net> Cc: Gerald Schaefer <gerald.schaefer(a)linux.ibm.com> Cc: Heiko Carstens <hca(a)linux.ibm.com> Cc: Helge Deller <deller(a)gmx.de> Cc: "James E.J. Bottomley" <James.Bottomley(a)HansenPartnership.com> Cc: Lorenzo Stoakes <lstoakes(a)gmail.com> Cc: Mike Kravetz <mike.kravetz(a)oracle.com> Cc: Muchun Song <muchun.song(a)linux.dev> Cc: Nicholas Piggin <npiggin(a)gmail.com> Cc: Palmer Dabbelt <palmer(a)dabbelt.com> Cc: Paul Walmsley <paul.walmsley(a)sifive.com> Cc: Peter Xu <peterx(a)redhat.com> Cc: Qi Zheng <zhengqi.arch(a)bytedance.com> Cc: SeongJae Park <sj(a)kernel.org> Cc: Sven Schnelle <svens(a)linux.ibm.com> Cc: Uladzislau Rezki (Sony) <urezki(a)gmail.com> Cc: Vasily Gorbik <gor(a)linux.ibm.com> Cc: Will Deacon <will(a)kernel.org> Cc: <stable(a)vger.kernel.org> [6.5+] Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> --- arch/arm64/mm/hugetlbpage.c | 17 +++-------------- 1 file changed, 3 insertions(+), 14 deletions(-) --- a/arch/arm64/mm/hugetlbpage.c~arm64-hugetlb-fix-set_huge_pte_at-to-work-with-all-swap-entries +++ a/arch/arm64/mm/hugetlbpage.c @@ -241,13 +241,6 @@ static void clear_flush(struct mm_struct flush_tlb_range(&vma, saddr, addr); } -static inline struct folio *hugetlb_swap_entry_to_folio(swp_entry_t entry) -{ - VM_BUG_ON(!is_migration_entry(entry) && !is_hwpoison_entry(entry)); - - return page_folio(pfn_to_page(swp_offset_pfn(entry))); -} - void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte, unsigned long sz) { @@ -257,13 +250,10 @@ void set_huge_pte_at(struct mm_struct *m unsigned long pfn, dpfn; pgprot_t hugeprot; - if (!pte_present(pte)) { - struct folio *folio; - - folio = hugetlb_swap_entry_to_folio(pte_to_swp_entry(pte)); - ncontig = num_contig_ptes(folio_size(folio), &pgsize); + ncontig = num_contig_ptes(sz, &pgsize); - for (i = 0; i < ncontig; i++, ptep++) + if (!pte_present(pte)) { + for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) set_pte_at(mm, addr, ptep, pte); return; } @@ -273,7 +263,6 @@ void set_huge_pte_at(struct mm_struct *m return; } - ncontig = find_num_contig(mm, addr, ptep, &pgsize); pfn = pte_pfn(pte); dpfn = pgsize >> PAGE_SHIFT; hugeprot = pte_pgprot(pte); _ Patches currently in -mm which might be from ryan.roberts(a)arm.com are mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at.patch arm64-hugetlb-fix-set_huge_pte_at-to-work-with-all-swap-entries.patch

1 year, 7 months

1
0
0 0

+ mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at.patch added to mm-hotfixes-unstable branch

by Andrew Morton

The patch titled Subject: mm: hugetlb: add huge page size param to set_huge_pte_at() has been added to the -mm mm-hotfixes-unstable branch. Its filename is mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at.patch This patch will shortly appear at https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patche… This patch will later appear in the mm-hotfixes-unstable branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next via the mm-everything branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm and is updated there every 2-3 working days ------------------------------------------------------ From: Ryan Roberts <ryan.roberts(a)arm.com> Subject: mm: hugetlb: add huge page size param to set_huge_pte_at() Date: Fri, 22 Sep 2023 12:58:03 +0100 Patch series "Fix set_huge_pte_at() panic on arm64", v2. This series fixes a bug in arm64's implementation of set_huge_pte_at(), which can result in an unprivileged user causing a kernel panic. The problem was triggered when running the new uffd poison mm selftest for HUGETLB memory. This test (and the uffd poison feature) was merged for v6.5-rc7. Ideally, I'd like to get this fix in for v6.6 and I've cc'ed stable (correctly this time) to get it backported to v6.5, where the issue first showed up. Description of Bug ------------------ arm64's huge pte implementation supports multiple huge page sizes, some of which are implemented in the page table with multiple contiguous entries. So set_huge_pte_at() needs to work out how big the logical pte is, so that it can also work out how many physical ptes (or pmds) need to be written. It previously did this by grabbing the folio out of the pte and querying its size. However, there are cases when the pte being set is actually a swap entry. But this also used to work fine, because for huge ptes, we only ever saw migration entries and hwpoison entries. And both of these types of swap entries have a PFN embedded, so the code would grab that and everything still worked out. But over time, more calls to set_huge_pte_at() have been added that set swap entry types that do not embed a PFN. And this causes the code to go bang. The triggering case is for the uffd poison test, commit 99aa77215ad0 ("selftests/mm: add uffd unit test for UFFDIO_POISON"), which causes a PTE_MARKER_POISONED swap entry to be set, coutesey of commit 8a13897fb0da ("mm: userfaultfd: support UFFDIO_POISON for hugetlbfs") - added in v6.5-rc7. Although review shows that there are other call sites that set PTE_MARKER_UFFD_WP (which also has no PFN), these don't trigger on arm64 because arm64 doesn't support UFFD WP. If CONFIG_DEBUG_VM is enabled, we do at least get a BUG(), but otherwise, it will dereference a bad pointer in page_folio(): static inline struct folio *hugetlb_swap_entry_to_folio(swp_entry_t entry) { VM_BUG_ON(!is_migration_entry(entry) && !is_hwpoison_entry(entry)); return page_folio(pfn_to_page(swp_offset_pfn(entry))); } Fix --- The simplest fix would have been to revert the dodgy cleanup commit 18f3962953e4 ("mm: hugetlb: kill set_huge_swap_pte_at()"), but since things have moved on, this would have required an audit of all the new set_huge_pte_at() call sites to see if they should be converted to set_huge_swap_pte_at(). As per the original intent of the change, it would also leave us open to future bugs when people invariably get it wrong and call the wrong helper. So instead, I've added a huge page size parameter to set_huge_pte_at(). This means that the arm64 code has the size in all cases. It's a bigger change, due to needing to touch the arches that implement the function, but it is entirely mechanical, so in my view, low risk. I've compile-tested all touched arches; arm64, parisc, powerpc, riscv, s390, sparc (and additionally x86_64). I've additionally booted and run mm selftests against arm64, where I observe the uffd poison test is fixed, and there are no other regressions. This patch (of 2): In order to fix a bug, arm64 needs to be told the size of the huge page for which the pte is being set in set_huge_pte_at(). Provide for this by adding an `unsigned long sz` parameter to the function. This follows the same pattern as huge_pte_clear(). This commit makes the required interface modifications to the core mm as well as all arches that implement this function (arm64, parisc, powerpc, riscv, s390, sparc). The actual arm64 bug will be fixed in a separate commit. No behavioral changes intended. Link: https://lkml.kernel.org/r/20230922115804.2043771-1-ryan.roberts@arm.com Link: https://lkml.kernel.org/r/20230922115804.2043771-2-ryan.roberts@arm.com Fixes: 8a13897fb0da ("mm: userfaultfd: support UFFDIO_POISON for hugetlbfs") Signed-off-by: Ryan Roberts <ryan.roberts(a)arm.com> Cc: Alexandre Ghiti <alex(a)ghiti.fr> Cc: Albert Ou <aou(a)eecs.berkeley.edu> Cc: Alexander Gordeev <agordeev(a)linux.ibm.com> Cc: Anshuman Khandual <anshuman.khandual(a)arm.com> Cc: Arnd Bergmann <arnd(a)arndb.de> Cc: Axel Rasmussen <axelrasmussen(a)google.com> Cc: Catalin Marinas <catalin.marinas(a)arm.com> Cc: Christian Borntraeger <borntraeger(a)linux.ibm.com> Cc: Christophe Leroy <christophe.leroy(a)csgroup.eu> Cc: Christoph Hellwig <hch(a)infradead.org> Cc: David S. Miller <davem(a)davemloft.net> Cc: Gerald Schaefer <gerald.schaefer(a)linux.ibm.com> Cc: Heiko Carstens <hca(a)linux.ibm.com> Cc: Helge Deller <deller(a)gmx.de> Cc: "James E.J. Bottomley" <James.Bottomley(a)HansenPartnership.com> Cc: Lorenzo Stoakes <lstoakes(a)gmail.com> Cc: Mike Kravetz <mike.kravetz(a)oracle.com> Cc: Muchun Song <muchun.song(a)linux.dev> Cc: Nicholas Piggin <npiggin(a)gmail.com> Cc: Palmer Dabbelt <palmer(a)dabbelt.com> Cc: Paul Walmsley <paul.walmsley(a)sifive.com> Cc: Peter Xu <peterx(a)redhat.com> Cc: Qi Zheng <zhengqi.arch(a)bytedance.com> Cc: Ryan Roberts <ryan.roberts(a)arm.com> Cc: SeongJae Park <sj(a)kernel.org> Cc: Sven Schnelle <svens(a)linux.ibm.com> Cc: Uladzislau Rezki (Sony) <urezki(a)gmail.com> Cc: Vasily Gorbik <gor(a)linux.ibm.com> Cc: Will Deacon <will(a)kernel.org> Cc: <stable(a)vger.kernel.org> [6.5+] Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> --- arch/arm64/include/asm/hugetlb.h | 2 arch/arm64/mm/hugetlbpage.c | 6 + arch/parisc/include/asm/hugetlb.h | 2 arch/parisc/mm/hugetlbpage.c | 2 arch/powerpc/include/asm/nohash/32/hugetlb-8xx.h | 3 arch/powerpc/mm/book3s64/hugetlbpage.c | 5 + arch/powerpc/mm/book3s64/radix_hugetlbpage.c | 3 arch/powerpc/mm/nohash/8xx.c | 3 arch/powerpc/mm/pgtable.c | 3 arch/riscv/include/asm/hugetlb.h | 3 arch/riscv/mm/hugetlbpage.c | 3 arch/s390/include/asm/hugetlb.h | 6 + arch/s390/mm/hugetlbpage.c | 8 ++ arch/sparc/include/asm/hugetlb.h | 6 + arch/sparc/mm/hugetlbpage.c | 8 ++ include/asm-generic/hugetlb.h | 2 include/linux/hugetlb.h | 6 + mm/damon/vaddr.c | 3 mm/hugetlb.c | 43 +++++++------ mm/migrate.c | 7 +- mm/rmap.c | 23 +++++- mm/vmalloc.c | 2 22 files changed, 100 insertions(+), 49 deletions(-) --- a/arch/arm64/include/asm/hugetlb.h~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/arch/arm64/include/asm/hugetlb.h @@ -28,7 +28,7 @@ pte_t arch_make_huge_pte(pte_t entry, un #define arch_make_huge_pte arch_make_huge_pte #define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT extern void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte); + pte_t *ptep, pte_t pte, unsigned long sz); #define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, --- a/arch/arm64/mm/hugetlbpage.c~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/arch/arm64/mm/hugetlbpage.c @@ -249,7 +249,7 @@ static inline struct folio *hugetlb_swap } void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte) + pte_t *ptep, pte_t pte, unsigned long sz) { size_t pgsize; int i; @@ -571,5 +571,7 @@ pte_t huge_ptep_modify_prot_start(struct void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t old_pte, pte_t pte) { - set_huge_pte_at(vma->vm_mm, addr, ptep, pte); + unsigned long psize = huge_page_size(hstate_vma(vma)); + + set_huge_pte_at(vma->vm_mm, addr, ptep, pte, psize); } --- a/arch/parisc/include/asm/hugetlb.h~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/arch/parisc/include/asm/hugetlb.h @@ -6,7 +6,7 @@ #define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte); + pte_t *ptep, pte_t pte, unsigned long sz); #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, --- a/arch/parisc/mm/hugetlbpage.c~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/arch/parisc/mm/hugetlbpage.c @@ -140,7 +140,7 @@ static void __set_huge_pte_at(struct mm_ } void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t entry) + pte_t *ptep, pte_t entry, unsigned long sz) { __set_huge_pte_at(mm, addr, ptep, entry); } --- a/arch/powerpc/include/asm/nohash/32/hugetlb-8xx.h~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/arch/powerpc/include/asm/nohash/32/hugetlb-8xx.h @@ -46,7 +46,8 @@ static inline int check_and_get_huge_psi } #define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT -void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte); +void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, + pte_t pte, unsigned long sz); #define __HAVE_ARCH_HUGE_PTE_CLEAR static inline void huge_pte_clear(struct mm_struct *mm, unsigned long addr, --- a/arch/powerpc/mm/book3s64/hugetlbpage.c~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/arch/powerpc/mm/book3s64/hugetlbpage.c @@ -143,11 +143,14 @@ pte_t huge_ptep_modify_prot_start(struct void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t old_pte, pte_t pte) { + unsigned long psize; if (radix_enabled()) return radix__huge_ptep_modify_prot_commit(vma, addr, ptep, old_pte, pte); - set_huge_pte_at(vma->vm_mm, addr, ptep, pte); + + psize = huge_page_size(hstate_vma(vma)); + set_huge_pte_at(vma->vm_mm, addr, ptep, pte, psize); } void __init hugetlbpage_init_defaultsize(void) --- a/arch/powerpc/mm/book3s64/radix_hugetlbpage.c~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/arch/powerpc/mm/book3s64/radix_hugetlbpage.c @@ -47,6 +47,7 @@ void radix__huge_ptep_modify_prot_commit pte_t old_pte, pte_t pte) { struct mm_struct *mm = vma->vm_mm; + unsigned long psize = huge_page_size(hstate_vma(vma)); /* * POWER9 NMMU must flush the TLB after clearing the PTE before @@ -58,5 +59,5 @@ void radix__huge_ptep_modify_prot_commit atomic_read(&mm->context.copros) > 0) radix__flush_hugetlb_page(vma, addr); - set_huge_pte_at(vma->vm_mm, addr, ptep, pte); + set_huge_pte_at(vma->vm_mm, addr, ptep, pte, psize); } --- a/arch/powerpc/mm/nohash/8xx.c~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/arch/powerpc/mm/nohash/8xx.c @@ -91,7 +91,8 @@ static int __ref __early_map_kernel_huge if (new && WARN_ON(pte_present(*ptep) && pgprot_val(prot))) return -EINVAL; - set_huge_pte_at(&init_mm, va, ptep, pte_mkhuge(pfn_pte(pa >> PAGE_SHIFT, prot))); + set_huge_pte_at(&init_mm, va, ptep, + pte_mkhuge(pfn_pte(pa >> PAGE_SHIFT, prot)), psize); return 0; } --- a/arch/powerpc/mm/pgtable.c~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/arch/powerpc/mm/pgtable.c @@ -288,7 +288,8 @@ int huge_ptep_set_access_flags(struct vm } #if defined(CONFIG_PPC_8xx) -void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte) +void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, + pte_t pte, unsigned long sz) { pmd_t *pmd = pmd_off(mm, addr); pte_basic_t val; --- a/arch/riscv/include/asm/hugetlb.h~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/arch/riscv/include/asm/hugetlb.h @@ -18,7 +18,8 @@ void huge_pte_clear(struct mm_struct *mm #define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT void set_huge_pte_at(struct mm_struct *mm, - unsigned long addr, pte_t *ptep, pte_t pte); + unsigned long addr, pte_t *ptep, pte_t pte, + unsigned long sz); #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR pte_t huge_ptep_get_and_clear(struct mm_struct *mm, --- a/arch/riscv/mm/hugetlbpage.c~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/arch/riscv/mm/hugetlbpage.c @@ -180,7 +180,8 @@ pte_t arch_make_huge_pte(pte_t entry, un void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, - pte_t pte) + pte_t pte, + unsigned long sz) { int i, pte_num; --- a/arch/s390/include/asm/hugetlb.h~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/arch/s390/include/asm/hugetlb.h @@ -16,6 +16,8 @@ #define hugepages_supported() (MACHINE_HAS_EDAT1) void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pte, unsigned long sz); +void __set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte); pte_t huge_ptep_get(pte_t *ptep); pte_t huge_ptep_get_and_clear(struct mm_struct *mm, @@ -65,7 +67,7 @@ static inline int huge_ptep_set_access_f int changed = !pte_same(huge_ptep_get(ptep), pte); if (changed) { huge_ptep_get_and_clear(vma->vm_mm, addr, ptep); - set_huge_pte_at(vma->vm_mm, addr, ptep, pte); + __set_huge_pte_at(vma->vm_mm, addr, ptep, pte); } return changed; } @@ -74,7 +76,7 @@ static inline void huge_ptep_set_wrprote unsigned long addr, pte_t *ptep) { pte_t pte = huge_ptep_get_and_clear(mm, addr, ptep); - set_huge_pte_at(mm, addr, ptep, pte_wrprotect(pte)); + __set_huge_pte_at(mm, addr, ptep, pte_wrprotect(pte)); } static inline pte_t mk_huge_pte(struct page *page, pgprot_t pgprot) --- a/arch/s390/mm/hugetlbpage.c~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/arch/s390/mm/hugetlbpage.c @@ -142,7 +142,7 @@ static void clear_huge_pte_skeys(struct __storage_key_init_range(paddr, paddr + size - 1); } -void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, +void __set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte) { unsigned long rste; @@ -163,6 +163,12 @@ void set_huge_pte_at(struct mm_struct *m set_pte(ptep, __pte(rste)); } +void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pte, unsigned long sz) +{ + __set_huge_pte_at(mm, addr, ptep, pte); +} + pte_t huge_ptep_get(pte_t *ptep) { return __rste_to_pte(pte_val(*ptep)); --- a/arch/sparc/include/asm/hugetlb.h~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/arch/sparc/include/asm/hugetlb.h @@ -14,6 +14,8 @@ extern struct pud_huge_patch_entry __pud #define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pte, unsigned long sz); +void __set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte); #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR @@ -32,7 +34,7 @@ static inline void huge_ptep_set_wrprote unsigned long addr, pte_t *ptep) { pte_t old_pte = *ptep; - set_huge_pte_at(mm, addr, ptep, pte_wrprotect(old_pte)); + __set_huge_pte_at(mm, addr, ptep, pte_wrprotect(old_pte)); } #define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS @@ -42,7 +44,7 @@ static inline int huge_ptep_set_access_f { int changed = !pte_same(*ptep, pte); if (changed) { - set_huge_pte_at(vma->vm_mm, addr, ptep, pte); + __set_huge_pte_at(vma->vm_mm, addr, ptep, pte); flush_tlb_page(vma, addr); } return changed; --- a/arch/sparc/mm/hugetlbpage.c~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/arch/sparc/mm/hugetlbpage.c @@ -328,7 +328,7 @@ pte_t *huge_pte_offset(struct mm_struct return pte_offset_huge(pmd, addr); } -void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, +void __set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t entry) { unsigned int nptes, orig_shift, shift; @@ -364,6 +364,12 @@ void set_huge_pte_at(struct mm_struct *m orig_shift); } +void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t entry, unsigned long sz) +{ + __set_huge_pte_at(mm, addr, ptep, entry); +} + pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { --- a/include/asm-generic/hugetlb.h~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/include/asm-generic/hugetlb.h @@ -76,7 +76,7 @@ static inline void hugetlb_free_pgd_rang #ifndef __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT static inline void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte) + pte_t *ptep, pte_t pte, unsigned long sz) { set_pte_at(mm, addr, ptep, pte); } --- a/include/linux/hugetlb.h~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/include/linux/hugetlb.h @@ -984,7 +984,9 @@ static inline void huge_ptep_modify_prot unsigned long addr, pte_t *ptep, pte_t old_pte, pte_t pte) { - set_huge_pte_at(vma->vm_mm, addr, ptep, pte); + unsigned long psize = huge_page_size(hstate_vma(vma)); + + set_huge_pte_at(vma->vm_mm, addr, ptep, pte, psize); } #endif @@ -1173,7 +1175,7 @@ static inline pte_t huge_ptep_clear_flus } static inline void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte) + pte_t *ptep, pte_t pte, unsigned long sz) { } --- a/mm/damon/vaddr.c~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/mm/damon/vaddr.c @@ -341,13 +341,14 @@ static void damon_hugetlb_mkold(pte_t *p bool referenced = false; pte_t entry = huge_ptep_get(pte); struct folio *folio = pfn_folio(pte_pfn(entry)); + unsigned long psize = huge_page_size(hstate_vma(vma)); folio_get(folio); if (pte_young(entry)) { referenced = true; entry = pte_mkold(entry); - set_huge_pte_at(mm, addr, pte, entry); + set_huge_pte_at(mm, addr, pte, entry, psize); } #ifdef CONFIG_MMU_NOTIFIER --- a/mm/hugetlb.c~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/mm/hugetlb.c @@ -4980,7 +4980,7 @@ static bool is_hugetlb_entry_hwpoisoned( static void hugetlb_install_folio(struct vm_area_struct *vma, pte_t *ptep, unsigned long addr, - struct folio *new_folio, pte_t old) + struct folio *new_folio, pte_t old, unsigned long sz) { pte_t newpte = make_huge_pte(vma, &new_folio->page, 1); @@ -4988,7 +4988,7 @@ hugetlb_install_folio(struct vm_area_str hugepage_add_new_anon_rmap(new_folio, vma, addr); if (userfaultfd_wp(vma) && huge_pte_uffd_wp(old)) newpte = huge_pte_mkuffd_wp(newpte); - set_huge_pte_at(vma->vm_mm, addr, ptep, newpte); + set_huge_pte_at(vma->vm_mm, addr, ptep, newpte, sz); hugetlb_count_add(pages_per_huge_page(hstate_vma(vma)), vma->vm_mm); folio_set_hugetlb_migratable(new_folio); } @@ -5065,7 +5065,7 @@ again: } else if (unlikely(is_hugetlb_entry_hwpoisoned(entry))) { if (!userfaultfd_wp(dst_vma)) entry = huge_pte_clear_uffd_wp(entry); - set_huge_pte_at(dst, addr, dst_pte, entry); + set_huge_pte_at(dst, addr, dst_pte, entry, sz); } else if (unlikely(is_hugetlb_entry_migration(entry))) { swp_entry_t swp_entry = pte_to_swp_entry(entry); bool uffd_wp = pte_swp_uffd_wp(entry); @@ -5080,18 +5080,18 @@ again: entry = swp_entry_to_pte(swp_entry); if (userfaultfd_wp(src_vma) && uffd_wp) entry = pte_swp_mkuffd_wp(entry); - set_huge_pte_at(src, addr, src_pte, entry); + set_huge_pte_at(src, addr, src_pte, entry, sz); } if (!userfaultfd_wp(dst_vma)) entry = huge_pte_clear_uffd_wp(entry); - set_huge_pte_at(dst, addr, dst_pte, entry); + set_huge_pte_at(dst, addr, dst_pte, entry, sz); } else if (unlikely(is_pte_marker(entry))) { pte_marker marker = copy_pte_marker( pte_to_swp_entry(entry), dst_vma); if (marker) set_huge_pte_at(dst, addr, dst_pte, - make_pte_marker(marker)); + make_pte_marker(marker), sz); } else { entry = huge_ptep_get(src_pte); pte_folio = page_folio(pte_page(entry)); @@ -5145,7 +5145,7 @@ again: goto again; } hugetlb_install_folio(dst_vma, dst_pte, addr, - new_folio, src_pte_old); + new_folio, src_pte_old, sz); spin_unlock(src_ptl); spin_unlock(dst_ptl); continue; @@ -5166,7 +5166,7 @@ again: if (!userfaultfd_wp(dst_vma)) entry = huge_pte_clear_uffd_wp(entry); - set_huge_pte_at(dst, addr, dst_pte, entry); + set_huge_pte_at(dst, addr, dst_pte, entry, sz); hugetlb_count_add(npages, dst); } spin_unlock(src_ptl); @@ -5184,7 +5184,8 @@ again: } static void move_huge_pte(struct vm_area_struct *vma, unsigned long old_addr, - unsigned long new_addr, pte_t *src_pte, pte_t *dst_pte) + unsigned long new_addr, pte_t *src_pte, pte_t *dst_pte, + unsigned long sz) { struct hstate *h = hstate_vma(vma); struct mm_struct *mm = vma->vm_mm; @@ -5202,7 +5203,7 @@ static void move_huge_pte(struct vm_area spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING); pte = huge_ptep_get_and_clear(mm, old_addr, src_pte); - set_huge_pte_at(mm, new_addr, dst_pte, pte); + set_huge_pte_at(mm, new_addr, dst_pte, pte, sz); if (src_ptl != dst_ptl) spin_unlock(src_ptl); @@ -5259,7 +5260,7 @@ int move_hugetlb_page_tables(struct vm_a if (!dst_pte) break; - move_huge_pte(vma, old_addr, new_addr, src_pte, dst_pte); + move_huge_pte(vma, old_addr, new_addr, src_pte, dst_pte, sz); } if (shared_pmd) @@ -5337,7 +5338,8 @@ static void __unmap_hugepage_range(struc if (pte_swp_uffd_wp_any(pte) && !(zap_flags & ZAP_FLAG_DROP_MARKER)) set_huge_pte_at(mm, address, ptep, - make_pte_marker(PTE_MARKER_UFFD_WP)); + make_pte_marker(PTE_MARKER_UFFD_WP), + sz); else huge_pte_clear(mm, address, ptep, sz); spin_unlock(ptl); @@ -5371,7 +5373,8 @@ static void __unmap_hugepage_range(struc if (huge_pte_uffd_wp(pte) && !(zap_flags & ZAP_FLAG_DROP_MARKER)) set_huge_pte_at(mm, address, ptep, - make_pte_marker(PTE_MARKER_UFFD_WP)); + make_pte_marker(PTE_MARKER_UFFD_WP), + sz); hugetlb_count_sub(pages_per_huge_page(h), mm); page_remove_rmap(page, vma, true); @@ -5676,7 +5679,7 @@ retry_avoidcopy: hugepage_add_new_anon_rmap(new_folio, vma, haddr); if (huge_pte_uffd_wp(pte)) newpte = huge_pte_mkuffd_wp(newpte); - set_huge_pte_at(mm, haddr, ptep, newpte); + set_huge_pte_at(mm, haddr, ptep, newpte, huge_page_size(h)); folio_set_hugetlb_migratable(new_folio); /* Make the old page be freed below */ new_folio = old_folio; @@ -5972,7 +5975,7 @@ static vm_fault_t hugetlb_no_page(struct */ if (unlikely(pte_marker_uffd_wp(old_pte))) new_pte = huge_pte_mkuffd_wp(new_pte); - set_huge_pte_at(mm, haddr, ptep, new_pte); + set_huge_pte_at(mm, haddr, ptep, new_pte, huge_page_size(h)); hugetlb_count_add(pages_per_huge_page(h), mm); if ((flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED)) { @@ -6261,7 +6264,8 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_ } _dst_pte = make_pte_marker(PTE_MARKER_POISONED); - set_huge_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte); + set_huge_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte, + huge_page_size(h)); /* No need to invalidate - it was non-present before */ update_mmu_cache(dst_vma, dst_addr, dst_pte); @@ -6412,7 +6416,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_ if (wp_enabled) _dst_pte = huge_pte_mkuffd_wp(_dst_pte); - set_huge_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte); + set_huge_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte, huge_page_size(h)); hugetlb_count_add(pages_per_huge_page(h), dst_mm); @@ -6598,7 +6602,7 @@ long hugetlb_change_protection(struct vm else if (uffd_wp_resolve) newpte = pte_swp_clear_uffd_wp(newpte); if (!pte_same(pte, newpte)) - set_huge_pte_at(mm, address, ptep, newpte); + set_huge_pte_at(mm, address, ptep, newpte, psize); } else if (unlikely(is_pte_marker(pte))) { /* No other markers apply for now. */ WARN_ON_ONCE(!pte_marker_uffd_wp(pte)); @@ -6623,7 +6627,8 @@ long hugetlb_change_protection(struct vm if (unlikely(uffd_wp)) /* Safe to modify directly (none->non-present). */ set_huge_pte_at(mm, address, ptep, - make_pte_marker(PTE_MARKER_UFFD_WP)); + make_pte_marker(PTE_MARKER_UFFD_WP), + psize); } spin_unlock(ptl); } --- a/mm/migrate.c~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/mm/migrate.c @@ -243,7 +243,9 @@ static bool remove_migration_pte(struct #ifdef CONFIG_HUGETLB_PAGE if (folio_test_hugetlb(folio)) { - unsigned int shift = huge_page_shift(hstate_vma(vma)); + struct hstate *h = hstate_vma(vma); + unsigned int shift = huge_page_shift(h); + unsigned long psize = huge_page_size(h); pte = arch_make_huge_pte(pte, shift, vma->vm_flags); if (folio_test_anon(folio)) @@ -251,7 +253,8 @@ static bool remove_migration_pte(struct rmap_flags); else page_dup_file_rmap(new, true); - set_huge_pte_at(vma->vm_mm, pvmw.address, pvmw.pte, pte); + set_huge_pte_at(vma->vm_mm, pvmw.address, pvmw.pte, pte, + psize); } else #endif { --- a/mm/rmap.c~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/mm/rmap.c @@ -1480,6 +1480,7 @@ static bool try_to_unmap_one(struct foli struct mmu_notifier_range range; enum ttu_flags flags = (enum ttu_flags)(long)arg; unsigned long pfn; + unsigned long hsz = 0; /* * When racing against e.g. zap_pte_range() on another cpu, @@ -1511,6 +1512,9 @@ static bool try_to_unmap_one(struct foli */ adjust_range_if_pmd_sharing_possible(vma, &range.start, &range.end); + + /* We need the huge page size for set_huge_pte_at() */ + hsz = huge_page_size(hstate_vma(vma)); } mmu_notifier_invalidate_range_start(&range); @@ -1628,7 +1632,8 @@ static bool try_to_unmap_one(struct foli pteval = swp_entry_to_pte(make_hwpoison_entry(subpage)); if (folio_test_hugetlb(folio)) { hugetlb_count_sub(folio_nr_pages(folio), mm); - set_huge_pte_at(mm, address, pvmw.pte, pteval); + set_huge_pte_at(mm, address, pvmw.pte, pteval, + hsz); } else { dec_mm_counter(mm, mm_counter(&folio->page)); set_pte_at(mm, address, pvmw.pte, pteval); @@ -1820,6 +1825,7 @@ static bool try_to_migrate_one(struct fo struct mmu_notifier_range range; enum ttu_flags flags = (enum ttu_flags)(long)arg; unsigned long pfn; + unsigned long hsz = 0; /* * When racing against e.g. zap_pte_range() on another cpu, @@ -1855,6 +1861,9 @@ static bool try_to_migrate_one(struct fo */ adjust_range_if_pmd_sharing_possible(vma, &range.start, &range.end); + + /* We need the huge page size for set_huge_pte_at() */ + hsz = huge_page_size(hstate_vma(vma)); } mmu_notifier_invalidate_range_start(&range); @@ -2020,7 +2029,8 @@ static bool try_to_migrate_one(struct fo pteval = swp_entry_to_pte(make_hwpoison_entry(subpage)); if (folio_test_hugetlb(folio)) { hugetlb_count_sub(folio_nr_pages(folio), mm); - set_huge_pte_at(mm, address, pvmw.pte, pteval); + set_huge_pte_at(mm, address, pvmw.pte, pteval, + hsz); } else { dec_mm_counter(mm, mm_counter(&folio->page)); set_pte_at(mm, address, pvmw.pte, pteval); @@ -2044,7 +2054,8 @@ static bool try_to_migrate_one(struct fo if (arch_unmap_one(mm, vma, address, pteval) < 0) { if (folio_test_hugetlb(folio)) - set_huge_pte_at(mm, address, pvmw.pte, pteval); + set_huge_pte_at(mm, address, pvmw.pte, + pteval, hsz); else set_pte_at(mm, address, pvmw.pte, pteval); ret = false; @@ -2058,7 +2069,8 @@ static bool try_to_migrate_one(struct fo if (anon_exclusive && page_try_share_anon_rmap(subpage)) { if (folio_test_hugetlb(folio)) - set_huge_pte_at(mm, address, pvmw.pte, pteval); + set_huge_pte_at(mm, address, pvmw.pte, + pteval, hsz); else set_pte_at(mm, address, pvmw.pte, pteval); ret = false; @@ -2090,7 +2102,8 @@ static bool try_to_migrate_one(struct fo if (pte_uffd_wp(pteval)) swp_pte = pte_swp_mkuffd_wp(swp_pte); if (folio_test_hugetlb(folio)) - set_huge_pte_at(mm, address, pvmw.pte, swp_pte); + set_huge_pte_at(mm, address, pvmw.pte, swp_pte, + hsz); else set_pte_at(mm, address, pvmw.pte, swp_pte); trace_set_migration_pte(address, pte_val(swp_pte), --- a/mm/vmalloc.c~mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at +++ a/mm/vmalloc.c @@ -111,7 +111,7 @@ static int vmap_pte_range(pmd_t *pmd, un pte_t entry = pfn_pte(pfn, prot); entry = arch_make_huge_pte(entry, ilog2(size), 0); - set_huge_pte_at(&init_mm, addr, pte, entry); + set_huge_pte_at(&init_mm, addr, pte, entry, size); pfn += PFN_DOWN(size); continue; } _ Patches currently in -mm which might be from ryan.roberts(a)arm.com are mm-hugetlb-add-huge-page-size-param-to-set_huge_pte_at.patch arm64-hugetlb-fix-set_huge_pte_at-to-work-with-all-swap-entries.patch

1 year, 7 months

1
0
0 0

[PATCH -stable,5.15 00/17] Netfilter stable fixes for 5.15

by Pablo Neira Ayuso

Hi Greg, Sasha, The following list shows the backported patches, this batch is targeting at garbage collection (GC) / set timeout fixes that address possible UaF and memleaks. I am using original commit IDs for reference: 1) 24138933b97b ("netfilter: nf_tables: don't skip expired elements during walk") 2) 5f68718b34a5 ("netfilter: nf_tables: GC transaction API to avoid race with control plane") 3) f6c383b8c31a ("netfilter: nf_tables: adapt set backend to use GC transaction API") 4) c92db3030492 ("netfilter: nft_set_hash: mark set element as dead when deleting from packet path") 5) a2dd0233cbc4 ("netfilter: nf_tables: remove busy mark and gc batch API") 6) 7845914f45f0 ("netfilter: nf_tables: don't fail inserts if duplicate has expired") 7) 6a33d8b73dfa ("netfilter: nf_tables: fix GC transaction races with netns and netlink event exit path") 8) 02c6c24402bf ("netfilter: nf_tables: GC transaction race with netns dismantle") 9) 720344340fb9 ("netfilter: nf_tables: GC transaction race with abort path") 10) 8357bc946a2a ("netfilter: nf_tables: use correct lock to protect gc_list") 11) 8e51830e29e1 ("netfilter: nf_tables: defer gc run if previous batch is still pending") 12) 2ee52ae94baa ("netfilter: nft_set_rbtree: skip sync GC for new elements in this transaction") 13) 96b33300fba8 ("netfilter: nft_set_rbtree: use read spinlock to avoid datapath contention") 14) 4a9e12ea7e70 ("netfilter: nft_set_pipapo: call nft_trans_gc_queue_sync() in catchall GC") 15) 6d365eabce3c ("netfilter: nft_set_pipapo: stop GC iteration if GC transaction allocation fails") 16) b079155faae9 ("netfilter: nft_set_hash: try later when GC hits EAGAIN on iteration") 17) cf5000a7787c ("netfilter: nf_tables: fix memleak when more than 255 elements expired") Please, apply. Thanks. Florian Westphal (4): netfilter: nf_tables: don't skip expired elements during walk netfilter: nf_tables: don't fail inserts if duplicate has expired netfilter: nf_tables: defer gc run if previous batch is still pending netfilter: nf_tables: fix memleak when more than 255 elements expired Pablo Neira Ayuso (13): netfilter: nf_tables: GC transaction API to avoid race with control plane netfilter: nf_tables: adapt set backend to use GC transaction API netfilter: nft_set_hash: mark set element as dead when deleting from packet path netfilter: nf_tables: remove busy mark and gc batch API netfilter: nf_tables: fix GC transaction races with netns and netlink event exit path netfilter: nf_tables: GC transaction race with netns dismantle netfilter: nf_tables: GC transaction race with abort path netfilter: nf_tables: use correct lock to protect gc_list netfilter: nft_set_rbtree: skip sync GC for new elements in this transaction netfilter: nft_set_rbtree: use read spinlock to avoid datapath contention netfilter: nft_set_pipapo: call nft_trans_gc_queue_sync() in catchall GC netfilter: nft_set_pipapo: stop GC iteration if GC transaction allocation fails netfilter: nft_set_hash: try later when GC hits EAGAIN on iteration include/net/netfilter/nf_tables.h | 126 +++++----- net/netfilter/nf_tables_api.c | 369 ++++++++++++++++++++++++------ net/netfilter/nft_set_hash.c | 87 ++++--- net/netfilter/nft_set_pipapo.c | 67 +++--- net/netfilter/nft_set_rbtree.c | 161 +++++++------ 5 files changed, 547 insertions(+), 263 deletions(-) -- 2.30.2

1 year, 7 months

1
17
0 0

[PATCH -stable,6.1 00/17] Netfilter stable fixes for 6.1

by Pablo Neira Ayuso

Hi Greg, Sasha, The following list shows the backported patches, this batch is targeting at garbage collection (GC) / set timeout fixes that results in UaF and memleaks. I am using original commit IDs for reference: 1) 24138933b97b ("netfilter: nf_tables: don't skip expired elements during walk") 2) 5f68718b34a5 ("netfilter: nf_tables: GC transaction API to avoid race with control plane") 3) f6c383b8c31a ("netfilter: nf_tables: adapt set backend to use GC transaction API") 4) c92db3030492 ("netfilter: nft_set_hash: mark set element as dead when deleting from packet path") 5) a2dd0233cbc4 ("netfilter: nf_tables: remove busy mark and gc batch API") 6) 7845914f45f0 ("netfilter: nf_tables: don't fail inserts if duplicate has expired") 7) 6a33d8b73dfa ("netfilter: nf_tables: fix GC transaction races with netns and netlink event exit path") 8) 02c6c24402bf ("netfilter: nf_tables: GC transaction race with netns dismantle") 9) 720344340fb9 ("netfilter: nf_tables: GC transaction race with abort path") 10) 8357bc946a2a ("netfilter: nf_tables: use correct lock to protect gc_list") 11) 8e51830e29e1 ("netfilter: nf_tables: defer gc run if previous batch is still pending") 12) 2ee52ae94baa ("netfilter: nft_set_rbtree: skip sync GC for new elements in this transaction") 13) 96b33300fba8 ("netfilter: nft_set_rbtree: use read spinlock to avoid datapath contention") 14) 4a9e12ea7e70 ("netfilter: nft_set_pipapo: call nft_trans_gc_queue_sync() in catchall GC") 15) 6d365eabce3c ("netfilter: nft_set_pipapo: stop GC iteration if GC transaction allocation fails") 16) b079155faae9 ("netfilter: nft_set_hash: try later when GC hits EAGAIN on iteration") 17) cf5000a7787c ("netfilter: nf_tables: fix memleak when more than 255 elements expired") Please, apply. Thanks. Florian Westphal (4): netfilter: nf_tables: don't skip expired elements during walk netfilter: nf_tables: don't fail inserts if duplicate has expired netfilter: nf_tables: defer gc run if previous batch is still pending netfilter: nf_tables: fix memleak when more than 255 elements expired Pablo Neira Ayuso (13): netfilter: nf_tables: GC transaction API to avoid race with control plane netfilter: nf_tables: adapt set backend to use GC transaction API netfilter: nft_set_hash: mark set element as dead when deleting from packet path netfilter: nf_tables: remove busy mark and gc batch API netfilter: nf_tables: fix GC transaction races with netns and netlink event exit path netfilter: nf_tables: GC transaction race with netns dismantle netfilter: nf_tables: GC transaction race with abort path netfilter: nf_tables: use correct lock to protect gc_list netfilter: nft_set_rbtree: skip sync GC for new elements in this transaction netfilter: nft_set_rbtree: use read spinlock to avoid datapath contention netfilter: nft_set_pipapo: call nft_trans_gc_queue_sync() in catchall GC netfilter: nft_set_pipapo: stop GC iteration if GC transaction allocation fails netfilter: nft_set_hash: try later when GC hits EAGAIN on iteration include/net/netfilter/nf_tables.h | 126 +++++----- net/netfilter/nf_tables_api.c | 369 ++++++++++++++++++++++++------ net/netfilter/nft_set_hash.c | 87 ++++--- net/netfilter/nft_set_pipapo.c | 67 +++--- net/netfilter/nft_set_rbtree.c | 161 +++++++------ 5 files changed, 547 insertions(+), 263 deletions(-) -- 2.30.2

1 year, 7 months

2
18
0 0

[PATCH v2 2/2] maple_tree: Add MAS_UNDERFLOW and MAS_OVERFLOW states

by Liam R. Howlett

When updating the maple tree iterator to avoid rewalks, an issue was introduced when shifting beyond the limits. This can be seen by trying to go to the previous address of 0, which would set the maple node to MAS_NONE and keep the range as the last entry. Subsequent calls to mas_find() would then search upwards from mas->last and skip the value at mas->index/mas->last. This showed up as a bug in mprotect which skips the actual VMA at the current range after attempting to go to the previous VMA from 0. Since MAS_NONE may already be set when searching for a value that isn't contained within a node, changing the handling of MAS_NONE in mas_find() would make the code more complicated and error prone. Furthermore, there was no way to tell which limit was hit, and thus which action to take (next or the entry at the current range). This solution is to add two states to track what happened with the previous iterator action. This allows for the expected behaviour of the next command to return the correct item (either the item at the range requested, or the next/previous). Tests are also added and updated accordingly. Reported-by: pedro.falcato(a)gmail.com Closes: https://bugs.archlinux.org/task/79656 Link: https://gist.github.com/heatd/85d2971fae1501b55b6ea401fbbe485b Link: https://lore.kernel.org/linux-mm/20230921181236.509072-1-Liam.Howlett@oracl… Fixes: 39193685d585 ("maple_tree: try harder to keep active node with mas_prev()") Cc: stable(a)vger.kernel.org Signed-off-by: Liam R. Howlett <Liam.Howlett(a)oracle.com> --- include/linux/maple_tree.h | 2 + lib/maple_tree.c | 221 +++++++++++++++++++++++++++---------- lib/test_maple_tree.c | 87 ++++++++++++--- 3 files changed, 237 insertions(+), 73 deletions(-) diff --git a/include/linux/maple_tree.h b/include/linux/maple_tree.h index f66f5f78f8cf..d01e850b570f 100644 --- a/include/linux/maple_tree.h +++ b/include/linux/maple_tree.h @@ -428,6 +428,8 @@ struct ma_wr_state { #define MAS_ROOT ((struct maple_enode *)5UL) #define MAS_NONE ((struct maple_enode *)9UL) #define MAS_PAUSE ((struct maple_enode *)17UL) +#define MAS_OVERFLOW ((struct maple_enode *)33UL) +#define MAS_UNDERFLOW ((struct maple_enode *)65UL) #define MA_ERROR(err) \ ((struct maple_enode *)(((unsigned long)err << 2) | 2UL)) diff --git a/lib/maple_tree.c b/lib/maple_tree.c index ee1ff0c59fd7..0e00a84e8e8f 100644 --- a/lib/maple_tree.c +++ b/lib/maple_tree.c @@ -256,6 +256,22 @@ bool mas_is_err(struct ma_state *mas) return xa_is_err(mas->node); } +static __always_inline bool mas_is_overflow(struct ma_state *mas) +{ + if (unlikely(mas->node == MAS_OVERFLOW)) + return true; + + return false; +} + +static __always_inline bool mas_is_underflow(struct ma_state *mas) +{ + if (unlikely(mas->node == MAS_UNDERFLOW)) + return true; + + return false; +} + static inline bool mas_searchable(struct ma_state *mas) { if (mas_is_none(mas)) @@ -4415,10 +4431,13 @@ static inline int mas_prev_node(struct ma_state *mas, unsigned long min) * * @mas: The maple state * @max: The minimum starting range + * @empty: Can be empty + * @set_underflow: Set the @mas->node to underflow state on limit. * * Return: The entry in the previous slot which is possibly NULL */ -static void *mas_prev_slot(struct ma_state *mas, unsigned long min, bool empty) +static void *mas_prev_slot(struct ma_state *mas, unsigned long min, bool empty, + bool set_underflow) { void *entry; void __rcu **slots; @@ -4435,7 +4454,6 @@ static void *mas_prev_slot(struct ma_state *mas, unsigned long min, bool empty) if (unlikely(mas_rewalk_if_dead(mas, node, save_point))) goto retry; -again: if (mas->min <= min) { pivot = mas_safe_min(mas, pivots, mas->offset); @@ -4443,9 +4461,10 @@ static void *mas_prev_slot(struct ma_state *mas, unsigned long min, bool empty) goto retry; if (pivot <= min) - return NULL; + goto underflow; } +again: if (likely(mas->offset)) { mas->offset--; mas->last = mas->index - 1; @@ -4457,7 +4476,7 @@ static void *mas_prev_slot(struct ma_state *mas, unsigned long min, bool empty) } if (mas_is_none(mas)) - return NULL; + goto underflow; mas->last = mas->max; node = mas_mn(mas); @@ -4474,10 +4493,19 @@ static void *mas_prev_slot(struct ma_state *mas, unsigned long min, bool empty) if (likely(entry)) return entry; - if (!empty) + if (!empty) { + if (mas->index <= min) + goto underflow; + goto again; + } return entry; + +underflow: + if (set_underflow) + mas->node = MAS_UNDERFLOW; + return NULL; } /* @@ -4567,10 +4595,13 @@ static inline int mas_next_node(struct ma_state *mas, struct maple_node *node, * @mas: The maple state * @max: The maximum starting range * @empty: Can be empty + * @set_overflow: Should @mas->node be set to overflow when the limit is + * reached. * * Return: The entry in the next slot which is possibly NULL */ -static void *mas_next_slot(struct ma_state *mas, unsigned long max, bool empty) +static void *mas_next_slot(struct ma_state *mas, unsigned long max, bool empty, + bool set_overflow) { void __rcu **slots; unsigned long *pivots; @@ -4589,22 +4620,22 @@ static void *mas_next_slot(struct ma_state *mas, unsigned long max, bool empty) if (unlikely(mas_rewalk_if_dead(mas, node, save_point))) goto retry; -again: if (mas->max >= max) { if (likely(mas->offset < data_end)) pivot = pivots[mas->offset]; else - return NULL; /* must be mas->max */ + goto overflow; if (unlikely(mas_rewalk_if_dead(mas, node, save_point))) goto retry; if (pivot >= max) - return NULL; + goto overflow; } if (likely(mas->offset < data_end)) { mas->index = pivots[mas->offset] + 1; +again: mas->offset++; if (likely(mas->offset < data_end)) mas->last = pivots[mas->offset]; @@ -4616,8 +4647,11 @@ static void *mas_next_slot(struct ma_state *mas, unsigned long max, bool empty) goto retry; } - if (mas_is_none(mas)) + if (WARN_ON_ONCE(mas_is_none(mas))) { + mas->node = MAS_OVERFLOW; return NULL; + goto overflow; + } mas->offset = 0; mas->index = mas->min; @@ -4636,12 +4670,20 @@ static void *mas_next_slot(struct ma_state *mas, unsigned long max, bool empty) return entry; if (!empty) { - if (!mas->offset) - data_end = 2; + if (mas->last >= max) + goto overflow; + + mas->index = mas->last + 1; + /* Node cannot end on NULL, so it's safe to short-cut here */ goto again; } return entry; + +overflow: + if (set_overflow) + mas->node = MAS_OVERFLOW; + return NULL; } /* @@ -4651,17 +4693,20 @@ static void *mas_next_slot(struct ma_state *mas, unsigned long max, bool empty) * * Set the @mas->node to the next entry and the range_start to * the beginning value for the entry. Does not check beyond @limit. - * Sets @mas->index and @mas->last to the limit if it is hit. + * Sets @mas->index and @mas->last to the range, Does not update @mas->index and + * @mas->last on overflow. * Restarts on dead nodes. * * Return: the next entry or %NULL. */ static inline void *mas_next_entry(struct ma_state *mas, unsigned long limit) { - if (mas->last >= limit) + if (mas->last >= limit) { + mas->node = MAS_OVERFLOW; return NULL; + } - return mas_next_slot(mas, limit, false); + return mas_next_slot(mas, limit, false, true); } /* @@ -4837,7 +4882,7 @@ void *mas_walk(struct ma_state *mas) { void *entry; - if (mas_is_none(mas) || mas_is_paused(mas) || mas_is_ptr(mas)) + if (!mas_is_active(mas) || !mas_is_start(mas)) mas->node = MAS_START; retry: entry = mas_state_walk(mas); @@ -5294,14 +5339,22 @@ static inline void mte_destroy_walk(struct maple_enode *enode, static void mas_wr_store_setup(struct ma_wr_state *wr_mas) { - if (mas_is_start(wr_mas->mas)) - return; + if (!mas_is_active(wr_mas->mas)) { + if (mas_is_start(wr_mas->mas)) + return; - if (unlikely(mas_is_paused(wr_mas->mas))) - goto reset; + if (unlikely(mas_is_paused(wr_mas->mas))) + goto reset; - if (unlikely(mas_is_none(wr_mas->mas))) - goto reset; + if (unlikely(mas_is_none(wr_mas->mas))) + goto reset; + + if (unlikely(mas_is_overflow(wr_mas->mas))) + goto reset; + + if (unlikely(mas_is_underflow(wr_mas->mas))) + goto reset; + } /* * A less strict version of mas_is_span_wr() where we allow spanning @@ -5595,8 +5648,25 @@ static inline bool mas_next_setup(struct ma_state *mas, unsigned long max, { bool was_none = mas_is_none(mas); - if (mas_is_none(mas) || mas_is_paused(mas)) + if (unlikely(mas->last >= max)) { + mas->node = MAS_OVERFLOW; + return true; + } + + if (mas_is_active(mas)) + return false; + + if (mas_is_none(mas) || mas_is_paused(mas)) { + mas->node = MAS_START; + } else if (mas_is_overflow(mas)) { + /* Overflowed before, but the max changed */ mas->node = MAS_START; + } else if (mas_is_underflow(mas)) { + mas->node = MAS_START; + *entry = mas_walk(mas); + if (*entry) + return true; + } if (mas_is_start(mas)) *entry = mas_walk(mas); /* Retries on dead nodes handled by mas_walk */ @@ -5615,6 +5685,7 @@ static inline bool mas_next_setup(struct ma_state *mas, unsigned long max, if (mas_is_none(mas)) return true; + return false; } @@ -5637,7 +5708,7 @@ void *mas_next(struct ma_state *mas, unsigned long max) return entry; /* Retries on dead nodes handled by mas_next_slot */ - return mas_next_slot(mas, max, false); + return mas_next_slot(mas, max, false, true); } EXPORT_SYMBOL_GPL(mas_next); @@ -5660,7 +5731,7 @@ void *mas_next_range(struct ma_state *mas, unsigned long max) return entry; /* Retries on dead nodes handled by mas_next_slot */ - return mas_next_slot(mas, max, true); + return mas_next_slot(mas, max, true, true); } EXPORT_SYMBOL_GPL(mas_next_range); @@ -5691,18 +5762,31 @@ EXPORT_SYMBOL_GPL(mt_next); static inline bool mas_prev_setup(struct ma_state *mas, unsigned long min, void **entry) { - if (mas->index <= min) - goto none; + if (unlikely(mas->index <= min)) { + mas->node = MAS_UNDERFLOW; + return true; + } - if (mas_is_none(mas) || mas_is_paused(mas)) + if (mas_is_active(mas)) + return false; + + if (mas_is_overflow(mas)) { mas->node = MAS_START; + *entry = mas_walk(mas); + if (*entry) + return true; + } - if (mas_is_start(mas)) { - mas_walk(mas); - if (!mas->index) - goto none; + if (mas_is_none(mas) || mas_is_paused(mas)) { + mas->node = MAS_START; + } else if (mas_is_underflow(mas)) { + /* underflowed before but the min changed */ + mas->node = MAS_START; } + if (mas_is_start(mas)) + mas_walk(mas); + if (unlikely(mas_is_ptr(mas))) { if (!mas->index) goto none; @@ -5747,7 +5831,7 @@ void *mas_prev(struct ma_state *mas, unsigned long min) if (mas_prev_setup(mas, min, &entry)) return entry; - return mas_prev_slot(mas, min, false); + return mas_prev_slot(mas, min, false, true); } EXPORT_SYMBOL_GPL(mas_prev); @@ -5770,7 +5854,7 @@ void *mas_prev_range(struct ma_state *mas, unsigned long min) if (mas_prev_setup(mas, min, &entry)) return entry; - return mas_prev_slot(mas, min, true); + return mas_prev_slot(mas, min, true, true); } EXPORT_SYMBOL_GPL(mas_prev_range); @@ -5828,24 +5912,35 @@ EXPORT_SYMBOL_GPL(mas_pause); static inline bool mas_find_setup(struct ma_state *mas, unsigned long max, void **entry) { - *entry = NULL; + if (mas_is_active(mas)) { + if (mas->last < max) + return false; - if (unlikely(mas_is_none(mas))) { + return true; + } + + if (mas_is_paused(mas)) { if (unlikely(mas->last >= max)) return true; - mas->index = mas->last; + mas->index = ++mas->last; mas->node = MAS_START; - } else if (unlikely(mas_is_paused(mas))) { + } else if (mas_is_none(mas)) { if (unlikely(mas->last >= max)) return true; + mas->index = mas->last; mas->node = MAS_START; - mas->index = ++mas->last; - } else if (unlikely(mas_is_ptr(mas))) - goto ptr_out_of_range; + } else if (mas_is_overflow(mas) || mas_is_underflow(mas)) { + if (mas->index > max) { + mas->node = MAS_OVERFLOW; + return true; + } + + mas->node = MAS_START; + } - if (unlikely(mas_is_start(mas))) { + if (mas_is_start(mas)) { /* First run or continue */ if (mas->index > max) return true; @@ -5895,7 +5990,7 @@ void *mas_find(struct ma_state *mas, unsigned long max) return entry; /* Retries on dead nodes handled by mas_next_slot */ - return mas_next_slot(mas, max, false); + return mas_next_slot(mas, max, false, false); } EXPORT_SYMBOL_GPL(mas_find); @@ -5913,13 +6008,13 @@ EXPORT_SYMBOL_GPL(mas_find); */ void *mas_find_range(struct ma_state *mas, unsigned long max) { - void *entry; + void *entry = NULL; if (mas_find_setup(mas, max, &entry)) return entry; /* Retries on dead nodes handled by mas_next_slot */ - return mas_next_slot(mas, max, true); + return mas_next_slot(mas, max, true, false); } EXPORT_SYMBOL_GPL(mas_find_range); @@ -5934,26 +6029,36 @@ EXPORT_SYMBOL_GPL(mas_find_range); static inline bool mas_find_rev_setup(struct ma_state *mas, unsigned long min, void **entry) { - *entry = NULL; - - if (unlikely(mas_is_none(mas))) { - if (mas->index <= min) - goto none; + if (mas_is_active(mas)) { + if (mas->index > min) + return false; - mas->last = mas->index; - mas->node = MAS_START; + return true; } - if (unlikely(mas_is_paused(mas))) { + if (mas_is_paused(mas)) { if (unlikely(mas->index <= min)) { mas->node = MAS_NONE; return true; } mas->node = MAS_START; mas->last = --mas->index; + } else if (mas_is_none(mas)) { + if (mas->index <= min) + goto none; + + mas->last = mas->index; + mas->node = MAS_START; + } else if (mas_is_underflow(mas) || mas_is_overflow(mas)) { + if (mas->last <= min) { + mas->node = MAS_UNDERFLOW; + return true; + } + + mas->node = MAS_START; } - if (unlikely(mas_is_start(mas))) { + if (mas_is_start(mas)) { /* First run or continue */ if (mas->index < min) return true; @@ -6004,13 +6109,13 @@ static inline bool mas_find_rev_setup(struct ma_state *mas, unsigned long min, */ void *mas_find_rev(struct ma_state *mas, unsigned long min) { - void *entry; + void *entry = NULL; if (mas_find_rev_setup(mas, min, &entry)) return entry; /* Retries on dead nodes handled by mas_prev_slot */ - return mas_prev_slot(mas, min, false); + return mas_prev_slot(mas, min, false, false); } EXPORT_SYMBOL_GPL(mas_find_rev); @@ -6030,13 +6135,13 @@ EXPORT_SYMBOL_GPL(mas_find_rev); */ void *mas_find_range_rev(struct ma_state *mas, unsigned long min) { - void *entry; + void *entry = NULL; if (mas_find_rev_setup(mas, min, &entry)) return entry; /* Retries on dead nodes handled by mas_prev_slot */ - return mas_prev_slot(mas, min, true); + return mas_prev_slot(mas, min, true, false); } EXPORT_SYMBOL_GPL(mas_find_range_rev); diff --git a/lib/test_maple_tree.c b/lib/test_maple_tree.c index 0674aebd4423..06959165e2f9 100644 --- a/lib/test_maple_tree.c +++ b/lib/test_maple_tree.c @@ -2166,7 +2166,7 @@ static noinline void __init next_prev_test(struct maple_tree *mt) MT_BUG_ON(mt, val != NULL); MT_BUG_ON(mt, mas.index != 0); MT_BUG_ON(mt, mas.last != 5); - MT_BUG_ON(mt, mas.node != MAS_NONE); + MT_BUG_ON(mt, mas.node != MAS_UNDERFLOW); mas.index = 0; mas.last = 5; @@ -2917,6 +2917,7 @@ static noinline void __init check_empty_area_fill(struct maple_tree *mt) * exists MAS_NONE active range * exists active active range * DNE active active set to last range + * ERANGE active MAS_OVERFLOW last range * * Function ENTRY Start Result index & last * mas_prev() @@ -2945,6 +2946,7 @@ static noinline void __init check_empty_area_fill(struct maple_tree *mt) * any MAS_ROOT MAS_NONE 0 * exists active active range * DNE active active last range + * ERANGE active MAS_UNDERFLOW last range * * Function ENTRY Start Result index & last * mas_find() @@ -2955,7 +2957,7 @@ static noinline void __init check_empty_area_fill(struct maple_tree *mt) * DNE MAS_START MAS_NONE 0 * DNE MAS_PAUSE MAS_NONE 0 * DNE MAS_ROOT MAS_NONE 0 - * DNE MAS_NONE MAS_NONE 0 + * DNE MAS_NONE MAS_NONE 1 * if index == 0 * exists MAS_START MAS_ROOT 0 * exists MAS_PAUSE MAS_ROOT 0 @@ -2967,7 +2969,7 @@ static noinline void __init check_empty_area_fill(struct maple_tree *mt) * DNE MAS_START active set to max * exists MAS_PAUSE active range * DNE MAS_PAUSE active set to max - * exists MAS_NONE active range + * exists MAS_NONE active range (start at last) * exists active active range * DNE active active last range (max < last) * @@ -2992,7 +2994,7 @@ static noinline void __init check_empty_area_fill(struct maple_tree *mt) * DNE MAS_START active set to min * exists MAS_PAUSE active range * DNE MAS_PAUSE active set to min - * exists MAS_NONE active range + * exists MAS_NONE active range (start at index) * exists active active range * DNE active active last range (min > index) * @@ -3039,10 +3041,10 @@ static noinline void __init check_state_handling(struct maple_tree *mt) mtree_store_range(mt, 0, 0, ptr, GFP_KERNEL); mas_lock(&mas); - /* prev: Start -> none */ + /* prev: Start -> underflow*/ entry = mas_prev(&mas, 0); MT_BUG_ON(mt, entry != NULL); - MT_BUG_ON(mt, mas.node != MAS_NONE); + MT_BUG_ON(mt, mas.node != MAS_UNDERFLOW); /* prev: Start -> root */ mas_set(&mas, 10); @@ -3069,7 +3071,7 @@ static noinline void __init check_state_handling(struct maple_tree *mt) MT_BUG_ON(mt, entry != NULL); MT_BUG_ON(mt, mas.node != MAS_NONE); - /* next: start -> none */ + /* next: start -> none*/ mas_set(&mas, 10); entry = mas_next(&mas, ULONG_MAX); MT_BUG_ON(mt, mas.index != 1); @@ -3268,25 +3270,46 @@ static noinline void __init check_state_handling(struct maple_tree *mt) MT_BUG_ON(mt, mas.last != 0x2500); MT_BUG_ON(mt, !mas_active(mas)); - /* next:active -> active out of range*/ + /* next:active -> active beyond data */ entry = mas_next(&mas, 0x2999); MT_BUG_ON(mt, entry != NULL); MT_BUG_ON(mt, mas.index != 0x2501); MT_BUG_ON(mt, mas.last != 0x2fff); MT_BUG_ON(mt, !mas_active(mas)); - /* Continue after out of range*/ + /* Continue after last range ends after max */ entry = mas_next(&mas, ULONG_MAX); MT_BUG_ON(mt, entry != ptr3); MT_BUG_ON(mt, mas.index != 0x3000); MT_BUG_ON(mt, mas.last != 0x3500); MT_BUG_ON(mt, !mas_active(mas)); - /* next:active -> active out of range*/ + /* next:active -> active continued */ + entry = mas_next(&mas, ULONG_MAX); + MT_BUG_ON(mt, entry != NULL); + MT_BUG_ON(mt, mas.index != 0x3501); + MT_BUG_ON(mt, mas.last != ULONG_MAX); + MT_BUG_ON(mt, !mas_active(mas)); + + /* next:active -> overflow */ entry = mas_next(&mas, ULONG_MAX); MT_BUG_ON(mt, entry != NULL); MT_BUG_ON(mt, mas.index != 0x3501); MT_BUG_ON(mt, mas.last != ULONG_MAX); + MT_BUG_ON(mt, mas.node != MAS_OVERFLOW); + + /* next:overflow -> overflow */ + entry = mas_next(&mas, ULONG_MAX); + MT_BUG_ON(mt, entry != NULL); + MT_BUG_ON(mt, mas.index != 0x3501); + MT_BUG_ON(mt, mas.last != ULONG_MAX); + MT_BUG_ON(mt, mas.node != MAS_OVERFLOW); + + /* prev:overflow -> active */ + entry = mas_prev(&mas, 0); + MT_BUG_ON(mt, entry != ptr3); + MT_BUG_ON(mt, mas.index != 0x3000); + MT_BUG_ON(mt, mas.last != 0x3500); MT_BUG_ON(mt, !mas_active(mas)); /* next: none -> active, skip value at location */ @@ -3307,11 +3330,46 @@ static noinline void __init check_state_handling(struct maple_tree *mt) MT_BUG_ON(mt, mas.last != 0x1500); MT_BUG_ON(mt, !mas_active(mas)); - /* prev:active -> active out of range*/ + /* prev:active -> active spanning end range */ + entry = mas_prev(&mas, 0x0100); + MT_BUG_ON(mt, entry != NULL); + MT_BUG_ON(mt, mas.index != 0); + MT_BUG_ON(mt, mas.last != 0x0FFF); + MT_BUG_ON(mt, !mas_active(mas)); + + /* prev:active -> underflow */ + entry = mas_prev(&mas, 0); + MT_BUG_ON(mt, entry != NULL); + MT_BUG_ON(mt, mas.index != 0); + MT_BUG_ON(mt, mas.last != 0x0FFF); + MT_BUG_ON(mt, mas.node != MAS_UNDERFLOW); + + /* prev:underflow -> underflow */ entry = mas_prev(&mas, 0); MT_BUG_ON(mt, entry != NULL); MT_BUG_ON(mt, mas.index != 0); MT_BUG_ON(mt, mas.last != 0x0FFF); + MT_BUG_ON(mt, mas.node != MAS_UNDERFLOW); + + /* next:underflow -> active */ + entry = mas_next(&mas, ULONG_MAX); + MT_BUG_ON(mt, entry != ptr); + MT_BUG_ON(mt, mas.index != 0x1000); + MT_BUG_ON(mt, mas.last != 0x1500); + MT_BUG_ON(mt, !mas_active(mas)); + + /* prev:first value -> underflow */ + entry = mas_prev(&mas, 0x1000); + MT_BUG_ON(mt, entry != NULL); + MT_BUG_ON(mt, mas.index != 0x1000); + MT_BUG_ON(mt, mas.last != 0x1500); + MT_BUG_ON(mt, mas.node != MAS_UNDERFLOW); + + /* find:underflow -> first value */ + entry = mas_find(&mas, ULONG_MAX); + MT_BUG_ON(mt, entry != ptr); + MT_BUG_ON(mt, mas.index != 0x1000); + MT_BUG_ON(mt, mas.last != 0x1500); MT_BUG_ON(mt, !mas_active(mas)); /* prev: pause ->active */ @@ -3325,14 +3383,14 @@ static noinline void __init check_state_handling(struct maple_tree *mt) MT_BUG_ON(mt, mas.last != 0x2500); MT_BUG_ON(mt, !mas_active(mas)); - /* prev:active -> active out of range*/ + /* prev:active -> active spanning min */ entry = mas_prev(&mas, 0x1600); MT_BUG_ON(mt, entry != NULL); MT_BUG_ON(mt, mas.index != 0x1501); MT_BUG_ON(mt, mas.last != 0x1FFF); MT_BUG_ON(mt, !mas_active(mas)); - /* prev: active ->active, continue*/ + /* prev: active ->active, continue */ entry = mas_prev(&mas, 0); MT_BUG_ON(mt, entry != ptr); MT_BUG_ON(mt, mas.index != 0x1000); @@ -3379,7 +3437,7 @@ static noinline void __init check_state_handling(struct maple_tree *mt) MT_BUG_ON(mt, mas.last != 0x2FFF); MT_BUG_ON(mt, !mas_active(mas)); - /* find: none ->active */ + /* find: overflow ->active */ entry = mas_find(&mas, 0x5000); MT_BUG_ON(mt, entry != ptr3); MT_BUG_ON(mt, mas.index != 0x3000); @@ -3778,7 +3836,6 @@ static int __init maple_tree_seed(void) check_empty_area_fill(&tree); mtree_destroy(&tree); - mt_init_flags(&tree, MT_FLAGS_ALLOC_RANGE); check_state_handling(&tree); mtree_destroy(&tree); -- 2.40.1

1 year, 7 months

1
0
0 0

[PATCH v2 1/2] maple_tree: Add mas_is_active() to detect in-tree walks

by Liam R. Howlett

Instead of constantly checking each possibility of the maple state, create a fast path that will skip over checking unlikely states. Cc: stable(a)vger.kernel.org Signed-off-by: Liam R. Howlett <Liam.Howlett(a)oracle.com> --- include/linux/maple_tree.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/linux/maple_tree.h b/include/linux/maple_tree.h index e41c70ac7744..f66f5f78f8cf 100644 --- a/include/linux/maple_tree.h +++ b/include/linux/maple_tree.h @@ -511,6 +511,15 @@ static inline bool mas_is_paused(const struct ma_state *mas) return mas->node == MAS_PAUSE; } +/* Check if the mas is pointing to a node or not */ +static inline bool mas_is_active(struct ma_state *mas) +{ + if ((unsigned long)mas->node >= MAPLE_RESERVED_RANGE) + return true; + + return false; +} + /** * mas_reset() - Reset a Maple Tree operation state. * @mas: Maple Tree operation state. -- 2.40.1

1 year, 7 months

1
0
0 0

[PATCH 1/3] mtd: rawnand: marvell: Ensure program page operations are successful

by Miquel Raynal

The NAND core complies with the ONFI specification, which itself mentions that after any program or erase operation, a status check should be performed to see whether the operation was finished *and* successful. The NAND core offers helpers to finish a page write (sending the "PAGE PROG" command, waiting for the NAND chip to be ready again, and checking the operation status). But in some cases, advanced controller drivers might want to optimize this and craft their own page write helper to leverage additional hardware capabilities, thus not always using the core facilities. Some drivers, like this one, do not use the core helper to finish a page write because the final cycles are automatically managed by the hardware. In this case, the additional care must be taken to manually perform the final status check. Let's read the NAND chip status at the end of the page write helper and return -EIO upon error. Cc: stable(a)vger.kernel.org Fixes: 02f26ecf8c77 ("mtd: nand: add reworked Marvell NAND controller driver") Reported-by: Aviram Dali <aviramd(a)marvell.com> Signed-off-by: Miquel Raynal <miquel.raynal(a)bootlin.com> --- Hello Aviram, I have not tested this, but based on your report I believe the status check is indeed missing here and could sometimes lead to unnoticed partial writes. Please test on your side and reply with your Tested-by if you validate the change. Any backport on kernels predating v4.17 will likely fail because of a folder rename, so you will have to do the backport manually if needed. Thanks, Miquèl --- drivers/mtd/nand/raw/marvell_nand.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/drivers/mtd/nand/raw/marvell_nand.c b/drivers/mtd/nand/raw/marvell_nand.c index 30c15e4e1cc0..576441095012 100644 --- a/drivers/mtd/nand/raw/marvell_nand.c +++ b/drivers/mtd/nand/raw/marvell_nand.c @@ -1162,6 +1162,7 @@ static int marvell_nfc_hw_ecc_hmg_do_write_page(struct nand_chip *chip, .ndcb[2] = NDCB2_ADDR5_PAGE(page), }; unsigned int oob_bytes = lt->spare_bytes + (raw ? lt->ecc_bytes : 0); + u8 status; int ret; /* NFCv2 needs more information about the operation being executed */ @@ -1195,7 +1196,18 @@ static int marvell_nfc_hw_ecc_hmg_do_write_page(struct nand_chip *chip, ret = marvell_nfc_wait_op(chip, PSEC_TO_MSEC(sdr->tPROG_max)); - return ret; + if (ret) + return ret; + + /* Check write status on the chip side */ + ret = nand_status_op(chip, &status); + if (ret) + return ret; + + if (status & NAND_STATUS_FAIL) + return -EIO; + + return 0; } static int marvell_nfc_hw_ecc_hmg_write_page_raw(struct nand_chip *chip, @@ -1624,6 +1636,7 @@ static int marvell_nfc_hw_ecc_bch_write_page(struct nand_chip *chip, int data_len = lt->data_bytes; int spare_len = lt->spare_bytes; int chunk, ret; + u8 status; marvell_nfc_select_target(chip, chip->cur_cs); @@ -1660,6 +1673,14 @@ static int marvell_nfc_hw_ecc_bch_write_page(struct nand_chip *chip, if (ret) return ret; + /* Check write status on the chip side */ + ret = nand_status_op(chip, &status); + if (ret) + return ret; + + if (status & NAND_STATUS_FAIL) + return -EIO; + return 0; } -- 2.34.1

1 year, 7 months

3
14
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-stable-mirror September 2023