Centralizes the definition of _GNU_SOURCE into lib.mk and addresses all
resulting macro redefinition warnings.
The initial attempt at this patch was abandoned because it affected
lines in many source files and caused a large amount of churn. However,
from earlier discussions, centralizing _GNU_SOURCE is still desireable.
This attempt limits the changes to 1 source file and 14 Makefiles.
This is condensed into a single commit to avoid redefinition warnings
from partial merges.
v1: https://lore.kernel.org/linux-kselftest/20240430235057.1351993-1-edliaw@goo…
v2: https://lore.kernel.org/linux-kselftest/20240507214254.2787305-1-edliaw@goo…
- Add -D_GNU_SOURCE to KHDR_INCLUDES so that it is in a single
location.
- Remove #define _GNU_SOURCE from source code to resolve redefinition
warnings.
v3: https://lore.kernel.org/linux-kselftest/20240509200022.253089-1-edliaw@goog…
- Rebase onto linux-next 20240508.
- Split patches by directory.
- Add -D_GNU_SOURCE directly to CFLAGS in lib.mk.
- Delete additional _GNU_SOURCE definitions from source code in
linux-next.
- Delete additional -D_GNU_SOURCE flags from Makefiles.
v4: https://lore.kernel.org/linux-kselftest/20240510000842.410729-1-edliaw@goog…
- Rebase onto linux-next 20240509.
- Remove Fixes tag from patches that drop _GNU_SOURCE definition.
- Restore space between comment and includes for selftests/damon.
v5: https://lore.kernel.org/linux-kselftest/20240522005913.3540131-1-edliaw@goo…
- Rebase onto linux-next 20240521
- Drop initial patches that modify KHDR_INCLUDES.
- Incorporate Mark Brown's patch to replace static_assert with warning.
- Don't drop #define _GNU_SOURCE from nolibc and wireguard.
- Change Makefiles for x86 and vDSO to append to CFLAGS.
v6: https://lore.kernel.org/linux-kselftest/20240624232718.1154427-1-edliaw@goo…
- Rewrite patch to use -D_GNU_SOURCE= form in lib.mk.
- Reduce the amount of churn significantly by allowing definition to
coexist with source code macro defines.
v7:
- Squash patch into a single commit.
Edward Liaw (1):
selftests: Centralize -D_GNU_SOURCE= to CFLAGS in lib.mk
tools/testing/selftests/exec/Makefile | 1 -
tools/testing/selftests/futex/functional/Makefile | 2 +-
tools/testing/selftests/intel_pstate/Makefile | 2 +-
tools/testing/selftests/iommu/Makefile | 2 --
tools/testing/selftests/kvm/Makefile | 2 +-
tools/testing/selftests/lib.mk | 3 +++
tools/testing/selftests/mm/thuge-gen.c | 2 +-
tools/testing/selftests/net/Makefile | 2 +-
tools/testing/selftests/net/tcp_ao/Makefile | 2 +-
tools/testing/selftests/proc/Makefile | 1 -
tools/testing/selftests/resctrl/Makefile | 2 +-
tools/testing/selftests/ring-buffer/Makefile | 1 -
tools/testing/selftests/riscv/mm/Makefile | 2 +-
tools/testing/selftests/sgx/Makefile | 2 +-
tools/testing/selftests/tmpfs/Makefile | 1 -
15 files changed, 12 insertions(+), 15 deletions(-)
--
2.45.2.803.g4e1b14247a-goog
From: Adrian Moreno <amorenoz(a)redhat.com>
[ Upstream commit a8763466669d21b570b26160d0a5e0a2ee529d22 ]
Netlink flags, although they don't have payload at the netlink level,
are represented as having "True" as value in pyroute2.
Without it, trying to add a flow with a flag-type action (e.g: pop_vlan)
fails with the following traceback:
Traceback (most recent call last):
File "[...]/ovs-dpctl.py", line 2498, in <module>
sys.exit(main(sys.argv))
^^^^^^^^^^^^^^
File "[...]/ovs-dpctl.py", line 2487, in main
ovsflow.add_flow(rep["dpifindex"], flow)
File "[...]/ovs-dpctl.py", line 2136, in add_flow
reply = self.nlm_request(
^^^^^^^^^^^^^^^^^
File "[...]/pyroute2/netlink/nlsocket.py", line 822, in nlm_request
return tuple(self._genlm_request(*argv, **kwarg))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "[...]/pyroute2/netlink/generic/__init__.py", line 126, in
nlm_request
return tuple(super().nlm_request(*argv, **kwarg))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "[...]/pyroute2/netlink/nlsocket.py", line 1124, in nlm_request
self.put(msg, msg_type, msg_flags, msg_seq=msg_seq)
File "[...]/pyroute2/netlink/nlsocket.py", line 389, in put
self.sendto_gate(msg, addr)
File "[...]/pyroute2/netlink/nlsocket.py", line 1056, in sendto_gate
msg.encode()
File "[...]/pyroute2/netlink/__init__.py", line 1245, in encode
offset = self.encode_nlas(offset)
^^^^^^^^^^^^^^^^^^^^^^^^
File "[...]/pyroute2/netlink/__init__.py", line 1560, in encode_nlas
nla_instance.setvalue(cell[1])
File "[...]/pyroute2/netlink/__init__.py", line 1265, in setvalue
nlv.setvalue(nla_tuple[1])
~~~~~~~~~^^^
IndexError: list index out of range
Signed-off-by: Adrian Moreno <amorenoz(a)redhat.com>
Acked-by: Aaron Conole <aconole(a)redhat.com>
Signed-off-by: David S. Miller <davem(a)davemloft.net>
Signed-off-by: Sasha Levin <sashal(a)kernel.org>
---
tools/testing/selftests/net/openvswitch/ovs-dpctl.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/net/openvswitch/ovs-dpctl.py b/tools/testing/selftests/net/openvswitch/ovs-dpctl.py
index 5e0e539a323d5..8b120718768ec 100644
--- a/tools/testing/selftests/net/openvswitch/ovs-dpctl.py
+++ b/tools/testing/selftests/net/openvswitch/ovs-dpctl.py
@@ -531,7 +531,7 @@ class ovsactions(nla):
for flat_act in parse_flat_map:
if parse_starts_block(actstr, flat_act[0], False):
actstr = actstr[len(flat_act[0]):]
- self["attrs"].append([flat_act[1]])
+ self["attrs"].append([flat_act[1], True])
actstr = actstr[strspn(actstr, ", ") :]
parsed = True
--
2.43.0
From: Adrian Moreno <amorenoz(a)redhat.com>
[ Upstream commit a8763466669d21b570b26160d0a5e0a2ee529d22 ]
Netlink flags, although they don't have payload at the netlink level,
are represented as having "True" as value in pyroute2.
Without it, trying to add a flow with a flag-type action (e.g: pop_vlan)
fails with the following traceback:
Traceback (most recent call last):
File "[...]/ovs-dpctl.py", line 2498, in <module>
sys.exit(main(sys.argv))
^^^^^^^^^^^^^^
File "[...]/ovs-dpctl.py", line 2487, in main
ovsflow.add_flow(rep["dpifindex"], flow)
File "[...]/ovs-dpctl.py", line 2136, in add_flow
reply = self.nlm_request(
^^^^^^^^^^^^^^^^^
File "[...]/pyroute2/netlink/nlsocket.py", line 822, in nlm_request
return tuple(self._genlm_request(*argv, **kwarg))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "[...]/pyroute2/netlink/generic/__init__.py", line 126, in
nlm_request
return tuple(super().nlm_request(*argv, **kwarg))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "[...]/pyroute2/netlink/nlsocket.py", line 1124, in nlm_request
self.put(msg, msg_type, msg_flags, msg_seq=msg_seq)
File "[...]/pyroute2/netlink/nlsocket.py", line 389, in put
self.sendto_gate(msg, addr)
File "[...]/pyroute2/netlink/nlsocket.py", line 1056, in sendto_gate
msg.encode()
File "[...]/pyroute2/netlink/__init__.py", line 1245, in encode
offset = self.encode_nlas(offset)
^^^^^^^^^^^^^^^^^^^^^^^^
File "[...]/pyroute2/netlink/__init__.py", line 1560, in encode_nlas
nla_instance.setvalue(cell[1])
File "[...]/pyroute2/netlink/__init__.py", line 1265, in setvalue
nlv.setvalue(nla_tuple[1])
~~~~~~~~~^^^
IndexError: list index out of range
Signed-off-by: Adrian Moreno <amorenoz(a)redhat.com>
Acked-by: Aaron Conole <aconole(a)redhat.com>
Signed-off-by: David S. Miller <davem(a)davemloft.net>
Signed-off-by: Sasha Levin <sashal(a)kernel.org>
---
tools/testing/selftests/net/openvswitch/ovs-dpctl.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/net/openvswitch/ovs-dpctl.py b/tools/testing/selftests/net/openvswitch/ovs-dpctl.py
index 5e0e539a323d5..8b120718768ec 100644
--- a/tools/testing/selftests/net/openvswitch/ovs-dpctl.py
+++ b/tools/testing/selftests/net/openvswitch/ovs-dpctl.py
@@ -531,7 +531,7 @@ class ovsactions(nla):
for flat_act in parse_flat_map:
if parse_starts_block(actstr, flat_act[0], False):
actstr = actstr[len(flat_act[0]):]
- self["attrs"].append([flat_act[1]])
+ self["attrs"].append([flat_act[1], True])
actstr = actstr[strspn(actstr, ", ") :]
parsed = True
--
2.43.0
Correctable memory errors are very common on servers with large
amount of memory, and are corrected by ECC, but with two
pain points to users:
1. Correction usually happens on the fly and adds latency overhead
2. Not-fully-proved theory states excessive correctable memory
errors can develop into uncorrectable memory error.
Soft offline is kernel's additional solution for memory pages
having (excessive) corrected memory errors. Impacted page is migrated
to healthy page if it is in use, then the original page is discarded
for any future use.
The actual policy on whether (and when) to soft offline should be
maintained by userspace, especially in case of an 1G HugeTLB page.
Soft-offline dissolves the HugeTLB page, either in-use or free, into
chunks of 4K pages, reducing HugeTLB pool capacity by 1 hugepage.
If userspace has not acknowledged such behavior, it may be surprised
when later mmap hugepages MAP_FAILED due to lack of hugepages.
In case of a transparent hugepage, it will be split into 4K pages
as well; userspace will stop enjoying the transparent performance.
In addition, discarding the entire 1G HugeTLB page only because of
corrected memory errors sounds very costly and kernel better not
doing under the hood. But today there are at least 2 such cases:
1. GHES driver sees both GHES_SEV_CORRECTED and
CPER_SEC_ERROR_THRESHOLD_EXCEEDED after parsing CPER.
2. RAS Correctable Errors Collector counts correctable errors per
PFN and when the counter for a PFN reaches threshold
In both cases, userspace has no control of the soft offline performed
by kernel's memory failure recovery.
This patch series give userspace the control of softofflining any page:
kernel only soft offlines raw page / transparent hugepage / HugeTLB
hugepage if userspace has agreed to. The interface to userspace is a
new sysctl called enable_soft_offline under /proc/sys/vm. By default
enable_soft_line is 1 to preserve existing behavior in kernel.
Changelog
v6 => v7
* incorporate feedbacks from Miaohe Lin <linmiaohe(a)huawei.com> and David
Rientjes <rientjes(a)google.com>
* remove PFN value from enable_soft_offline log
* save/restore enable_soft_offline in run_vmtests.sh
* v7 is based on commit 7c89bdbd3778 ("khugepaged: simplify the
allocation of slab caches")
v5 => v6:
* incorporate feedbacks from Miaohe Lin <linmiaohe(a)huawei.com>
* add a ':' in soft offline log.
* close hugetlbfs file descriptor in selftest.
* no need to "return" after ksft_exit_fail_msg.
v4 => v5:
* incorporate feedbacks from Muhammad Usama Anjum
<usama.anjum(a)collabora.com>
* refactor selftest to use what available in kselftest.h
v3 => v4:
* incorporate feedbacks from Miaohe Lin <linmiaohe(a)huawei.com>,
Andrew Morton <akpm(a)linux-foundation.org>, and
Oscar Salvador <osalvador(a)suse.de>.
* insert a refactor commit to unify soft offline's logs to follow
"Soft offline: 0x${pfn}: ${message}" format.
* some rewords in document: fail => will not perform.
* v4 is still based on commit 83a7eefedc9b ("Linux 6.10-rc3"),
akpm/mm-stable.
v2 => v3:
* incorporate feedbacks from Miaohe Lin <linmiaohe(a)huawei.com>,
Lance Yang <ioworker0(a)gmail.com>, Oscar Salvador <osalvador(a)suse.de>,
and David Rientjes <rientjes(a)google.com>.
* release potential refcount if enable_soft_offline is 0.
* soft_offline_page() returns EOPNOTSUPP if enable_soft_offline is 0.
* refactor hugetlb-soft-offline.c, for example, introduce
test_soft_offline_common to reduce repeated code.
* rewrite enable_soft_offline's documentation, adds more details about
the cost of soft-offline for transparent and hugetlb hugepages, and
components that are impacted when enable_soft_offline becomes 0.
* fix typos in commit messages.
* v3 is still based on commit 83a7eefedc9b ("Linux 6.10-rc3").
v1 => v2:
* incorporate feedbacks from both Miaohe Lin <linmiaohe(a)huawei.com> and
Jane Chu <jane.chu(a)oracle.com>.
* make the switch to control all pages, instead of HugeTLB specific.
* change the API from
/sys/kernel/mm/hugepages/hugepages-${size}kB/softoffline_corrected_errors
to /proc/sys/vm/enable_soft_offline.
* minor update to test code.
* update documentation of the user control API.
* v2 is based on commit 83a7eefedc9b ("Linux 6.10-rc3").
Jiaqi Yan (4):
mm/memory-failure: refactor log format in soft offline code
mm/memory-failure: userspace controls soft-offlining pages
selftest/mm: test enable_soft_offline behaviors
docs: mm: add enable_soft_offline sysctl
Documentation/admin-guide/sysctl/vm.rst | 32 +++
mm/memory-failure.c | 37 ++-
tools/testing/selftests/mm/.gitignore | 1 +
tools/testing/selftests/mm/Makefile | 1 +
.../selftests/mm/hugetlb-soft-offline.c | 228 ++++++++++++++++++
tools/testing/selftests/mm/run_vmtests.sh | 6 +
6 files changed, 297 insertions(+), 8 deletions(-)
create mode 100644 tools/testing/selftests/mm/hugetlb-soft-offline.c
--
2.45.2.803.g4e1b14247a-goog
Currently, if a user wants to run pmtu.sh and cover all the provided test
cases, they need to install the Open vSwitch userspace utilities. This
dependency is difficult for users as well as CI environments, because the
userspace build and setup may require lots of support and devel packages
to be installed, system setup to be correct, and things like permissions
and selinux policies to be properly configured.
The kernel selftest suite includes an ovs-dpctl.py utility which can
interact with the openvswitch module directly. This lets developers and
CI environments run without needing too many extra dependencies - just
the pyroute2 python package.
This series enhances the ovs-dpctl utility to provide support for set()
and tunnel() flow specifiers, better ipv6 handling support, and the
ability to add tunnel vports, and LWT interfaces. Finally, it modifies
the pmtu.sh script to call the ovs-dpctl.py utility rather than the
typical OVS userspace utilities. The pmtu.sh can still fall back on
the Open vSwitch userspace utilities if the ovs-dpctl.py script can't
be used.
Aaron Conole (7):
selftests: openvswitch: Support explicit tunnel port creation.
selftests: openvswitch: Refactor actions parsing.
selftests: openvswitch: Add set() and set_masked() support.
selftests: openvswitch: Add support for tunnel() key.
selftests: openvswitch: Support implicit ipv6 arguments.
selftests: net: Use the provided dpctl rather than the vswitchd for
tests.
selftests: net: add config for openvswitch
tools/testing/selftests/net/config | 5 +
.../selftests/net/openvswitch/ovs-dpctl.py | 368 +++++++++++++++---
tools/testing/selftests/net/pmtu.sh | 145 +++++--
3 files changed, 451 insertions(+), 67 deletions(-)
--
2.45.1