* Resending because I accidentally forgot to include Lorenzo in the
"to" list.
Android uses the ashmem driver [1] for creating shared memory regions
between processes. The ashmem driver exposes an ioctl command for
processes to restrict the permissions an ashmem buffer can be mapped
with.
Buffers are created with the ability to be mapped as readable, writable,
and executable. Processes remove the ability to map some ashmem buffers
as executable to ensure that those buffers cannot be used to inject
malicious code for another process to run. Other buffers retain their
ability to be mapped as executable, as these buffers can be used for
just-in-time (JIT) compilation. So there is a need to be able to remove
the ability to map a buffer as executable on a per-buffer basis.
Android is currently trying to migrate towards replacing its ashmem
driver usage with memfd. Part of the transition involved introducing a
library that serves to abstract away how shared memory regions are
allocated (i.e. ashmem vs memfd). This allows clients to use a single
interface for restricting how a buffer can be mapped without having to
worry about how it is handled for ashmem (through the ioctl
command mentioned earlier) or memfd (through file seals).
While memfd has support for preventing buffers from being mapped as
writable beyond a certain point in time (thanks to
F_SEAL_FUTURE_WRITE), it does not have a similar interface to prevent
buffers from being mapped as executable beyond a certain point.
However, that could be implemented as a file seal (F_SEAL_FUTURE_EXEC)
which works similarly to F_SEAL_FUTURE_WRITE.
F_SEAL_FUTURE_WRITE was chosen as a template for how this new seal
should behave, instead of F_SEAL_WRITE, for the following reasons:
1. Having the new seal behave like F_SEAL_FUTURE_WRITE matches the
behavior that was present with ashmem. This aids in seamlessly
transitioning clients away from ashmem to memfd.
2. Making the new seal behave like F_SEAL_WRITE would mean that no
mappings that could become executable in the future (i.e. via
mprotect()) can exist when the seal is applied. However, there are
known cases (e.g. CursorWindow [2]) where restrictions are applied
on how a buffer can be mapped after a mapping has already been made.
That mapping may have VM_MAYEXEC set, which would not allow the seal
to be applied successfully.
Therefore, the F_SEAL_FUTURE_EXEC seal was designed to have the same
semantics as F_SEAL_FUTURE_WRITE.
Note: this series depends on Lorenzo's work [3], [4], [5] from Andrew
Morton's mm-unstable branch [6], which reworks memfd's file seal checks,
allowing for newer file seals to be implemented in a cleaner fashion.
Changes from v1 ==> v2:
- Changed the return code to be -EPERM instead of -EACCES when
attempting to map an exec sealed file with PROT_EXEC to align
to mmap()'s man page. Thank you Kalesh Singh for spotting this!
- Rebased on top of Lorenzo's work to cleanup memfd file seal checks in
mmap() ([3], [4], and [5]). Thank you for this Lorenzo!
- Changed to deny PROT_EXEC mappings only if the mapping is shared,
instead of for both shared and private mappings, after discussing
this with Lorenzo.
Opens:
- Lorenzo brought up that this patch may negatively impact the usage of
MFD_NOEXEC_SCOPE_NOEXEC_ENFORCED [7]. However, it is not clear to me
why that is the case. At the moment, my intent is for the executable
permissions of the file to be disjoint from the ability to create
executable mappings.
Links:
[1] https://cs.android.com/android/kernel/superproject/+/common-android-mainlin…
[2] https://developer.android.com/reference/android/database/CursorWindow
[3] https://lore.kernel.org/all/cover.1732804776.git.lorenzo.stoakes@oracle.com/
[4] https://lkml.kernel.org/r/20241206212846.210835-1-lorenzo.stoakes@oracle.com
[5] https://lkml.kernel.org/r/7dee6c5d-480b-4c24-b98e-6fa47dbd8a23@lucifer.local
[6] https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git/tree/?h=mm-unst…
[7] https://lore.kernel.org/all/3a53b154-1e46-45fb-a559-65afa7a8a788@lucifer.lo…
Links to previous versions:
v1: https://lore.kernel.org/all/20241206010930.3871336-1-isaacmanjarres@google.…
Isaac J. Manjarres (2):
mm/memfd: Add support for F_SEAL_FUTURE_EXEC to memfd
selftests/memfd: Add tests for F_SEAL_FUTURE_EXEC
include/uapi/linux/fcntl.h | 1 +
mm/memfd.c | 39 ++++++++++-
tools/testing/selftests/memfd/memfd_test.c | 79 ++++++++++++++++++++++
3 files changed, 118 insertions(+), 1 deletion(-)
--
2.47.1.613.gc27f4b7a9f-goog
When compiling the pointer masking tests with -Wall this warning
is present:
pointer_masking.c: In function ‘test_tagged_addr_abi_sysctl’:
pointer_masking.c:203:9: warning: ignoring return value of ‘pwrite’
declared with attribute ‘warn_unused_result’ [-Wunused-result]
203 | pwrite(fd, &value, 1, 0); |
^~~~~~~~~~~~~~~~~~~~~~~~ pointer_masking.c:208:9: warning:
ignoring return value of ‘pwrite’ declared with attribute
‘warn_unused_result’ [-Wunused-result]
208 | pwrite(fd, &value, 1, 0);
I came across this on riscv64-linux-gnu-gcc (Ubuntu
11.4.0-1ubuntu1~22.04).
Fix this by checking that the number of bytes written equal the expected
number of bytes written.
Fixes: 7470b5afd150 ("riscv: selftests: Add a pointer masking test")
Signed-off-by: Charlie Jenkins <charlie(a)rivosinc.com>
Reviewed-by: Andrew Jones <ajones(a)ventanamicro.com>
---
Changes in v6:
- Add back ksft_test_result() (Samuel)
- Link to v5: https://lore.kernel.org/r/20241206-fix_warnings_pointer_masking_tests-v5-1-…
Changes in v5:
- No longer skip second pwrite if first one fails
- Use wrapper function instead of goto (Drew)
- Link to v4: https://lore.kernel.org/r/20241205-fix_warnings_pointer_masking_tests-v4-1-…
Changes in v4:
- Skip sysctl_enabled test if first pwrite failed
- Link to v3: https://lore.kernel.org/r/20241205-fix_warnings_pointer_masking_tests-v3-1-…
Changes in v3:
- Fix sysctl enabled test case (Drew/Alex)
- Move pwrite err condition into goto (Drew)
- Link to v2: https://lore.kernel.org/r/20241204-fix_warnings_pointer_masking_tests-v2-1-…
Changes in v2:
- I had ret != 2 for testing, I changed it to be ret != 1.
- Link to v1: https://lore.kernel.org/r/20241204-fix_warnings_pointer_masking_tests-v1-1-…
---
.../testing/selftests/riscv/abi/pointer_masking.c | 28 +++++++++++++++++-----
1 file changed, 22 insertions(+), 6 deletions(-)
diff --git a/tools/testing/selftests/riscv/abi/pointer_masking.c b/tools/testing/selftests/riscv/abi/pointer_masking.c
index dee41b7ee3e323150d55523c8acbf3ec38857b87..059d2e87eb1f737caf44f692b239bf3e49c233b4 100644
--- a/tools/testing/selftests/riscv/abi/pointer_masking.c
+++ b/tools/testing/selftests/riscv/abi/pointer_masking.c
@@ -185,8 +185,20 @@ static void test_fork_exec(void)
}
}
+static bool pwrite_wrapper(int fd, void *buf, size_t count, const char *msg)
+{
+ int ret = pwrite(fd, buf, count, 0);
+
+ if (ret != count) {
+ ksft_perror(msg);
+ return false;
+ }
+ return true;
+}
+
static void test_tagged_addr_abi_sysctl(void)
{
+ char *err_pwrite_msg = "failed to write to /proc/sys/abi/tagged_addr_disabled\n";
char value;
int fd;
@@ -200,14 +212,18 @@ static void test_tagged_addr_abi_sysctl(void)
}
value = '1';
- pwrite(fd, &value, 1, 0);
- ksft_test_result(set_tagged_addr_ctrl(min_pmlen, true) == -EINVAL,
- "sysctl disabled\n");
+ if (!pwrite_wrapper(fd, &value, 1, "write '1'"))
+ ksft_test_result_fail(err_pwrite_msg);
+ else
+ ksft_test_result(set_tagged_addr_ctrl(min_pmlen, true) == -EINVAL,
+ "sysctl disabled\n");
value = '0';
- pwrite(fd, &value, 1, 0);
- ksft_test_result(set_tagged_addr_ctrl(min_pmlen, true) == 0,
- "sysctl enabled\n");
+ if (!pwrite_wrapper(fd, &value, 1, "write '0'"))
+ ksft_test_result_fail(err_pwrite_msg);
+ else
+ ksft_test_result(set_tagged_addr_ctrl(min_pmlen, true) == 0,
+ "sysctl enabled\n");
set_tagged_addr_ctrl(0, false);
---
base-commit: 40384c840ea1944d7c5a392e8975ed088ecf0b37
change-id: 20241204-fix_warnings_pointer_masking_tests-3860e4f35429
--
- Charlie
This patch series includes some netns-related improvements and fixes for
rtnetlink, to make link creation more intuitive:
1) Creating link in another net namespace doesn't conflict with link
names in current one.
2) Refector rtnetlink link creation. Create link in target namespace
directly.
So that
# ip link add netns ns1 link-netns ns2 tun0 type gre ...
will create tun0 in ns1, rather than create it in ns2 and move to ns1.
And don't conflict with another interface named "tun0" in current netns.
Patch 01 serves for 1) to avoids link name conflict in different netns.
To achieve 2), there're mainly 3 steps:
- Patch 02 packs newlink() parameters into a struct, including
the original "src_net" along with more netns context. No semantic
changes are introduced.
- Patch 03 ~ 07 converts device drivers to use the explicit netns
extracted from params.
- Patch 08 ~ 09 removes the old netns parameter, and converts
rtnetlink to create device in target netns directly.
Patch 10 ~ 11 adds some tests for link name and link netns.
BTW please note there're some issues found in current code:
- In amt_newlink() drivers/net/amt.c:
amt->net = net;
...
amt->stream_dev = dev_get_by_index(net, ...
Uses net, but amt_lookup_upper_dev() only searches in dev_net.
So the AMT device may not be properly deleted if it's in a different
netns from lower dev.
- In gtp_newlink() in drivers/net/gtp.c:
gtp->net = src_net;
...
gn = net_generic(dev_net(dev), gtp_net_id);
list_add_rcu(>p->list, &gn->gtp_dev_list);
Uses src_net, but priv is linked to list in dev_net. So it may not be
properly deleted on removal of link netns.
- In pfcp_newlink() in drivers/net/pfcp.c:
pfcp->net = net;
...
pn = net_generic(dev_net(dev), pfcp_net_id);
list_add_rcu(&pfcp->list, &pn->pfcp_dev_list);
Same as above.
- In lowpan_newlink() in net/ieee802154/6lowpan/core.c:
wdev = dev_get_by_index(dev_net(ldev), nla_get_u32(tb[IFLA_LINK]));
Looks for IFLA_LINK in dev_net, but in theory the ifindex is defined
in link netns.
---
v7:
- Add selftest kconfig.
- Remove a duplicated test of ip6gre.
v6:
link: https://lore.kernel.org/all/20241218130909.2173-1-shaw.leon@gmail.com/
- Split prototype, driver and rtnetlink changes.
- Add more tests for link netns.
- Fix IPv6 tunnel net overwriten in ndo_init().
- Reorder variable declarations.
- Exclude a ip_tunnel-specific patch.
v5:
link: https://lore.kernel.org/all/20241209140151.231257-1-shaw.leon@gmail.com/
- Fix function doc in batman-adv.
- Include peer_net in rtnl newlink parameters.
v4:
link: https://lore.kernel.org/all/20241118143244.1773-1-shaw.leon@gmail.com/
- Pack newlink() parameters to a single struct.
- Use ynl async_msg_queue.empty() in selftest.
v3:
link: https://lore.kernel.org/all/20241113125715.150201-1-shaw.leon@gmail.com/
- Drop "netns_atomic" flag and module parameter. Add netns parameter to
newlink() instead, and convert drivers accordingly.
- Move python NetNSEnter helper to net selftest lib.
v2:
link: https://lore.kernel.org/all/20241107133004.7469-1-shaw.leon@gmail.com/
- Check NLM_F_EXCL to ensure only link creation is affected.
- Add self tests for link name/ifindex conflict and notifications
in different netns.
- Changes in dummy driver and ynl in order to add the test case.
v1:
link: https://lore.kernel.org/all/20241023023146.372653-1-shaw.leon@gmail.com/
Xiao Liang (11):
rtnetlink: Lookup device in target netns when creating link
rtnetlink: Pack newlink() params into struct
net: Use link netns in newlink() of rtnl_link_ops
ieee802154: 6lowpan: Use link netns in newlink() of rtnl_link_ops
net: ip_tunnel: Use link netns in newlink() of rtnl_link_ops
net: ipv6: Use link netns in newlink() of rtnl_link_ops
net: xfrm: Use link netns in newlink() of rtnl_link_ops
rtnetlink: Remove "net" from newlink params
rtnetlink: Create link directly in target net namespace
selftests: net: Add python context manager for netns entering
selftests: net: Add test cases for link and peer netns
drivers/infiniband/ulp/ipoib/ipoib_netlink.c | 11 +-
drivers/net/amt.c | 16 +-
drivers/net/bareudp.c | 11 +-
drivers/net/bonding/bond_netlink.c | 8 +-
drivers/net/can/dev/netlink.c | 4 +-
drivers/net/can/vxcan.c | 9 +-
.../ethernet/qualcomm/rmnet/rmnet_config.c | 11 +-
drivers/net/geneve.c | 11 +-
drivers/net/gtp.c | 9 +-
drivers/net/ipvlan/ipvlan.h | 4 +-
drivers/net/ipvlan/ipvlan_main.c | 15 +-
drivers/net/ipvlan/ipvtap.c | 10 +-
drivers/net/macsec.c | 15 +-
drivers/net/macvlan.c | 8 +-
drivers/net/macvtap.c | 11 +-
drivers/net/netkit.c | 9 +-
drivers/net/pfcp.c | 11 +-
drivers/net/ppp/ppp_generic.c | 10 +-
drivers/net/team/team_core.c | 7 +-
drivers/net/veth.c | 9 +-
drivers/net/vrf.c | 11 +-
drivers/net/vxlan/vxlan_core.c | 11 +-
drivers/net/wireguard/device.c | 11 +-
drivers/net/wireless/virtual/virt_wifi.c | 14 +-
drivers/net/wwan/wwan_core.c | 25 +++-
include/net/ip_tunnels.h | 5 +-
include/net/rtnetlink.h | 44 +++++-
net/8021q/vlan_netlink.c | 15 +-
net/batman-adv/soft-interface.c | 16 +-
net/bridge/br_netlink.c | 12 +-
net/caif/chnl_net.c | 6 +-
net/core/rtnetlink.c | 35 +++--
net/hsr/hsr_netlink.c | 14 +-
net/ieee802154/6lowpan/core.c | 9 +-
net/ipv4/ip_gre.c | 27 ++--
net/ipv4/ip_tunnel.c | 10 +-
net/ipv4/ip_vti.c | 10 +-
net/ipv4/ipip.c | 14 +-
net/ipv6/ip6_gre.c | 42 ++++--
net/ipv6/ip6_tunnel.c | 20 ++-
net/ipv6/ip6_vti.c | 16 +-
net/ipv6/sit.c | 18 ++-
net/xfrm/xfrm_interface_core.c | 15 +-
tools/testing/selftests/net/Makefile | 1 +
tools/testing/selftests/net/config | 5 +
.../testing/selftests/net/lib/py/__init__.py | 2 +-
tools/testing/selftests/net/lib/py/netns.py | 18 +++
tools/testing/selftests/net/link_netns.py | 141 ++++++++++++++++++
tools/testing/selftests/net/netns-name.sh | 10 ++
49 files changed, 550 insertions(+), 226 deletions(-)
create mode 100755 tools/testing/selftests/net/link_netns.py
--
2.47.1
When I implemented virtio's hash-related features to tun/tap [1],
I found tun/tap does not fill the entire region reserved for the virtio
header, leaving some uninitialized hole in the middle of the buffer
after read()/recvmesg().
This series fills the uninitialized hole. More concretely, the
num_buffers field will be initialized with 1, and the other fields will
be inialized with 0. Setting the num_buffers field to 1 is mandated by
virtio 1.0 [2].
The change to virtio header is preceded by another change that refactors
tun and tap to unify their virtio-related code.
[1]: https://lore.kernel.org/r/20241008-rss-v5-0-f3cf68df005d@daynix.com
[2]: https://lore.kernel.org/r/20241227084256-mutt-send-email-mst@kernel.org/
Signed-off-by: Akihiko Odaki <akihiko.odaki(a)daynix.com>
---
Akihiko Odaki (3):
tun: Unify vnet implementation
tun: Pad virtio header with zero
tun: Set num_buffers for virtio 1.0
MAINTAINERS | 1 +
drivers/net/Kconfig | 5 ++
drivers/net/Makefile | 1 +
drivers/net/tap.c | 174 ++++++----------------------------------
drivers/net/tun.c | 212 ++++++++-----------------------------------------
drivers/net/tun_vnet.c | 191 ++++++++++++++++++++++++++++++++++++++++++++
drivers/net/tun_vnet.h | 24 ++++++
7 files changed, 281 insertions(+), 327 deletions(-)
---
base-commit: a32e14f8aef69b42826cf0998b068a43d486a9e9
change-id: 20241230-tun-66e10a49b0c7
Best regards,
--
Akihiko Odaki <akihiko.odaki(a)daynix.com>
Implement comprehensive testing for netconsole userdata entry handling,
demonstrating correct behavior when creating maximum entries and
preventing unauthorized overflow.
Refactor existing test infrastructure to support modular, reusable
helper functions that validate strict entry limit enforcement.
Also, add a warning if update_userdata() sees more than
MAX_USERDATA_ITEMS entries. This shouldn't happen and it is a bug that
shouldn't be silently ignored.
Signed-off-by: Breno Leitao <leitao(a)debian.org>
---
Changes in v2:
- Add the new script (netcons_overflow.sh) in
tools/testing/selftests/drivers/net/Makefile as suggested by Simon
Horman
- Link to v1: https://lore.kernel.org/r/20241204-netcons_overflow_test-v1-0-a85a8d0ace21@…
---
Breno Leitao (4):
netconsole: Warn if MAX_USERDATA_ITEMS limit is exceeded
netconsole: selftest: Split the helpers from the selftest
netconsole: selftest: Delete all userdata keys
netconsole: selftest: verify userdata entry limit
MAINTAINERS | 3 +-
drivers/net/netconsole.c | 2 +-
tools/testing/selftests/drivers/net/Makefile | 1 +
.../selftests/drivers/net/lib/sh/lib_netcons.sh | 225 +++++++++++++++++++++
.../testing/selftests/drivers/net/netcons_basic.sh | 218 +-------------------
.../selftests/drivers/net/netcons_overflow.sh | 67 ++++++
6 files changed, 297 insertions(+), 219 deletions(-)
---
base-commit: 94c16fd4df9089931f674fb9aaec41ea20b0fd7a
change-id: 20241204-netcons_overflow_test-eaf735d1f743
Best regards,
--
Breno Leitao <leitao(a)debian.org>
After commit b1f202060afe ("mm: remap unused subpages to shared zeropage
when splitting isolated thp"), cow test cases involving swapping out
THPs via madvise(MADV_PAGEOUT) started to be skipped due to the
subsequent check via pagemap determining that the memory was not
actually swapped out. Logs similar to this were emitted:
...
# [RUN] Basic COW after fork() ... with swapped-out, PTE-mapped THP (16 kB)
ok 2 # SKIP MADV_PAGEOUT did not work, is swap enabled?
# [RUN] Basic COW after fork() ... with single PTE of swapped-out THP (16 kB)
ok 3 # SKIP MADV_PAGEOUT did not work, is swap enabled?
# [RUN] Basic COW after fork() ... with swapped-out, PTE-mapped THP (32 kB)
ok 4 # SKIP MADV_PAGEOUT did not work, is swap enabled?
...
The commit in question introduces the behaviour of scanning THPs and if
their content is predominantly zero, it splits them and replaces the
pages which are wholly zero with the zero page. These cow test cases
were getting caught up in this.
So let's avoid that by filling the contents of all allocated memory with
a non-zero value. With this in place, the tests are passing again.
Signed-off-by: Ryan Roberts <ryan.roberts(a)arm.com>
---
Applies on top of mm-unstable (f349e79bfbf3)
Thanks,
Ryan
tools/testing/selftests/mm/cow.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/tools/testing/selftests/mm/cow.c b/tools/testing/selftests/mm/cow.c
index 32c6ccc2a6be..1238e1c5aae1 100644
--- a/tools/testing/selftests/mm/cow.c
+++ b/tools/testing/selftests/mm/cow.c
@@ -758,7 +758,7 @@ static void do_run_with_base_page(test_fn fn, bool swapout)
}
/* Populate a base page. */
- memset(mem, 0, pagesize);
+ memset(mem, 1, pagesize);
if (swapout) {
madvise(mem, pagesize, MADV_PAGEOUT);
@@ -824,12 +824,12 @@ static void do_run_with_thp(test_fn fn, enum thp_run thp_run, size_t thpsize)
* Try to populate a THP. Touch the first sub-page and test if
* we get the last sub-page populated automatically.
*/
- mem[0] = 0;
+ mem[0] = 1;
if (!pagemap_is_populated(pagemap_fd, mem + thpsize - pagesize)) {
ksft_test_result_skip("Did not get a THP populated\n");
goto munmap;
}
- memset(mem, 0, thpsize);
+ memset(mem, 1, thpsize);
size = thpsize;
switch (thp_run) {
@@ -1012,7 +1012,7 @@ static void run_with_hugetlb(test_fn fn, const char *desc, size_t hugetlbsize)
}
/* Populate an huge page. */
- memset(mem, 0, hugetlbsize);
+ memset(mem, 1, hugetlbsize);
/*
* We need a total of two hugetlb pages to handle COW/unsharing
--
2.43.0
On Wed, Jan 08, 2025 at 11:39:40AM +0530, Dev Jain wrote:
>
> On 07/01/25 8:44 pm, Thomas Weißschuh wrote:
> > During the execution of validate_complete_va_space() a lot of memory is
> > on the VM subsystem. When running on a low memory subsystem an OOM may
> > be triggered, when writing to the dump file as the filesystem may also
> > require memory.
> >
> > On my test system with 1100MiB physical memory:
> >
> > Tasks state (memory values in pages):
> > [ pid ] uid tgid total_vm rss rss_anon rss_file rss_shmem pgtables_bytes swapents oom_score_adj name
> > [ 57] 0 57 34359215953 695 256 0 439 1064390656 0 0 virtual_address
> >
> > Out of memory: Killed process 57 (virtual_address) total-vm:137436863812kB, anon-rss:1024kB, file-rss:0kB, shmem-rss:1756kB, UID:0 pgtables:1039444kB oom_score_adj:0
> > <snip>
> > fault_in_iov_iter_readable+0x4a/0xd0
> > generic_perform_write+0x9c/0x280
> > shmem_file_write_iter+0x86/0x90
> > vfs_write+0x29c/0x480
> > ksys_write+0x6c/0xe0
> > do_syscall_64+0x9e/0x1a0
> > entry_SYSCALL_64_after_hwframe+0x77/0x7f
> >
> > Write the dumped data into /dev/null instead which does not require
> > additional memory during write(), making the code simpler as a
> > side-effect.
> >
> > Signed-off-by: Thomas Weißschuh<thomas.weissschuh(a)linutronix.de>
> > ---
> > tools/testing/selftests/mm/virtual_address_range.c | 6 ++----
> > 1 file changed, 2 insertions(+), 4 deletions(-)
> >
> > diff --git a/tools/testing/selftests/mm/virtual_address_range.c b/tools/testing/selftests/mm/virtual_address_range.c
> > index 484f82c7b7c871f82a7d9ec6d6c649f2ab1eb0cd..4042fd878acd702d23da2c3293292de33bd48143 100644
> > --- a/tools/testing/selftests/mm/virtual_address_range.c
> > +++ b/tools/testing/selftests/mm/virtual_address_range.c
> > @@ -103,10 +103,9 @@ static int validate_complete_va_space(void)
> > FILE *file;
> > int fd;
> > - fd = open("va_dump", O_CREAT | O_WRONLY, 0600);
> > - unlink("va_dump");
> > + fd = open("/dev/null", O_WRONLY);
> > if (fd < 0) {
> > - ksft_test_result_skip("cannot create or open dump file\n");
> > + ksft_test_result_skip("cannot create or open /dev/null\n");
> > ksft_finished();
> > }
> > @@ -152,7 +151,6 @@ static int validate_complete_va_space(void)
> > while (start_addr + hop < end_addr) {
> > if (write(fd, (void *)(start_addr + hop), 1) != 1)
> > return 1;
> > - lseek(fd, 0, SEEK_SET);
> > hop += MAP_CHUNK_SIZE;
> > }
> >
>
> The reason I had not used /dev/null was that write() was succeeding to /dev/null
> even from an address not in my VA space. I was puzzled about this behaviour of
> /dev/null and I chose to ignore it and just use a real file.
That makes sense and I can reproduce your example.
Switching to another dummy file which reads the written data like
/dev/random also leads to OOM, so wouldn't help either.
Thanks for the explanation.
@Andrew, could you drop this patch?
> To test this behaviour, run the following program:
[..]
PS: Your mail contained HTML and did not make it to the list archives.
(And the text variant of the example program was corrupted)
This patch series implements a new char misc driver, /dev/ntsync, which is used
to implement Windows NT synchronization primitives.
NT synchronization primitives are unique in that the wait functions both are
vectored, operate on multiple types of object with different behaviour (mutex,
semaphore, event), and affect the state of the objects they wait on. This model
is not compatible with existing kernel synchronization objects or interfaces,
and therefore the ntsync driver implements its own wait queues and locking.
This patch series is rebased against the "char-misc-next" branch of
gregkh/char-misc.git.
== Background ==
The Wine project emulates the Windows API in user space. One particular part of
that API, namely the NT synchronization primitives, have historically been
implemented via RPC to a dedicated "kernel" process. However, more recent
applications use these APIs more strenuously, and the overhead of RPC has become
a bottleneck.
The NT synchronization APIs are too complex to implement on top of existing
primitives without sacrificing correctness. Certain operations, such as
NtPulseEvent() or the "wait-for-all" mode of NtWaitForMultipleObjects(), require
direct control over the underlying wait queue, and implementing a wait queue
sufficiently robust for Wine in user space is not possible. This proposed
driver, therefore, implements the problematic interfaces directly in the Linux
kernel.
This driver was presented at Linux Plumbers Conference 2023. For those further
interested in the history of synchronization in Wine and past attempts to solve
this problem in user space, a recording of the presentation can be viewed here:
https://www.youtube.com/watch?v=NjU4nyWyhU8
== Performance ==
The performance measurements described below are copied from earlier versions of
the patch set. While some of the code has changed, I do not currently anticipate
that it has changed drastically enough to affect those measurements.
The gain in performance varies wildly depending on the application in question
and the user's hardware. For some games NT synchronization is not a bottleneck
and no change can be observed, but for others frame rate improvements of 50 to
150 percent are not atypical. The following table lists frame rate measurements
from a variety of games on a variety of hardware, taken by users Dmitry
Skvortsov, FuzzyQuils, OnMars, and myself:
Game Upstream ntsync improvement
===========================================================================
Anger Foot 69 99 43%
Call of Juarez 99.8 224.1 125%
Dirt 3 110.6 860.7 678%
Forza Horizon 5 108 160 48%
Lara Croft: Temple of Osiris 141 326 131%
Metro 2033 164.4 199.2 21%
Resident Evil 2 26 77 196%
The Crew 26 51 96%
Tiny Tina's Wonderlands 130 360 177%
Total War Saga: Troy 109 146 34%
===========================================================================
== Patches ==
The intended semantics of the patches are broadly intended to match those of the
corresponding Windows functions. For those not already familiar with the Windows
functions (or their undocumented behaviour), patch 27/28 provides a detailed
specification, and individual patches also include a brief description of the
API they are implementing.
The patches making use of this driver in Wine can be retrieved or browsed here:
https://repo.or.cz/wine/zf.git/shortlog/refs/heads/ntsync7
== Previous versions ==
Changes from v6:
* rename NTSYNC_IOC_SEM_POST to NTSYNC_IOC_SEM_RELEASE (matching the NT
terminology instead of POSIX),
* change object creation ioctls to return the fds directly in the return value
instead of through the args struct, which simplifies the API a bit.
* Link to v6: https://lore.kernel.org/lkml/20241209185904.507350-1-zfigura@codeweavers.co…
* Link to v5: https://lore.kernel.org/lkml/20240519202454.1192826-1-zfigura@codeweavers.c…
* Link to v4: https://lore.kernel.org/lkml/20240416010837.333694-1-zfigura@codeweavers.co…
* Link to v3: https://lore.kernel.org/lkml/20240329000621.148791-1-zfigura@codeweavers.co…
* Link to v2: https://lore.kernel.org/lkml/20240219223833.95710-1-zfigura@codeweavers.com/
* Link to v1: https://lore.kernel.org/lkml/20240214233645.9273-1-zfigura@codeweavers.com/
* Link to RFC v2: https://lore.kernel.org/lkml/20240131021356.10322-1-zfigura@codeweavers.com/
* Link to RFC v1: https://lore.kernel.org/lkml/20240124004028.16826-1-zfigura@codeweavers.com/
Elizabeth Figura (30):
ntsync: Return the fd from NTSYNC_IOC_CREATE_SEM.
ntsync: Rename NTSYNC_IOC_SEM_POST to NTSYNC_IOC_SEM_RELEASE.
ntsync: Introduce NTSYNC_IOC_WAIT_ANY.
ntsync: Introduce NTSYNC_IOC_WAIT_ALL.
ntsync: Introduce NTSYNC_IOC_CREATE_MUTEX.
ntsync: Introduce NTSYNC_IOC_MUTEX_UNLOCK.
ntsync: Introduce NTSYNC_IOC_MUTEX_KILL.
ntsync: Introduce NTSYNC_IOC_CREATE_EVENT.
ntsync: Introduce NTSYNC_IOC_EVENT_SET.
ntsync: Introduce NTSYNC_IOC_EVENT_RESET.
ntsync: Introduce NTSYNC_IOC_EVENT_PULSE.
ntsync: Introduce NTSYNC_IOC_SEM_READ.
ntsync: Introduce NTSYNC_IOC_MUTEX_READ.
ntsync: Introduce NTSYNC_IOC_EVENT_READ.
ntsync: Introduce alertable waits.
selftests: ntsync: Add some tests for semaphore state.
selftests: ntsync: Add some tests for mutex state.
selftests: ntsync: Add some tests for NTSYNC_IOC_WAIT_ANY.
selftests: ntsync: Add some tests for NTSYNC_IOC_WAIT_ALL.
selftests: ntsync: Add some tests for wakeup signaling with
WINESYNC_IOC_WAIT_ANY.
selftests: ntsync: Add some tests for wakeup signaling with
WINESYNC_IOC_WAIT_ALL.
selftests: ntsync: Add some tests for manual-reset event state.
selftests: ntsync: Add some tests for auto-reset event state.
selftests: ntsync: Add some tests for wakeup signaling with events.
selftests: ntsync: Add tests for alertable waits.
selftests: ntsync: Add some tests for wakeup signaling via alerts.
selftests: ntsync: Add a stress test for contended waits.
maintainers: Add an entry for ntsync.
docs: ntsync: Add documentation for the ntsync uAPI.
ntsync: No longer depend on BROKEN.
Documentation/userspace-api/index.rst | 1 +
Documentation/userspace-api/ntsync.rst | 385 +++++
MAINTAINERS | 9 +
drivers/misc/Kconfig | 1 -
drivers/misc/ntsync.c | 992 +++++++++++-
include/uapi/linux/ntsync.h | 42 +-
tools/testing/selftests/Makefile | 1 +
.../selftests/drivers/ntsync/.gitignore | 1 +
.../testing/selftests/drivers/ntsync/Makefile | 7 +
tools/testing/selftests/drivers/ntsync/config | 1 +
.../testing/selftests/drivers/ntsync/ntsync.c | 1343 +++++++++++++++++
11 files changed, 2767 insertions(+), 16 deletions(-)
create mode 100644 Documentation/userspace-api/ntsync.rst
create mode 100644 tools/testing/selftests/drivers/ntsync/.gitignore
create mode 100644 tools/testing/selftests/drivers/ntsync/Makefile
create mode 100644 tools/testing/selftests/drivers/ntsync/config
create mode 100644 tools/testing/selftests/drivers/ntsync/ntsync.c
base-commit: cdd30ebb1b9f36159d66f088b61aee264e649d7a
--
2.45.2