On Wed, Aug 10, 2022 at 11:38 PM Mike Rapoport rppt@kernel.org wrote:

On Mon, Aug 08, 2022 at 10:56:12AM -0700, Axel Rasmussen wrote:

...

We clearly want to ensure both userfaultfd(2) and /dev/userfaultfd keep working into the future, so just run the test twice, using each interface.

Instead of always testing both userfaultfd(2) and /dev/userfaultfd, let the user choose which to test.

As with other test features, change the behavior based on a new command line flag. Introduce the idea of "test mods", which are generic (not specific to a test type) modifications to the behavior of the test. This is sort of borrowed from this RFC patch series [1], but simplified a bit.

The benefit is, in "typical" configurations this test is somewhat slow (say, 30sec or something). Testing both clearly doubles it, so it may not always be desirable, as users are likely to use one or the other, but never both, in the "real world".

Acked-by: Peter Xu peterx@redhat.com Signed-off-by: Axel Rasmussen axelrasmussen@google.com

With a few nits below

Acked-by: Mike Rapoport rppt@linux.ibm.com

Thanks for reviewing, Mike!

...

---

tools/testing/selftests/vm/userfaultfd.c | 69 ++++++++++++++++++++---- 1 file changed, 60 insertions(+), 9 deletions(-)

diff --git a/tools/testing/selftests/vm/userfaultfd.c b/tools/testing/selftests/vm/userfaultfd.c index 7c3f1b0ab468..cae72867c173 100644 --- a/tools/testing/selftests/vm/userfaultfd.c +++ b/tools/testing/selftests/vm/userfaultfd.c @@ -77,6 +77,11 @@ static int bounces; #define TEST_SHMEM 3 static int test_type;

+#define UFFD_FLAGS (O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY)

+/* test using /dev/userfaultfd, instead of userfaultfd(2) */ +static bool test_dev_userfaultfd;

/* exercise the test_uffdio_*_eexist every ALARM_INTERVAL_SECS */ #define ALARM_INTERVAL_SECS 10 static volatile bool test_uffdio_copy_eexist = true; @@ -125,6 +130,8 @@ struct uffd_stats { const char *examples = "# Run anonymous memory test on 100MiB region with 99999 bounces:\n" "./userfaultfd anon 100 99999\n\n"

• "# Run the same anonymous memory test, but using /dev/userfaultfd:\n"
• "./userfaultfd anon:dev 100 99999\n\n" "# Run share memory test on 1GiB region with 99 bounces:\n" "./userfaultfd shmem 1000 99\n\n" "# Run hugetlb memory test on 256MiB region with 50 bounces:\n"

@@ -141,6 +148,14 @@ static void usage(void) "[hugetlbfs_file]\n\n"); fprintf(stderr, "Supported <test type>: anon, hugetlb, " "hugetlb_shared, shmem\n\n");

• fprintf(stderr, "'Test mods' can be joined to the test type string with a ':'. "
  

•         "Supported mods:\n");
  

• fprintf(stderr, "\tsyscall - Use userfaultfd(2) (default)\n");
  

• fprintf(stderr, "\tdev - Use /dev/userfaultfd instead of userfaultfd(2)\n");
  

• fprintf(stderr, "\nExample test mod usage:\n");
  

• fprintf(stderr, "# Run anonymous memory test with /dev/userfaultfd:\n");
  

• fprintf(stderr, "./userfaultfd anon:dev 100 99999\n\n");
  

• fprintf(stderr, "Examples:\n\n");
  fprintf(stderr, "%s", examples);
  exit(1);
  

@@ -154,12 +169,14 @@ static void usage(void) ret, __LINE__); \ } while (0)

-#define err(fmt, ...) \ +#define errexit(exitcode, fmt, ...) \ do { \ _err(fmt, ##__VA_ARGS__); \

•         exit(1);                        \
  

•         exit(exitcode);                 \
  } while (0)
  
  

+#define err(fmt, ...) errexit(1, fmt, ##__VA_ARGS__)

static void uffd_stats_reset(struct uffd_stats *uffd_stats, unsigned long n_cpus) { @@ -383,13 +400,29 @@ static void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls) } }

+static int __userfaultfd_open_dev(void) +{

• int fd, _uffd = -1;
  

Nit: the initialization here is not necessary, _uffd is always set from ioctl() return value.

Agreed.

...

• fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
  

• if (fd < 0)
  

•         return -1;
  

• _uffd = ioctl(fd, USERFAULTFD_IOC_NEW, UFFD_FLAGS);
  

• close(fd);
  

• return _uffd;
  

+}

static void userfaultfd_open(uint64_t *features) { struct uffdio_api uffdio_api;

• uffd = syscall(__NR_userfaultfd, O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY);
  

• if (test_dev_userfaultfd)
  

•         uffd = __userfaultfd_open_dev();
  

• else
  

•         uffd = syscall(__NR_userfaultfd, UFFD_FLAGS);
  if (uffd < 0)
  

•         err("userfaultfd syscall not available in this kernel");
  

•         errexit(KSFT_SKIP, "creating userfaultfd failed");
  

I'm not sure if this should be KSFT_SKIP. If creation of uffd failed because anything except ENOSYS I'd consider the test failing.

Agreed, but looking at it I think it's more complicated.

In __userfaultfd_open_dev, I think: - If we fail to open /dev/userfaultfd for any reason, that's a KSFT_SKIP - If USERFAULTFD_IOC_NEW returns ENOTTY that's a KSFT_SKIP, otherwise a real test failure.

But then for syscall(__NR_userfaultfd, ...), I think ENOSYS is a KSFT_SKIP, but any other errno is a real test failure.

I'll send an updated version which has those semantics. I think to do so we need to move some of the error handling into __userfaultfd_open_dev, instead of having its caller do it.

...

  uffd_flags = fcntl(uffd, F_GETFD, NULL);

  uffdio_api.api = UFFD_API;

@@ -1584,8 +1617,6 @@ unsigned long default_huge_page_size(void)

static void set_test_type(const char *type) {

• uint64_t features = UFFD_API_FEATURES;
  

• if (!strcmp(type, "anon")) {
          test_type = TEST_ANON;
          uffd_test_ops = &anon_uffd_test_ops;
  

@@ -1603,9 +1634,29 @@ static void set_test_type(const char *type) test_type = TEST_SHMEM; uffd_test_ops = &shmem_uffd_test_ops; test_uffdio_minor = true;

• } else {
  

•         err("Unknown test type: %s", type);
  }
  

+}

+static void parse_test_type_arg(const char *raw_type) +{

• char *buf = strdup(raw_type);
  

• uint64_t features = UFFD_API_FEATURES;
  

• while (buf) {
  

•         const char *token = strsep(&buf, ":");
  

•         if (!test_type)
  

•                 set_test_type(token);
  

•         else if (!strcmp(token, "dev"))
  

•                 test_dev_userfaultfd = true;
  

•         else if (!strcmp(token, "syscall"))
  

•                 test_dev_userfaultfd = false;
  

•         else
  

•                 err("unrecognized test mod '%s'", token);
  

• }
  

• if (!test_type)
  

•         err("failed to parse test type argument: '%s'", raw_type);
  
  if (test_type == TEST_HUGETLB)
          page_size = default_huge_page_size();
  

@@ -1653,7 +1704,7 @@ int main(int argc, char **argv) err("failed to arm SIGALRM"); alarm(ALARM_INTERVAL_SECS);

• set_test_type(argv[1]);
  

• parse_test_type_arg(argv[1]);
  
  nr_cpus = sysconf(_SC_NPROCESSORS_ONLN);
  nr_pages_per_cpu = atol(argv[2]) * 1024*1024 / page_size /
  

-- 2.37.1.559.g78731f0fdb-goog

-- Sincerely yours, Mike.

On Mon, Aug 08, 2022 at 10:56:13AM -0700, Axel Rasmussen wrote:

Explain the different ways to create a new userfaultfd, and how access control works for each way.

Acked-by: Peter Xu peterx@redhat.com Signed-off-by: Axel Rasmussen axelrasmussen@google.com

---

Documentation/admin-guide/mm/userfaultfd.rst | 41 ++++++++++++++++++-- Documentation/admin-guide/sysctl/vm.rst | 3 ++ 2 files changed, 41 insertions(+), 3 deletions(-)

diff --git a/Documentation/admin-guide/mm/userfaultfd.rst b/Documentation/admin-guide/mm/userfaultfd.rst index 6528036093e1..a76c9dc1865b 100644 --- a/Documentation/admin-guide/mm/userfaultfd.rst +++ b/Documentation/admin-guide/mm/userfaultfd.rst @@ -17,7 +17,10 @@ of the ``PROT_NONE+SIGSEGV`` trick. Design ====== -Userfaults are delivered and resolved through the ``userfaultfd`` syscall. +Userspace creates a new userfaultfd, initializes it, and registers one or more +regions of virtual memory with it. Then, any page faults which occur within the +region(s) result in a message being delivered to the userfaultfd, notifying +userspace of the fault. The ``userfaultfd`` (aside from registering and unregistering virtual memory ranges) provides two primary functionalities: @@ -34,12 +37,11 @@ The real advantage of userfaults if compared to regular virtual memory management of mremap/mprotect is that the userfaults in all their operations never involve heavyweight structures like vmas (in fact the ``userfaultfd`` runtime load never takes the mmap_lock for writing).

Vmas are not suitable for page- (or hugepage) granular fault tracking when dealing with virtual address spaces that could span Terabytes. Too many vmas would be needed for that. -The ``userfaultfd`` once opened by invoking the syscall, can also be +The ``userfaultfd``, once created, can also be passed using unix domain sockets to a manager process, so the same manager process could handle the userfaults of a multitude of different processes without them being aware about what is going on @@ -50,6 +52,39 @@ is a corner case that would currently return ``-EBUSY``). API === +Creating a userfaultfd +----------------------

+There are two ways to create a new userfaultfd, each of which provide ways to +restrict access to this functionality (since historically userfaultfds which +handle kernel page faults have been a useful tool for exploiting the kernel).

+The first way, supported since userfaultfd was introduced, is the +userfaultfd(2) syscall. Access to this is controlled in several ways:

+- Any user can always create a userfaultfd which traps userspace page faults

• only. Such a userfaultfd can be created using the userfaultfd(2) syscall
• with the flag UFFD_USER_MODE_ONLY.

+- In order to also trap kernel page faults for the address space, then either

I think "then" is excessive here ^

• the process needs the CAP_SYS_PTRACE capability, or the system must have
• vm.unprivileged_userfaultfd set to 1. By default, vm.unprivileged_userfaultfd
• is set to 0.

+The second way, added to the kernel more recently, is by opening and issuing a

Maybe:

..., is by opening /dev/userfaultfd and issuing USERFAULTFD_IOC_NEW ioctl to it.

+USERFAULTFD_IOC_NEW ioctl to /dev/userfaultfd. This method yields equivalent +userfaultfds to the userfaultfd(2) syscall.

+Unlike userfaultfd(2), access to /dev/userfaultfd is controlled via normal +filesystem permissions (user/group/mode), which gives fine grained access to +userfaultfd specifically, without also granting other unrelated privileges at +the same time (as e.g. granting CAP_SYS_PTRACE would do). Users who have access +to /dev/userfaultfd can always create userfaultfds that trap kernel page faults; +vm.unprivileged_userfaultfd is not considered.

+Initializing a userfaultfd +--------------------------

When first opened the ``userfaultfd`` must be enabled invoking the ``UFFDIO_API`` ioctl specifying a ``uffdio_api.api`` value set to ``UFFD_API`` (or a later API version) which will specify the ``read/POLLIN`` protocol diff --git a/Documentation/admin-guide/sysctl/vm.rst b/Documentation/admin-guide/sysctl/vm.rst index f74f722ad702..b3e40b42e1b3 100644 --- a/Documentation/admin-guide/sysctl/vm.rst +++ b/Documentation/admin-guide/sysctl/vm.rst @@ -927,6 +927,9 @@ calls without any restrictions. The default value is 0. +Another way to control permissions for userfaultfd is to use +/dev/userfaultfd instead of userfaultfd(2). See +Documentation/admin-guide/mm/userfaultfd.rst. user_reserve_kbytes =================== -- 2.37.1.559.g78731f0fdb-goog