On Mon, 2022-03-28 at 09:43 +0800, Xiaomeng Tong wrote:

On Sun, 27 Mar 2022 15:20:42 +0000, Trond Myklebust wrote:

...

On Sun, 2022-03-27 at 16:02 +0800, Xiaomeng Tong wrote:

...

The bug is here:         if (!server ||         server->pnfs_curr_ld->id != dev->cbd_layout_type) {

The list iterator value 'server' will *always* be set and non- NULL by list_for_each_entry_rcu, so it is incorrect to assume that the iterator value will be NULL if the list is empty or no element is found (In fact, it will be a bogus pointer to an invalid struct object containing the HEAD, which is used for above check at next outer loop). Otherwise it may bypass the check in theory (iif server->pnfs_curr_ld->id == dev->cbd_layout_type, 'server' now is a bogus pointer) and lead to invalid memory access passing the check.

To fix the bug, use a new variable 'iter' as the list iterator, while use the original variable 'server' as a dedicated pointer to point to the found element.

Cc: stable@vger.kernel.org Fixes: 1be5683b03a76 ("pnfs: CB_NOTIFY_DEVICEID") Signed-off-by: Xiaomeng Tong xiam0nd.tong@gmail.com

---

fs/nfs/callback_proc.c | 9 +++++----  1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/fs/nfs/callback_proc.c b/fs/nfs/callback_proc.c index c343666d9a42..84779785dc8d 100644 --- a/fs/nfs/callback_proc.c +++ b/fs/nfs/callback_proc.c @@ -361,7 +361,7 @@ __be32 nfs4_callback_devicenotify(void *argp, void *resp,         uint32_t i;         __be32 res = 0;         struct nfs_client *clp = cps->clp; -       struct nfs_server *server = NULL; +       struct nfs_server *server = NULL, *iter;           if (!clp) {                 res = cpu_to_be32(NFS4ERR_OP_NOT_IN_SESSION); @@ -374,10 +374,11 @@ __be32 nfs4_callback_devicenotify(void *argp, void *resp,                 if (!server ||                     server->pnfs_curr_ld->id != dev-

...

cbd_layout_type)

{                         rcu_read_lock(); -                       list_for_each_entry_rcu(server, &clp-

...

cl_superblocks, client_link)

-                               if (server->pnfs_curr_ld && -                                   server->pnfs_curr_ld->id == dev-

...

cbd_layout_type) {

+                       list_for_each_entry_rcu(iter, &clp-

...

cl_superblocks, client_link)

+                               if (iter->pnfs_curr_ld && +                                   iter->pnfs_curr_ld->id == dev-

...

cbd_layout_type) {

rcu_read_unlock(); +                                       server = iter;

Hmm... We're not holding any locks on the super block for 'iter' here, so nothing is preventing it from going away while we're.

ok, i am not a 'rcu lock' expert, i will make it hold the rcu_read_lock() if necessary.

...

Given that we really only want a pointer to the struct pnfs_layoutdriver_type anyway, why not just convert the code to save a pointer to that (and do it while holding the rcu_read_lock())?

Maybe it's not that simple. If you only save a pointer to that and still use 'server' as the list iterator of list_for_each_entry_rcu, there could be problem.

I.e., if no element found in list_for_each_entry_rcu in the first outer 'for' loop, and now 'server' is a bogus pointer to an invalid struct, and continue to go into the second outer 'for' loop, and the check below will lead to invalid memory access (server->pnfs_curr_ld->id), even can potentialy be bypassed with crafted data to make the condition false and mistakely run nfs4_delete_deviceid(server->pnfs_curr_ld, clp, &dev->cbd_dev_id); with bogus 'server'.

if (!server ||     server->pnfs_curr_ld->id != dev->cbd_layout_type) {

...

The struct pnfs_layoutdriver is always expected to be a statically allocated structure, so it won't go away as long as the pNFS driver module remains loaded.

Let's just do the following.

8<----------------------------------------------- From 7c9d845f0612e5bcd23456a2ec43be8ac43458f1 Mon Sep 17 00:00:00 2001 From: Trond Myklebust trond.myklebust@hammerspace.com Date: Mon, 28 Mar 2022 08:36:34 -0400 Subject: [PATCH] NFSv4/pNFS: Fix another issue with a list iterator pointing to the head

In nfs4_callback_devicenotify(), if we don't find a matching entry for the deviceid, we're left with a pointer to 'struct nfs_server' that actually points to the list of super blocks associated with our struct nfs_client. Furthermore, even if we have a valid pointer, nothing pins the super block, and so the struct nfs_server could end up getting freed while we're using it.

Since all we want is a pointer to the struct pnfs_layoutdriver_type, let's skip all the iteration over super blocks, and just use APIs to find the layout driver directly.

Reported-by: Xiaomeng Tong xiam0nd.tong@gmail.com Fixes: 1be5683b03a7 ("pnfs: CB_NOTIFY_DEVICEID") Signed-off-by: Trond Myklebust trond.myklebust@hammerspace.com --- fs/nfs/callback_proc.c | 27 +++++++++------------------ fs/nfs/pnfs.c | 11 +++++++++++ fs/nfs/pnfs.h | 2 ++ 3 files changed, 22 insertions(+), 18 deletions(-)

diff --git a/fs/nfs/callback_proc.c b/fs/nfs/callback_proc.c index 39d1ec870d90..c8520284dda7 100644 --- a/fs/nfs/callback_proc.c +++ b/fs/nfs/callback_proc.c @@ -358,12 +358,11 @@ __be32 nfs4_callback_devicenotify(void *argp, void *resp, struct cb_process_state *cps) { struct cb_devicenotifyargs *args = argp; + const struct pnfs_layoutdriver_type *ld = NULL; uint32_t i; __be32 res = 0; - struct nfs_client *clp = cps->clp; - struct nfs_server *server = NULL;

- if (!clp) { + if (!cps->clp) { res = cpu_to_be32(NFS4ERR_OP_NOT_IN_SESSION); goto out; } @@ -371,23 +370,15 @@ __be32 nfs4_callback_devicenotify(void *argp, void *resp, for (i = 0; i < args->ndevs; i++) { struct cb_devicenotifyitem *dev = &args->devs[i];

- if (!server || - server->pnfs_curr_ld->id != dev->cbd_layout_type) { - rcu_read_lock(); - list_for_each_entry_rcu(server, &clp->cl_superblocks, client_link) - if (server->pnfs_curr_ld && - server->pnfs_curr_ld->id == dev->cbd_layout_type) { - rcu_read_unlock(); - goto found; - } - rcu_read_unlock(); - continue; + if (!ld || ld->id != dev->cbd_layout_type) { + pnfs_put_layoutdriver(ld); + ld = pnfs_find_layoutdriver(dev->cbd_layout_type); + if (!ld) + continue; } - - found: - nfs4_delete_deviceid(server->pnfs_curr_ld, clp, &dev->cbd_dev_id); + nfs4_delete_deviceid(ld, cps->clp, &dev->cbd_dev_id); } - + pnfs_put_layoutdriver(ld); out: kfree(args->devs); return res; diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index de318bb5d349..856c962273c7 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -92,6 +92,17 @@ find_pnfs_driver(u32 id) return local; }

+const struct pnfs_layoutdriver_type *pnfs_find_layoutdriver(u32 id) +{ + return find_pnfs_driver(id); +} + +void pnfs_put_layoutdriver(const struct pnfs_layoutdriver_type *ld) +{ + if (ld) + module_put(ld->owner); +} + void unset_pnfs_layoutdriver(struct nfs_server *nfss) { diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index f4d7548d67b2..07f11489e4e9 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -234,6 +234,8 @@ struct pnfs_devicelist {

extern int pnfs_register_layoutdriver(struct pnfs_layoutdriver_type *); extern void pnfs_unregister_layoutdriver(struct pnfs_layoutdriver_type *); +extern const struct pnfs_layoutdriver_type *pnfs_find_layoutdriver(u32 id); +extern void pnfs_put_layoutdriver(const struct pnfs_layoutdriver_type *ld);

/* nfs4proc.c */ extern size_t max_response_pages(struct nfs_server *server);

On Mon, 28 Mar 2022 13:24:57 +0000, Trond Myklebust wrote:

Let's just do the following.

8<----------------------------------------------- From 7c9d845f0612e5bcd23456a2ec43be8ac43458f1 Mon Sep 17 00:00:00 2001 From: Trond Myklebust trond.myklebust@hammerspace.com Date: Mon, 28 Mar 2022 08:36:34 -0400 Subject: [PATCH] NFSv4/pNFS: Fix another issue with a list iterator pointing to the head

In nfs4_callback_devicenotify(), if we don't find a matching entry for the deviceid, we're left with a pointer to 'struct nfs_server' that actually points to the list of super blocks associated with our struct nfs_client. Furthermore, even if we have a valid pointer, nothing pins the super block, and so the struct nfs_server could end up getting freed while we're using it.

Since all we want is a pointer to the struct pnfs_layoutdriver_type, let's skip all the iteration over super blocks, and just use APIs to find the layout driver directly.

Reported-by: Xiaomeng Tong xiam0nd.tong@gmail.com Fixes: 1be5683b03a7 ("pnfs: CB_NOTIFY_DEVICEID") Signed-off-by: Trond Myklebust trond.myklebust@hammerspace.com

---

fs/nfs/callback_proc.c | 27 +++++++++------------------ fs/nfs/pnfs.c | 11 +++++++++++ fs/nfs/pnfs.h | 2 ++ 3 files changed, 22 insertions(+), 18 deletions(-)

diff --git a/fs/nfs/callback_proc.c b/fs/nfs/callback_proc.c index 39d1ec870d90..c8520284dda7 100644 --- a/fs/nfs/callback_proc.c +++ b/fs/nfs/callback_proc.c @@ -358,12 +358,11 @@ __be32 nfs4_callback_devicenotify(void *argp, void *resp, struct cb_process_state *cps) { struct cb_devicenotifyargs *args = argp;

• const struct pnfs_layoutdriver_type *ld = NULL; uint32_t i; __be32 res = 0;

• struct nfs_client *clp = cps->clp;
• struct nfs_server *server = NULL;

• if (!clp) {

• if (!cps->clp) { res = cpu_to_be32(NFS4ERR_OP_NOT_IN_SESSION); goto out; }

@@ -371,23 +370,15 @@ __be32 nfs4_callback_devicenotify(void *argp, void *resp, for (i = 0; i < args->ndevs; i++) { struct cb_devicenotifyitem *dev = &args->devs[i];

• if (!server ||
  

•     server->pnfs_curr_ld->id != dev->cbd_layout_type) {
  

• 	rcu_read_lock();
  

• 	list_for_each_entry_rcu(server, &clp->cl_superblocks, client_link)
  

• 		if (server->pnfs_curr_ld &&
  

• 		    server->pnfs_curr_ld->id == dev->cbd_layout_type) {
  

• 			rcu_read_unlock();
  

• 			goto found;
  

• 		}
  

• 	rcu_read_unlock();
  

• 	continue;
  

• if (!ld || ld->id != dev->cbd_layout_type) {
  

• 	pnfs_put_layoutdriver(ld);
  

• 	ld = pnfs_find_layoutdriver(dev->cbd_layout_type);
  

• 	if (!ld)
  

• 		continue;
  

  }

• found:

• nfs4_delete_deviceid(server->pnfs_curr_ld, clp, &dev->cbd_dev_id);
  

• nfs4_delete_deviceid(ld, cps->clp, &dev->cbd_dev_id);
  

  }

• pnfs_put_layoutdriver(ld);

out: kfree(args->devs); return res; diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index de318bb5d349..856c962273c7 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -92,6 +92,17 @@ find_pnfs_driver(u32 id) return local; } +const struct pnfs_layoutdriver_type *pnfs_find_layoutdriver(u32 id) +{

• return find_pnfs_driver(id);

+}

+void pnfs_put_layoutdriver(const struct pnfs_layoutdriver_type *ld) +{

• if (ld)

• module_put(ld->owner);
  

+}

void unset_pnfs_layoutdriver(struct nfs_server *nfss) { diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index f4d7548d67b2..07f11489e4e9 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -234,6 +234,8 @@ struct pnfs_devicelist { extern int pnfs_register_layoutdriver(struct pnfs_layoutdriver_type *); extern void pnfs_unregister_layoutdriver(struct pnfs_layoutdriver_type *); +extern const struct pnfs_layoutdriver_type *pnfs_find_layoutdriver(u32 id); +extern void pnfs_put_layoutdriver(const struct pnfs_layoutdriver_type *ld); /* nfs4proc.c */ extern size_t max_response_pages(struct nfs_server *server); --

Thank you, i have resend a PATCH v2 with fix as you suggested, and also with some changes, please check it.

-- Xiaomeng Tong