The original problem was from nvme-over-tcp code, who mistakenly uses kernel_sendpage() to send pages allocated by __get_free_pages() without __GFP_COMP flag. Such pages don't have refcount (page_count is 0) on tail pages, sending them by kernel_sendpage() may trigger a kernel panic from a corrupted kernel heap, because these pages are incorrectly freed in network stack as page_count 0 pages.
This patch introduces a helper sendpage_ok(), it returns true if the checking page, - is not slab page: PageSlab(page) is false. - has page refcount: page_count(page) is not zero
All drivers who want to send page to remote end by kernel_sendpage() may use this helper to check whether the page is OK. If the helper does not return true, the driver should try other non sendpage method (e.g. sock_no_sendpage()) to handle the page.
Signed-off-by: Coly Li colyli@suse.de Cc: Chaitanya Kulkarni chaitanya.kulkarni@wdc.com Cc: Christoph Hellwig hch@lst.de Cc: Hannes Reinecke hare@suse.de Cc: Jan Kara jack@suse.com Cc: Jens Axboe axboe@kernel.dk Cc: Mikhail Skorzhinskii mskorzhinskiy@solarflare.com Cc: Philipp Reisner philipp.reisner@linbit.com Cc: Sagi Grimberg sagi@grimberg.me Cc: Vlastimil Babka vbabka@suse.com Cc: stable@vger.kernel.org --- include/linux/net.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
diff --git a/include/linux/net.h b/include/linux/net.h index d48ff1180879..97e8f1a8a427 100644 --- a/include/linux/net.h +++ b/include/linux/net.h @@ -286,6 +286,21 @@ do { \ #define net_get_random_once_wait(buf, nbytes) \ get_random_once_wait((buf), (nbytes))
+/* + * E.g. XFS meta- & log-data is in slab pages, or bcache meta + * data pages, or other high order pages allocated by + * __get_free_pages() without __GFP_COMP, which have a page_count + * of 0 and/or have PageSlab() set. We cannot use send_page for + * those, as that does get_page(); put_page(); and would cause + * either a VM_BUG directly, or __page_cache_release a page that + * would actually still be referenced by someone, leading to some + * obscure delayed Oops somewhere else. + */ +static inline bool sendpage_ok(struct page *page) +{ + return (!PageSlab(page) && page_count(page) >= 1); +} + int kernel_sendmsg(struct socket *sock, struct msghdr *msg, struct kvec *vec, size_t num, size_t len); int kernel_sendmsg_locked(struct sock *sk, struct msghdr *msg,
Currently nvme_tcp_try_send_data() doesn't use kernel_sendpage() to send slab pages. But for pages allocated by __get_free_pages() without __GFP_COMP, which also have refcount as 0, they are still sent by kernel_sendpage() to remote end, this is problematic.
The new introduced helper sendpage_ok() checks both PageSlab tag and page_count counter, and returns true if the checking page is OK to be sent by kernel_sendpage().
This patch fixes the page checking issue of nvme_tcp_try_send_data() with sendpage_ok(). If sendpage_ok() returns true, send this page by kernel_sendpage(), otherwise use sock_no_sendpage to handle this page.
Signed-off-by: Coly Li colyli@suse.de Cc: Chaitanya Kulkarni chaitanya.kulkarni@wdc.com Cc: Christoph Hellwig hch@lst.de Cc: Hannes Reinecke hare@suse.de Cc: Jan Kara jack@suse.com Cc: Jens Axboe axboe@kernel.dk Cc: Mikhail Skorzhinskii mskorzhinskiy@solarflare.com Cc: Philipp Reisner philipp.reisner@linbit.com Cc: Sagi Grimberg sagi@grimberg.me Cc: Vlastimil Babka vbabka@suse.com Cc: stable@vger.kernel.org --- drivers/nvme/host/tcp.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c index 62fbaecdc960..902fe742762b 100644 --- a/drivers/nvme/host/tcp.c +++ b/drivers/nvme/host/tcp.c @@ -912,12 +912,11 @@ static int nvme_tcp_try_send_data(struct nvme_tcp_request *req) else flags |= MSG_MORE | MSG_SENDPAGE_NOTLAST;
- /* can't zcopy slab pages */ - if (unlikely(PageSlab(page))) { - ret = sock_no_sendpage(queue->sock, page, offset, len, + if (sendpage_ok(page)) { + ret = kernel_sendpage(queue->sock, page, offset, len, flags); } else { - ret = kernel_sendpage(queue->sock, page, offset, len, + ret = sock_no_sendpage(queue->sock, page, offset, len, flags); } if (ret <= 0)
In _drbd_send_page() a page is checked by following code before sending it by kernel_sendpage(), (page_count(page) < 1) || PageSlab(page) If the check is true, this page won't be send by kernel_sendpage() and handled by sock_no_sendpage().
This kind of check is exactly what macro sendpage_ok() does, which is introduced into include/linux/net.h to solve a similar send page issue in nvme-tcp code.
This patch uses macro sendpage_ok() to replace the open coded checks to page type and refcount in _drbd_send_page(), as a code cleanup.
Signed-off-by: Coly Li colyli@suse.de Cc: Philipp Reisner philipp.reisner@linbit.com Cc: Sagi Grimberg sagi@grimberg.me --- drivers/block/drbd/drbd_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c index cb687ccdbd96..55dc0c91781e 100644 --- a/drivers/block/drbd/drbd_main.c +++ b/drivers/block/drbd/drbd_main.c @@ -1553,7 +1553,7 @@ static int _drbd_send_page(struct drbd_peer_device *peer_device, struct page *pa * put_page(); and would cause either a VM_BUG directly, or * __page_cache_release a page that would actually still be referenced * by someone, leading to some obscure delayed Oops somewhere else. */ - if (drbd_disable_sendpage || (page_count(page) < 1) || PageSlab(page)) + if (drbd_disable_sendpage || !sendpage_ok(page)) return _drbd_no_send_page(peer_device, page, offset, size, msg_flags);
msg_flags |= MSG_NOSIGNAL;
Hi Coly,
I love your patch! Yet something to improve:
[auto build test ERROR on block/for-next] [also build test ERROR on net/master net-next/master ipvs/master linus/master v5.8 next-20200814] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Coly-Li/net-introduce-helper-sendpa... base: https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git for-next config: i386-randconfig-s002-20200816 (attached as .config) compiler: gcc-9 (Debian 9.3.0-15) 9.3.0 reproduce: # apt-get install sparse # sparse version: v0.6.2-168-g9554805c-dirty # save the attached .config to linux build tree make W=1 C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' ARCH=i386
If you fix the issue, kindly add following tag as appropriate Reported-by: kernel test robot lkp@intel.com
All errors (new ones prefixed by >>):
In file included from net/core/stream.c:18: include/linux/net.h: In function 'sendpage_ok':
include/linux/net.h:301:30: error: implicit declaration of function 'page_count'; did you mean 'file_count'? [-Werror=implicit-function-declaration]
301 | return (!PageSlab(page) && page_count(page) >= 1); | ^~~~~~~~~~ | file_count In file included from include/linux/mm.h:27, from include/linux/bvec.h:13, from include/linux/skbuff.h:17, from include/linux/tcp.h:17, from net/core/stream.c:20: include/linux/page_ref.h: At top level:
include/linux/page_ref.h:70:19: error: static declaration of 'page_count' follows non-static declaration
70 | static inline int page_count(struct page *page) | ^~~~~~~~~~ In file included from net/core/stream.c:18: include/linux/net.h:301:30: note: previous implicit declaration of 'page_count' was here 301 | return (!PageSlab(page) && page_count(page) >= 1); | ^~~~~~~~~~ cc1: some warnings being treated as errors -- In file included from net/ipv6/ip6_fib.c:20: include/linux/net.h: In function 'sendpage_ok':
include/linux/net.h:301:30: error: implicit declaration of function 'page_count'; did you mean 'file_count'? [-Werror=implicit-function-declaration]
301 | return (!PageSlab(page) && page_count(page) >= 1); | ^~~~~~~~~~ | file_count In file included from include/linux/mm.h:27, from include/linux/bvec.h:13, from include/linux/skbuff.h:17, from include/linux/if_ether.h:19, from include/uapi/linux/ethtool.h:19, from include/linux/ethtool.h:18, from include/linux/netdevice.h:37, from net/ipv6/ip6_fib.c:22: include/linux/page_ref.h: At top level:
include/linux/page_ref.h:70:19: error: static declaration of 'page_count' follows non-static declaration
70 | static inline int page_count(struct page *page) | ^~~~~~~~~~ In file included from net/ipv6/ip6_fib.c:20: include/linux/net.h:301:30: note: previous implicit declaration of 'page_count' was here 301 | return (!PageSlab(page) && page_count(page) >= 1); | ^~~~~~~~~~ net/ipv6/ip6_fib.c: In function 'fib6_add': net/ipv6/ip6_fib.c:1373:25: warning: variable 'pn' set but not used [-Wunused-but-set-variable] 1373 | struct fib6_node *fn, *pn = NULL; | ^~ cc1: some warnings being treated as errors -- In file included from net/ipv6/udp.c:24: include/linux/net.h: In function 'sendpage_ok':
include/linux/net.h:301:30: error: implicit declaration of function 'page_count'; did you mean 'file_count'? [-Werror=implicit-function-declaration]
301 | return (!PageSlab(page) && page_count(page) >= 1); | ^~~~~~~~~~ | file_count In file included from include/linux/mm.h:27, from include/linux/bvec.h:13, from include/linux/skbuff.h:17, from include/linux/if_ether.h:19, from include/uapi/linux/ethtool.h:19, from include/linux/ethtool.h:18, from include/linux/netdevice.h:37, from net/ipv6/udp.c:26: include/linux/page_ref.h: At top level:
include/linux/page_ref.h:70:19: error: static declaration of 'page_count' follows non-static declaration
70 | static inline int page_count(struct page *page) | ^~~~~~~~~~ In file included from net/ipv6/udp.c:24: include/linux/net.h:301:30: note: previous implicit declaration of 'page_count' was here 301 | return (!PageSlab(page) && page_count(page) >= 1); | ^~~~~~~~~~ net/ipv6/udp.c:1029:30: warning: no previous prototype for 'udp_v6_early_demux' [-Wmissing-prototypes] 1029 | INDIRECT_CALLABLE_SCOPE void udp_v6_early_demux(struct sk_buff *skb) | ^~~~~~~~~~~~~~~~~~ net/ipv6/udp.c:1070:29: warning: no previous prototype for 'udpv6_rcv' [-Wmissing-prototypes] 1070 | INDIRECT_CALLABLE_SCOPE int udpv6_rcv(struct sk_buff *skb) | ^~~~~~~~~ cc1: some warnings being treated as errors
vim +301 include/linux/net.h
283 284 #define net_get_random_once(buf, nbytes) \ 285 get_random_once((buf), (nbytes)) 286 #define net_get_random_once_wait(buf, nbytes) \ 287 get_random_once_wait((buf), (nbytes)) 288 289 /* 290 * E.g. XFS meta- & log-data is in slab pages, or bcache meta 291 * data pages, or other high order pages allocated by 292 * __get_free_pages() without __GFP_COMP, which have a page_count 293 * of 0 and/or have PageSlab() set. We cannot use send_page for 294 * those, as that does get_page(); put_page(); and would cause 295 * either a VM_BUG directly, or __page_cache_release a page that 296 * would actually still be referenced by someone, leading to some 297 * obscure delayed Oops somewhere else. 298 */ 299 static inline bool sendpage_ok(struct page *page) 300 {
301 return (!PageSlab(page) && page_count(page) >= 1);
302 } 303
--- 0-DAY CI Kernel Test Service, Intel Corporation https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
linux-stable-mirror@lists.linaro.org
-
Coly Li -
kernel test robot