On Thu 2021-02-11 18:37:52, John Ogness wrote:
If message sizes average larger than expected (more than 32 characters), the data_ring will wrap before the desc_ring. Once the data_ring wraps, it will start invalidating descriptors. These invalid descriptors hang around until they are eventually recycled when the desc_ring wraps. Readers do not care about invalid descriptors, but they still need to iterate past them. If the average message size is much larger than 32 characters, then there will be many invalid descriptors preceding the valid descriptors.
The function prb_first_valid_seq() always begins at the oldest descriptor and searches for the first valid descriptor. This can be rather expensive for the above scenario. And, in fact, because of its heavy usage in /dev/kmsg, there have been reports of long delays and even RCU stalls.
For code that does not need to search from the oldest record, replace prb_first_valid_seq() usage with prb_read_valid_*() functions, which provide a start sequence number to search from.
Fixes: 896fbe20b4e2333fb55 ("printk: use the lockless ringbuffer") Reported-by: kernel test robot oliver.sang@intel.com Reported-by: J. Avila elavila@google.com Signed-off-by: John Ogness john.ogness@linutronix.de
Could you please push this fix into the stable releases based on 5.10 and 5.11, please?
The patch fixes a visible performance regression. It has landed in the mainline as the commit 13791c80b0cdf54d92fc542 ("printk: avoid prb_first_valid_seq() where possible").
It should apply cleanly.
Best Regards, Petr
On Tue, Feb 23, 2021 at 10:02:14AM +0100, Petr Mladek wrote:
On Thu 2021-02-11 18:37:52, John Ogness wrote:
If message sizes average larger than expected (more than 32 characters), the data_ring will wrap before the desc_ring. Once the data_ring wraps, it will start invalidating descriptors. These invalid descriptors hang around until they are eventually recycled when the desc_ring wraps. Readers do not care about invalid descriptors, but they still need to iterate past them. If the average message size is much larger than 32 characters, then there will be many invalid descriptors preceding the valid descriptors.
The function prb_first_valid_seq() always begins at the oldest descriptor and searches for the first valid descriptor. This can be rather expensive for the above scenario. And, in fact, because of its heavy usage in /dev/kmsg, there have been reports of long delays and even RCU stalls.
For code that does not need to search from the oldest record, replace prb_first_valid_seq() usage with prb_read_valid_*() functions, which provide a start sequence number to search from.
Fixes: 896fbe20b4e2333fb55 ("printk: use the lockless ringbuffer") Reported-by: kernel test robot oliver.sang@intel.com Reported-by: J. Avila elavila@google.com Signed-off-by: John Ogness john.ogness@linutronix.de
Could you please push this fix into the stable releases based on 5.10 and 5.11, please?
The patch fixes a visible performance regression. It has landed in the mainline as the commit 13791c80b0cdf54d92fc542 ("printk: avoid prb_first_valid_seq() where possible").
It should apply cleanly.
Already queued up, thanks.
greg k-h
linux-stable-mirror@lists.linaro.org
-
Greg KH -
Petr Mladek