Reply inline.
On Thu, Nov 4, 2021 at 11:56 AM Greg KH gregkh@linuxfoundation.org wrote:
On Thu, Nov 04, 2021 at 11:14:55AM -0700, Yi Fan wrote:
Resend the email using plain text.
I found some kernel performance regression issues that might be related w/ 4.14.y LTS commit.
4.14.y commit: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v...
The issue is observed when "console=" is used as a kernel parameter to disable the kernel console.
What exact "performance issue" are you seeing?
[YF] one kernel thread was randomly blocked for more than ~40 milliseconds, causing a certain task to fail to process in time. [YF] the issue is highly random on a single device. But it might happen a few times per 24 hours on a certain percentage of devices. The overall percentage of devices that show the issue seems quite stable over a long period of time (somehow the magic number is ~40%.). [YF] local test on a pool of devices does not show any correlation w/ any particular devices. [YF] local test after reverting the above single commit passes, no issue is observed.
And what kernel version are you seeing it on?
[YF] it was first found on some products w/ kernel version 4.14.210. through bisection, we located the commit on 4.14.200.
I browsed android common kernel logs and the upstream stable kernel tree, found some related changes.
printk: handle blank console arguments passed in. (link: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v...) Revert "init/console: Use ttynull as a fallback when there is no console" (link: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v...)
It looks like upstream also noticed the regression introduced by the commit, and the workaround is to use "ttynull" to handle "console=" case. But the "ttynull" was reverted due to some other reasons mentioned in the commit message.
Any insight or recommendation will be appreciated.
What problem exactly are you now seeing? And does it also happen on 5.15?
[YF] we do not perform any tests on 5.15 yet. so no idea about whether the issue happens on 5.15.
thanks,
greg k-h