Please CC me in any replies as I am not subscribed to the list. This is a legitimate request as I often need more than two days especially on busy work days or weekends. On Tue, 2020-11-17 at 09:01 +0100, Pavel Machek wrote:
On Sat 2020-11-14 17:40:36, Hussam Al-Tayeb wrote:
Hello. I would like to suggest lengthening the review period for stable releases from 48 hours to 7 days. The rationale is that 48 hours is not enough for people to test those stable releases and make sure there are no regressions for particular workflows.
You should probably cc stable list and Greg with this.
And yes, I believe that would be good idea.
Plus the period is very often shorter than advertised, which might be also good to fix.
Best regards, pavel
On Sat 2020-11-14 17:40:36, Hussam Al-Tayeb wrote:
Hello. I would like to suggest lengthening the review period for stable releases from 48 hours to 7 days. The rationale is that 48 hours is not enough for people to test those stable releases and make sure there are no regressions for particular workflows.
Disclaimer: I am mostly a user of stable
It's hard to make a good decision here. I share your position the 48-ish hours are a fairly short amound of time, and increasing it would grant more time for tests. As for me, I might resume testing -rc on a regular base as I used to in the past - which is a time-consuming procedure, and since I do that as a hobby, sometimes more important things are in the way. But I have to concede the number of issues that occured only here was never high, and I don't expect it would grow significantly.
On the other hand the pace of the stable patches became fairly high¹, so during a week of -rc review a *lot* of them will queue up and I predict we'll see requests for fast-laning some of them. Also, a release would immediately be followed by the next -rc review period, a procedure that gives me a bad feeling.
So for me, I'd appreciate an extension of the review period, even if it's just four days. But I understand if people prefer to keep the procedures simple, and get fixes out of the door as soon as possible.
My 2¢
Christoph
¹ If somebody made statistics on the development of the number of patches for stable kernels (in count/second), I'd be curious to see the numbers.
On Tue, Nov 17, 2020 at 11:29:16PM +0100, Christoph Biedl wrote:
On Sat 2020-11-14 17:40:36, Hussam Al-Tayeb wrote:
Hello. I would like to suggest lengthening the review period for stable releases from 48 hours to 7 days. The rationale is that 48 hours is not enough for people to test those stable releases and make sure there are no regressions for particular workflows.
Disclaimer: I am mostly a user of stable
It's hard to make a good decision here. I share your position the 48-ish hours are a fairly short amound of time, and increasing it would grant more time for tests. As for me, I might resume testing -rc on a regular base as I used to in the past - which is a time-consuming procedure, and since I do that as a hobby, sometimes more important things are in the way. But I have to concede the number of issues that occured only here was never high, and I don't expect it would grow significantly.
On the other hand the pace of the stable patches became fairly high¹, so during a week of -rc review a *lot* of them will queue up and I predict we'll see requests for fast-laning some of them. Also, a release would immediately be followed by the next -rc review period, a procedure that gives me a bad feeling.
So for me, I'd appreciate an extension of the review period, even if it's just four days. But I understand if people prefer to keep the procedures simple, and get fixes out of the door as soon as possible.
That's the thing, these releases almost always contain fixes that we know people are having in the real world, or they fix reported security issues, so we need to get them out to everyone as soon as possible.
If you are only a week or so behind (because your testing framework takes a week), that's fine, let us know if we broke something last week and we will be glad to revert it or find the fixup patch for it that is in Linus's tree. I work with almost all of the major SoC vendors and they get back to me on this type of delayed cycle because their tests do take longer, and it works fine.
Also, note that I do a release when the testers that I have come to rely on tell me that all is good. I think they are running some 50k+ tests on each release at the moment, so while quantity isn't a substitute for quality, it is a good indication that nothing regressed here which is what I am looking for. Recently these tests are coming back sooner than 2 days, which is great and why I do a release quicker at times (sometimes it is because of just logistics reasons).
So slowing down releases is not the answer. Getting back to me when you have issues is the solution. 1-2-4 weeks is fine, just let us know if you have regressions when you find them, otherwise we don't know that there is an issue that needs to be resolved.
¹ If somebody made statistics on the development of the number of patches for stable kernels (in count/second), I'd be curious to see the numbers.
Yes, I have those numbers, we run about 30-35 patches/day in the stable releases at the moment. You can see this in the spreadsheet I keep at: https://github.com/gregkh/kernel-history in the kernel_stats.ods file, look at the tabs on the bottom to see the rate of change for the different stable releases. It usually is a few releases old, but I try to update it monthly.
thanks,
greg k-h
On Tue, Nov 17, 2020 at 11:29:16PM +0100, Christoph Biedl wrote:
On the other hand the pace of the stable patches became fairly high¹, so during a week of -rc review a *lot* of them will queue up and I predict we'll see requests for fast-laning some of them. Also, a release would immediately be followed by the next -rc review period, a procedure that gives me a bad feeling.
Keep in mind that the stable tree derives itself from Linus's tree - it's not a development tree on it's own and we don't control how many fixes flow into Linus's tree (and as a result into the stable tree).
This means that it doesn't matter how long the review window is open for, you'll be getting the same time to review a single patch - whether we do 200 patches twice a week or 400 patches once a week. We can't create time by moving review windows around.
On Wed, 2020-11-18 at 09:09 -0500, Sasha Levin wrote:
On Tue, Nov 17, 2020 at 11:29:16PM +0100, Christoph Biedl wrote:
On the other hand the pace of the stable patches became fairly high¹, so during a week of -rc review a *lot* of them will queue up and I predict we'll see requests for fast-laning some of them. Also, a release would immediately be followed by the next -rc review period, a procedure that gives me a bad feeling.
Keep in mind that the stable tree derives itself from Linus's tree - it's not a development tree on it's own and we don't control how many fixes flow into Linus's tree (and as a result into the stable tree).
This means that it doesn't matter how long the review window is open for, you'll be getting the same time to review a single patch - whether we do 200 patches twice a week or 400 patches once a week. We can't create time by moving review windows around.
How long does it take for patches reaching Linux's tree to propagate down to the stable trees and is there is mechanism for identifying followup patches? For instance, patch A fixes bug X but we eventually find out that this patch did not fix all occurrences of the bug or caused a regression and hence the author immediately sent patch B for inclusion in mainline (Linux's tree). Is patch B automatically identified for inclusion in stable as well?
In short, is there a guarantee that stable trees are as stable or better than mainline through the current SOP?
Regards, Hussam.
On Wed, Nov 18, 2020 at 08:02:16PM +0200, Hussam Al-Tayeb wrote:
On Wed, 2020-11-18 at 09:09 -0500, Sasha Levin wrote:
On Tue, Nov 17, 2020 at 11:29:16PM +0100, Christoph Biedl wrote:
On the other hand the pace of the stable patches became fairly high¹, so during a week of -rc review a *lot* of them will queue up and I predict we'll see requests for fast-laning some of them. Also, a release would immediately be followed by the next -rc review period, a procedure that gives me a bad feeling.
Keep in mind that the stable tree derives itself from Linus's tree - it's not a development tree on it's own and we don't control how many fixes flow into Linus's tree (and as a result into the stable tree).
This means that it doesn't matter how long the review window is open for, you'll be getting the same time to review a single patch - whether we do 200 patches twice a week or 400 patches once a week. We can't create time by moving review windows around.
How long does it take for patches reaching Linux's tree to propagate down to the stable trees
It depends from a week, to a day, sometimes if it's important, an hour, and sometimes months.
and is there is mechanism for identifying followup patches?
Yes, happens all the time, don't you see this? If we have missed any fixes for fixes, please let us know, but our tools usually catch these pretty well these days.
In short, is there a guarantee that stable trees are as stable or better than mainline through the current SOP?
There's no guarantees in life, especially for free software :)
thanks,
greg k-h
linux-stable-mirror@lists.linaro.org
-
Christoph Biedl -
Greg KH -
Hussam Al-Tayeb -
Sasha Levin