Am 05.11.2017 um 08:46 schrieb Willy Tarreau:
On Sun, Nov 05, 2017 at 06:59:48AM +0000, Harsh Shandilya wrote:
Is this not pushed yet? I only see 3.10.107
Now it is there. Please avoid to rely on it for too long and quickly upgrade to 4.4 or any other maintained version that suits your needs.
Willy
even if EOL has to come once for sure, its the last kernel which can be used on certain devices (embedded) since the kernel is growing bigger and bigger and wont run good anymore with limited resource
Sebastian
On Tue, Nov 14, 2017 at 11:00:46PM +0100, Sebastian Gottschall wrote:
Am 05.11.2017 um 08:46 schrieb Willy Tarreau:
On Sun, Nov 05, 2017 at 06:59:48AM +0000, Harsh Shandilya wrote:
Is this not pushed yet? I only see 3.10.107
Now it is there. Please avoid to rely on it for too long and quickly upgrade to 4.4 or any other maintained version that suits your needs.
Willy
even if EOL has to come once for sure, its the last kernel which can be used on certain devices (embedded) since the kernel is growing bigger and bigger and wont run good anymore with limited resource
It's not totally true, each new kernel also provides options to disable certain parts that are not used by everyone (syscalls, subsystems, etc).
And anyway the end of life has been indicated on kernel.org for 18 months and in every announce in 2017, so it cannot be a surprize anymore :-) At least nobody seemed to complain for all this time!
Cheers, Willy
Am 14.11.2017 um 23:18 schrieb Willy Tarreau:
On Tue, Nov 14, 2017 at 11:00:46PM +0100, Sebastian Gottschall wrote:
Am 05.11.2017 um 08:46 schrieb Willy Tarreau:
On Sun, Nov 05, 2017 at 06:59:48AM +0000, Harsh Shandilya wrote:
Is this not pushed yet? I only see 3.10.107
Now it is there. Please avoid to rely on it for too long and quickly upgrade to 4.4 or any other maintained version that suits your needs.
Willy
even if EOL has to come once for sure, its the last kernel which can be used on certain devices (embedded) since the kernel is growing bigger and bigger and wont run good anymore with limited resource
It's not totally true, each new kernel also provides options to disable certain parts that are not used by everyone (syscalls, subsystems, etc).
And anyway the end of life has been indicated on kernel.org for 18 months and in every announce in 2017, so it cannot be a surprize anymore :-) At least nobody seemed to complain for all this time!
itsn no surprise for sure, but that also means i have to stay on the old kernel for these special devices and your argument about disable certain parts which simply turned bigger over time is no option
since it would remove features which existed before. its not that i enable all features of the kernel. i use every kernel with the same options (some are adjusted since they are renamed or moved)
but even then the kernel is turning into a ram and space eating monster if i look on devices with 16 mb ram and 4 mb flash. this is mainly for maintaining older hardware with latest updates. the more recent hardware is getting better here
you dont seem to know how it is to work on wireless routers :-)
Cheers, Willy
On Tue, Nov 14, 2017 at 11:40:31PM +0100, Sebastian Gottschall wrote:
And anyway the end of life has been indicated on kernel.org for 18 months and in every announce in 2017, so it cannot be a surprize anymore :-) At least nobody seemed to complain for all this time!
itsn no surprise for sure, but that also means i have to stay on the old kernel for these special devices and your argument about disable certain parts which simply turned bigger over time is no option
since it would remove features which existed before. its not that i enable all features of the kernel. i use every kernel with the same options (some are adjusted since they are renamed or moved)
Then I have a few questions : - how did you choose this kernel ? Or did you choose the hardware based on the kernel size ? - what would have you done if 3.10 had not been LTS ? - have you at least tried other kernels before claiming they are much larger ? Following your principle, 3.2 should be smaller and 3.16 not much larger. The former offers you about 6 extra months of maintenance, the latter 3.5 years (https://www.kernel.org/category/releases.html).
but even then the kernel is turning into a ram and space eating monster if i look on devices with 16 mb ram and 4 mb flash. this is mainly for maintaining older hardware with latest updates.
So why didn't you ask if it was possible to pursue the maintenance a bit a long time ago ? LTS maintenance is a collective effort and is done based on usage. If enough people have good reasons for going further it can be enough a justification to push the deadline. Now it's too late.
the more recent hardware is getting better here
you dont seem to know how it is to work on wireless routers :-)
Yes I do, I've been distributing a full blown load balancer distro on a 10 MB image (running on 3.10 as well). But I also know that sometimes you make some nice space savings on new kernels (xz/zstd compression, ability to remove certain useless stuff in these environments such as FS ACLs or mandatory locks, etc). Sure, upgrading to a new kernel on existing hardware is always a challenge. But it's also an interesting one.
Also just to give you an idea, I've just compared the size of these kernels configured with "make allnoconfig" (and I verified that all of them were compressed using gzip) :
3.10.108 : 875 kB 4.4.97 : 522 kB 4.9.61 : 561 kB 4.14 : 566 kB
So the argument that migrating away from 3.10 is hard due to the size doesn't stand much here :-)
Willy
Am 15.11.2017 um 05:32 schrieb Willy Tarreau:
On Tue, Nov 14, 2017 at 11:40:31PM +0100, Sebastian Gottschall wrote:
And anyway the end of life has been indicated on kernel.org for 18 months and in every announce in 2017, so it cannot be a surprize anymore :-) At least nobody seemed to complain for all this time!
itsn no surprise for sure, but that also means i have to stay on the old kernel for these special devices and your argument about disable certain parts which simply turned bigger over time is no option
since it would remove features which existed before. its not that i enable all features of the kernel. i use every kernel with the same options (some are adjusted since they are renamed or moved)
Then I have a few questions :
- how did you choose this kernel ? Or did you choose the hardware based on the kernel size ?
i did not choose it. i port regular all kernels to the platforms i use including 4.4 and 4.9
but a few of these which are already ported to 4.4 and 4.9 will still run 3.10 for resource problems.
- what would have you done if 3.10 had not been LTS ?
using another LTS at that point :-)
- have you at least tried other kernels before claiming they are much larger ? Following your principle, 3.2 should be smaller and 3.16 not much larger. The former offers you about 6 extra months of maintenance, the latter 3.5 years (https://www.kernel.org/category/releases.html).
i also used 3.2 before. sure
dont get me wrong i work with all kernels, also with latests. i do also not complain that 3.10 is now EOL
i just wanted to throw some stones on the bloated kernel problem which is increasing
but even then the kernel is turning into a ram and space eating monster if i look on devices with 16 mb ram and 4 mb flash. this is mainly for maintaining older hardware with latest updates.
So why didn't you ask if it was possible to pursue the maintenance a bit a long time ago ? LTS maintenance is a collective effort and is done based on usage. If enough people have good reasons for going further it can be enough a justification to push the deadline. Now it's too late.
didnt know that. the LTS deadline was defined a long time ago. i follow up the mailing lists mainly for reviewing patches
and reporting feedback if required. if it see such a discussion i may get in touch with it too, but with hundrets of emails every days here its hard to follow anything
the more recent hardware is getting better here
you dont seem to know how it is to work on wireless routers :-)
Yes I do, I've been distributing a full blown load balancer distro on a 10 MB image (running on 3.10 as well). But I also know that sometimes you make some nice space savings on new kernels (xz/zstd compression, ability to remove certain useless stuff in these environments such as FS ACLs or mandatory locks, etc). Sure, upgrading to a new kernel on existing hardware is always a challenge. But it's also an interesting one.
i do use xz and i do use a modified squashfs which is even smaller than the xz one in the kernel
smallest router i run with linux has 8 mb ram and 2 mb flash. so i do know how to get all very small. 10 mb image is no issue for me. the 4 mb flashes devices are my problem.
Also just to give you an idea, I've just compared the size of these kernels configured with "make allnoconfig" (and I verified that all of them were compressed using gzip) :
3.10.108 : 875 kB 4.4.97 : 522 kB 4.9.61 : 561 kB 4.14 : 566 kB
its a little bit unrealistic since you have to count in network subsystem, filesystem and drivers.
standard kernel with xz compression is about 800 - 900 kb for me in 3.10 and 4.4 / 4.9 etc. is often more than 1 - 1.2 mb
sometimes just the 100 kb more count in and turn into a problem since i have to remove something from the image to get it fitting
if you are really interested i can give you a real comparisation using a comparable config on 3.10, 4.4 and 4.9 for a standard mips target
So the argument that migrating away from 3.10 is hard due to the size doesn't stand much here :-)
its turning harder. i already ported 4.4 and 4.9 as i said. so i tried already if they are running or not. they do run, but they are bigger and do not fit for some targets
Willy
On Wed, Nov 15, 2017 at 09:09:34AM +0100, Sebastian Gottschall wrote:
Am 15.11.2017 um 05:32 schrieb Willy Tarreau:
On Tue, Nov 14, 2017 at 11:40:31PM +0100, Sebastian Gottschall wrote:
And anyway the end of life has been indicated on kernel.org for 18 months and in every announce in 2017, so it cannot be a surprize anymore :-) At least nobody seemed to complain for all this time!
itsn no surprise for sure, but that also means i have to stay on the old kernel for these special devices and your argument about disable certain parts which simply turned bigger over time is no option
since it would remove features which existed before. its not that i enable all features of the kernel. i use every kernel with the same options (some are adjusted since they are renamed or moved)
Then I have a few questions :
- how did you choose this kernel ? Or did you choose the hardware based on the kernel size ?
i did not choose it. i port regular all kernels to the platforms i use including 4.4 and 4.9
but a few of these which are already ported to 4.4 and 4.9 will still run 3.10 for resource problems.
- what would have you done if 3.10 had not been LTS ?
using another LTS at that point :-)
- have you at least tried other kernels before claiming they are much larger ? Following your principle, 3.2 should be smaller and 3.16 not much larger. The former offers you about 6 extra months of maintenance, the latter 3.5 years (https://www.kernel.org/category/releases.html).
i also used 3.2 before. sure
dont get me wrong i work with all kernels, also with latests. i do also not complain that 3.10 is now EOL
i just wanted to throw some stones on the bloated kernel problem which is increasing
People used to be working on that, but then it seemed like the "size" got to a point that people were comfortable with it. Are you sure that just changing some build options would not make your image smaller? Letting people know sometime in the past few years that the kernel was getting "too big" for you would have been good to do :)
What exactly is the size limits you are hitting? What is your .config?
thanks,
greg k-h
i just wanted to throw some stones on the bloated kernel problem which is increasing
People used to be working on that, but then it seemed like the "size" got to a point that people were comfortable with it. Are you sure that
There's also a lot of pushback to things that add a ton of ifdefs.
just changing some build options would not make your image smaller? Letting people know sometime in the past few years that the kernel was getting "too big" for you would have been good to do :)
It's also an increasingly hard problem to deal with because the scale of big machines means the algorithms themselves in a modern Linux OS just don't make sense for a tiddly embedded router.
I know lots of people build them that way but if you compare it with one of the more conservative *BSD builds you have to wonder why not use BSD instead - especially with nanoBSD ?
(and BSD has the reverse problem - most BSD does not scale to a modern bigger machine of course).
Alan "1.2.13 was the last true Linux" ;-)
On Fri, Nov 17, 2017 at 11:46:20PM +0000, Alan Cox wrote:
i just wanted to throw some stones on the bloated kernel problem which is increasing
People used to be working on that, but then it seemed like the "size" got to a point that people were comfortable with it. Are you sure that
There's also a lot of pushback to things that add a ton of ifdefs.
just changing some build options would not make your image smaller? Letting people know sometime in the past few years that the kernel was getting "too big" for you would have been good to do :)
It's also an increasingly hard problem to deal with because the scale of big machines means the algorithms themselves in a modern Linux OS just don't make sense for a tiddly embedded router.
It's true but it's also true that a lot of these algorithm can be tuned at build time. We have various memory allocators, tiny RCU, the ability to disable SMP, we can even tune certain filesystems to use more or less buffers, etc. It's not that bad at all and I'm not sure that many other OSes have this level of flexibility.
I know lots of people build them that way but if you compare it with one of the more conservative *BSD builds you have to wonder why not use BSD instead - especially with nanoBSD ?
Well, I maintain a kernel image that I use as a bootloader / recovery image to (re-)install some machines. The kernel+rootfs image fits in 1.7 MB, and in that size it supports a few network drivers, SATA, serial console, kexec, EXT2+VFAT and a few other stuff. Obviously it's a bit limited, but it serves this purpose well as it needs to fit into a 2 MB partition where GRUB is already installed. It started with 2.6.16 about 12 years ago, then 2.6.35, now 3.10.104. The image has increased a bit over time, but it also required a lot less patches to shrink it.
If that can be useful, I can try to port these patches to modern kernels to get them merged. However some of them add new options (eg: enable diag/stats in igb driver, etc) and would probably need to be inverted to disable certain features based on a central option to make the kernel much smaller.
(and BSD has the reverse problem - most BSD does not scale to a modern bigger machine of course).
Alan "1.2.13 was the last true Linux" ;-)
I've been using this one for a while and have even deployed it a few times as LAN->PPP gateways by reconverting old 2 MB RAM 386's (1.6 MB in fact since 384kB were lost and not remappable by then). It was unbeatable for this purpose, though I'm not aiming at making this possible nowadays, most machines have at least a bit more RAM :-)
Willy
It's true but it's also true that a lot of these algorithm can be tuned at build time. We have various memory allocators, tiny RCU, the ability to disable SMP, we can even tune certain filesystems to use more or less buffers, etc. It's not that bad at all and I'm not sure that many other OSes have this level of flexibility.
It's not tuning the algorithms that is the problem. The problem is the choice of algorithm. RCU is a lovely example. The correct solution for a small single or dual CPU device is not to have RCU in the first place. Our tty layer is another - it's about ten times the size it needs to be becauuse it has to handle all sorts of crazy stuff you don't need on a router (to the point we now have an optional second 'tty' layer choice.
Do we want to do that with everything though - a dumb scheduler alternative (the one in Linux 1.2 is actually really good for a little uniprocessor), a new VFS, a simple block layer ?
Likewise on a low memory embedded router you don't need the scheduler logic we have, you don't need the VFS design Linux uses, you don't want the dcache, you don't want most of the disk optimisations, the tty layer and so on.
384kB were lost and not remappable by then). It was unbeatable for this purpose, though I'm not aiming at making this possible nowadays, most machines have at least a bit more RAM :-)
Memory costs power so there are pressures in both directions.
Alan
Am 18.11.2017 um 00:46 schrieb Alan Cox:
i just wanted to throw some stones on the bloated kernel problem which is increasing
People used to be working on that, but then it seemed like the "size" got to a point that people were comfortable with it. Are you sure that
There's also a lot of pushback to things that add a ton of ifdefs.
just changing some build options would not make your image smaller? Letting people know sometime in the past few years that the kernel was getting "too big" for you would have been good to do :)
It's also an increasingly hard problem to deal with because the scale of big machines means the algorithms themselves in a modern Linux OS just don't make sense for a tiddly embedded router.
I know lots of people build them that way but if you compare it with one of the more conservative *BSD builds you have to wonder why not use BSD instead - especially with nanoBSD ?
(and BSD has the reverse problem - most BSD does not scale to a modern bigger machine of course).
Alan "1.2.13 was the last true Linux" ;-)
easy. i have to port alot of cpu platforms to bsd then and since i'm doing
wireless routers i will run into a problem with all the drivers i have to rewrite for bsd and the bsd wireless stack is simply shit
its basicly a impossible task. and most of my code is gpl licensed. mmh mixing with bsd code, not a good idea
linux-stable-mirror@lists.linaro.org
-
Alan Cox -
Greg KH -
Sebastian Gottschall -
Willy Tarreau