On Fri, May 06, 2016 at 12:20:40PM -0400, Jon Masters wrote:
> But is it really worth trying after so long of the right thing not
> happening? If anyone really cared about making general purpose distros boot
> on embedded boards, efforts to compel standards would have happened years
> ago. To do it right, we would need to have a couple of vendors involved who
> could compel vendors to comply.
Distros care and currently do ship on such systems - Debian stable lists
a bunch of boards (something like 20 IIRC) as actively tested for
example. The board and SoC vendors are to an extent irrelevant here,
when people do this they often don't use any of the software the board
vendors provide and just use things like upstream u-boot. Coming up
with something that covers all the cases with minimal code for the
distros will make life easier for them, if board vendors want to pick it
up as well in what they're shipping (and ideally in how they spec their
media usage) that's great but we're already winning even if all the
board vendors totally ignore it.
On Fri, May 6, 2016 at 2:10 AM, Tom Rini <trini(a)konsulko.com> wrote:
> On Thu, May 05, 2016 at 10:21:25PM +0200, Alexander Graf wrote:
>> On 05.05.16 17:21, Grant Likely wrote:
>> > On Thu, May 5, 2016 at 12:45 PM, Marcin Juszkiewicz
>> > <marcin.juszkiewicz(a)linaro.org> wrote:
>> >> Recently my angry post on Google+ [1] got so many comments that it was clear
>> >> that it would be better to move to some mailing list with discussion.
>> >>
>> >> As it is about boot loaders and Linaro has engineers from most of SoC vendor
>> >> companies I thought that this will be best one.
>> >>
>> >> 1. https://plus.google.com/u/0/+MarcinJuszkiewicz/posts/J79qhndV6FY
>> >>
>> >>
>> >> All started when I got Pine64 board (based on Allwinner A64 SoC) and had
>> >> same issue as on several boards in past - boot loader written in some random
>> >> place on SD card.
>> >>
>> >> Days where people used Texas Instruments SoC chips were great - in-cpu boot
>> >> loader knew how to read MBR partition table and was able to load 1st stage
>> >> boot loader (called MLO) from it as long it was FAT filesystem.
>> >>
>> >> GPU used by Raspberry/Pi is able to read MBR, finds 1st partition and reads
>> >> firmware files from there as long it is FAT.
>> >>
>> >> Chromebooks have some SPI flash to keep boot loaders and use GPT
>> >> partitioning to find where from load kernel (or another boot loader).
>> >>
>> >> And then we have all those boards where vendors decided that SPI flash for
>> >> boot loader is too expensive so it will be read from SD card instead. From
>> >> any random place of course...
>> >>
>> >>
>> >> Then we have distributions. And instead of generating bunch of images per
>> >> board they want to make one clean image which will be able to handle as much
>> >> as possible.
>> >>
>> >> If there are UEFI machines on a list of supported ones then GPT partitioning
>> >> will be used, boot loader will be stored in "EFI system area" and it boots.
>> >> This is how AArch64 SBSA/SBBR machines work.
>> >>
>> >> But there are also all those U-Boot (or fastboot/redboot/whateverboot) ones.
>> >> They are usually handled by taking image from previous stage and adding boot
>> >> loader(s) by separate script. And this is where "fun" starts...
>> >>
>> >> GPT takes first 17KB of storage media as it allow to store information about
>> >> 128 partitions. Sure, no one is using so many on such devices but still
>> >> space is reserved.
>> >>
>> >> But most of chips expects boot loader(s) to be stored:
>> >>
>> >> - right after MBR
>> >> - from 1KB
>> >> - from 8KB
>> >> - any other random place
>> >>
>> >> So scripts start to be sets of magic written to handle all those SoCs...
>> >>
>> >> Solution for existing SoCs is usually adding 1MB of SPI flash during design
>> >> phase of device and store boot loader(s) there. But it is so expensive
>> >> someone would say when it is in 10-30 cents range...
>> >>
>> >
>> > To try and summarize, what you're asking for is to define the usage
>> > model for eMMC/SD when both the firmware* and OS are stored on the
>> > same media. Some argue that these things should always be on separate
>> > devices, but while the debate is interesting, it doesn't match the
>> > reality of how hardware is being built. In which case, the derived
>> > requirements are:
>> >
>> > 1) Co-exist with MBR partitioning
>> > 2) Co-exist with GPT partitioning
>> > 3) Be detectable --- partitioning tools must respect it
>> > 4) Be speced. Write it down so that tool and SoC developers can see it
>> > as a requirement
>> > 5) Be usable regardless of firmware type (UEFI, U-Boot, Little Kernel, etc)
>> > 6) Support some form of firmware non-volatile storage (variable storage)
>> >
>> > It would be really nice if we could also have:
>> > 7) Support SoCs that hardcode boot code to specific locations
>> > (after-MBR, 1K, 8K, random)
>> > - May not be able to support all variants, but it is a worthy design goal.
>> >
>> > Agreed?
>> >
>> > * I'm ignoring eMMC's separate boot area because that solution has
>> > firmware and OS logically separated. Strong recommendation is for SoCs
>> > to boot from boot area. Then normal GPT/MBR partitioning works just
>> > fine. The rest of this discussion only applies If the SoC cannot do
>> > that
>> >
>> > (For the following discussion, I refer to the UEFI spec because that
>> > is where GPT is defined, but the expectation is that anything
>> > described here can equally be used by non-UEFI platforms)
>> >
>> > I've just read through the UEFI GPT spec, and here are the constraints:
>> > - MBR must be at the start of LBA0 (0 - 0.5k)
>> > - Primary GPT must be at the start of LBA1 (0.5k to 4k, but may
>> > collide with fw),
>> > - It /seems/ like the GPT Header and GPT table can be separated by
>> > some blocks. The GPT header has a PartitionEntryLBA field which
>> > describes where the actual table of partitions starts.
>> > - GPTHeader is only 92 bytes.
>> > - It should be possible to have: GPTHeader @ start of LBA1 and
>> > GPTPartitionTable @ an LBA that doesn't conflict with firmware.
>> >
>> > I think we have everything we need to work around the location of the
>> > FW boot image without breaking the UEFI spec. The biggest problem is
>> > making sure partitioning tools don't go stomping over required
>> > firmware data and rendering systems unbootable. I *think* we can solve
>> > that problem by extending the MBR definition to block out a required
>> > region and then work around that. Tools can generically see the
>> > special region in the MBR and work around it accordingly.
>>
>> So what's the goal here? Are we trying to force GPT on systems whose
>> vendors never intended them to run with GPT?
>>
>> It really shouldn't matter at the end of the day whether we use GPT or
>> MBR. All uEFI firmware implementations I'm aware of support both. So if
>> you have a device whose bootloader collides with the GPT, just use MBR.
>>
>> As for the "protection" mechanism, I don't think it's a problem either.
>> IIRC parted starts to create partitions with a sensible alignment (1 or
>> 2MB). Most boot loaders fit perfectly fine within that gap.
>>
>> So this really boils down to
>>
>> - use GPT for systems that were designed for it or
>> - use MBR with alignment gap to first partition
>>
>> end of story. There shouldn't be any middle ground. At least I haven't
>> seen any so far :).
That is a good point. MBR is a pain to deal with, but I don't think
there is anything that it is absolutely required for in UEFI land.
> This, for many use cases is also true. A reason that various SoCs pick
> a magic location that is MBR compatible and not strictly GPT compatible
> is that they don't see a use case for GPT-not-MBR being used.
> By-and-large saying your SD card shall have an MBR and shall leave a gap
> that is also well aligned for the card anyhow is enough for (enough)
> firmware to reside in the magic locations.
I think the key issue is how do the tools know what they are allowed
to do. If (for example) the system is booted into a recovery/install
image and needs to repartition and install onto the eMMC, can we get
away from the tools requiring board specific knowledge? A generic
partitioning/install tool needs to know:
- Is an MBR required (ie. SoC reads MBR to find firmware)
- Is FW location at a fixed offset?
- Is GPT supported?
If the tools can get that information, then it can make good decisions
when reimaging a device, and it will make Marcin happier I'm sure. :-)
I /don't/ think the general tools should need to know how to install
the firmware itself. That is still a nasty board-specific problem....
(although even here we could make things better if we had a spec for
managing the SoC's FW partition. I would like to see separate steps
for FW provisioning, and OS install. ie. A board-specific tool to prep
an SD card with the bare minimum for FW, and then generic tools to
complete partitioning and install.
For on-board devices (eMMC), FW provisioning should only be needed once
For removable devices (SD), FW provisioning is only needed when FW
must be on SD. (ie. no-onboard eMMC, or for FW recovery/upgrade)
g.
On Fri, May 6, 2016 at 2:10 AM, Tom Rini <trini(a)konsulko.com> wrote:
> On Thu, May 05, 2016 at 05:30:55PM +0100, Grant Likely wrote:
>> On Thu, May 5, 2016 at 4:59 PM, Mark Brown <broonie(a)kernel.org> wrote:
>> > On Thu, May 05, 2016 at 09:01:05PM +0530, Amit Kucheria wrote:
>> >> On Thu, May 5, 2016 at 5:15 PM, Marcin Juszkiewicz
>> >
>> >> > Solution for existing SoCs is usually adding 1MB of SPI flash during design
>> >> > phase of device and store boot loader(s) there. But it is so expensive
>> >> > someone would say when it is in 10-30 cents range...
>> >
>> >> > Even 96boards CE specification totally ignored that fact while it could be a
>> >> > way of showing how to make popular board. Instead it became
>> >> > yet-another-board-to-laugh (EE spec did not improve much).
>> >
>> >> > Is there a way to get it improved? At least for new designs?
>> >
>> >> Yes! I've added this suggestion to a list of suggestions for evolution
>> >> of the 96boards spec.
>> >
>> > We already went round the houses repeatedly on that one :(
>>
>> It's not so dismal as all that! With the board size and price point
>> the CE spec aims at, it is very unlikely that vendors will build
>> boards with a separate storage device for FW.* However, by and large
>> these platforms will be built with eMMC and eMMC has a separate boot
>> area that is logically separate from the main pool. We can add a
>> requirement for future versions to boot from the boot area when
>> booting from eMMC.
>
> Saying that supporting the eMMC boot partitions shall be bootable AND
> documenting how the various relevant EXT_CSD flags need to be set be
> public is important. It won't do everyone good if you can't program the
> board to boot from the boot partition.
Yes, of course. This is definitely an aspirational item. It doesn't do
any good for current SoCs that already have the boot scheme fixed, but
it is a place where we can put pressure on vendors for what to do in
future SoCs.
g.
On 05/05/2016 12:05 PM, Bill Gatliff wrote:
>
> On Thu, May 5, 2016 at 11:50 AM Martin Stadtler
> <martin.stadtler(a)linaro.org <mailto:martin.stadtler@linaro.org>> wrote:
>
> Specifically for the 96boards, the spec is a recommended view, but
> its not meant to be constraining, however it does allow one to
> then show a best practice, that others can adopt. That's where
> the RPB comes in to play, again to demonstrate and not restrict.
>
>
> Sorry to jump in on this, since my horse in this race is pretty small...
>
> That whole "best practice" point is REALLY important. But it's more
> nuanced than just "do it this way":
>
> A. Vendors will do what they do, and they'll have their reasons. If
> the community offers them a "best practices" guideline, especially one
> that's easy to adopt (in full or in part), then hopefully they'll be
> less likely to stray.
>
> B. If that best-practices document offers more than a
> one-size-fits-all recommendation, then so much the better. Again, it
> keeps necessary variants closer to the community than they might have
> been; a partial victory isn't a total loss.
>
> C. (This is my main point.) When I see a document that says "best
> practices", then I understand that there's no binding requirement per
> se, and that it's likely that there will be deviations.
>
> D. When that's the case, I'm more likely to produce code that's easily
> adapted to the various permutations of best-practices that I
> encounter, often with the blessing of my customer. Doubly so if those
> variations are predictable.
>
> Otherwise, a customer will tell me to just "code to the
> requirement"---and that's all they'll pay for. Then my solution isn't
> likely to be as widely deployable, which cuts off opportunities to
> recycle that solution back to my point (A), even if the code becomes
> public.
>
> I can't always code for every possibility. But, with a best-practices
> guideline that says "if you can do it X way, then do so; if you can't,
> but you can do Y, that's almost as good; else, for gods' sake don't do
> Z", I can better plan for where the changes might arise later.
>
> Maybe I won't code the solution for everyone, but I'm likely to get a
> lot closer than I would have.
One thing I like to see in Best Practices guide is what is the benefit
of the following the best practice. From that it is much easier to
make the assessment of whether the promised result is worth following
the guidance. All of the best practices people here are talking about
appear to be geared toward a frictionless connection to the ARM Linux
ecosystem. That's something many software focused Linaro participants
care about, but is that something manufacturers care about? Usually I
only hear about saving pennies leading to profits at scale being a
priority. So if we can talk up gaining scale by following the
practices, there's a better chance members will listen.
--
Brendan Conoboy / RHEL Development Coordinator / Red Hat, Inc.
Recently my angry post on Google+ [1] got so many comments that it was
clear that it would be better to move to some mailing list with discussion.
As it is about boot loaders and Linaro has engineers from most of SoC
vendor companies I thought that this will be best one.
1. https://plus.google.com/u/0/+MarcinJuszkiewicz/posts/J79qhndV6FY
All started when I got Pine64 board (based on Allwinner A64 SoC) and had
same issue as on several boards in past - boot loader written in some
random place on SD card.
Days where people used Texas Instruments SoC chips were great - in-cpu
boot loader knew how to read MBR partition table and was able to load
1st stage boot loader (called MLO) from it as long it was FAT filesystem.
GPU used by Raspberry/Pi is able to read MBR, finds 1st partition and
reads firmware files from there as long it is FAT.
Chromebooks have some SPI flash to keep boot loaders and use GPT
partitioning to find where from load kernel (or another boot loader).
And then we have all those boards where vendors decided that SPI flash
for boot loader is too expensive so it will be read from SD card
instead. From any random place of course...
Then we have distributions. And instead of generating bunch of images
per board they want to make one clean image which will be able to handle
as much as possible.
If there are UEFI machines on a list of supported ones then GPT
partitioning will be used, boot loader will be stored in "EFI system
area" and it boots. This is how AArch64 SBSA/SBBR machines work.
But there are also all those U-Boot (or fastboot/redboot/whateverboot)
ones. They are usually handled by taking image from previous stage and
adding boot loader(s) by separate script. And this is where "fun" starts...
GPT takes first 17KB of storage media as it allow to store information
about 128 partitions. Sure, no one is using so many on such devices but
still space is reserved.
But most of chips expects boot loader(s) to be stored:
- right after MBR
- from 1KB
- from 8KB
- any other random place
So scripts start to be sets of magic written to handle all those SoCs...
Solution for existing SoCs is usually adding 1MB of SPI flash during
design phase of device and store boot loader(s) there. But it is so
expensive someone would say when it is in 10-30 cents range...
Even 96boards CE specification totally ignored that fact while it could
be a way of showing how to make popular board. Instead it became
yet-another-board-to-laugh (EE spec did not improve much).
Is there a way to get it improved? At least for new designs?
+1
On 5 May 2016 at 20:08, Bill Gatliff <bgat(a)billgatliff.com> wrote:
>
>
> On Thu, May 5, 2016 at 11:50 AM Martin Stadtler <
> martin.stadtler(a)linaro.org> wrote:
>
>> I would add, that you need to draw a line in the sand between the
>> 'consumer' (don't flame me, I am just struggling to find a better term)
>> boards and those that are positioned for production/enterprise.
>>
>
> One way to draw the line is whether the board is delivered to be used
> as-is, vs. with the expectation of further development activities that will
> change the work product that shipped with the board.
>
> In the first case, all you care is that it works when it leaves the dock.
> In the latter case, you care more that it can't be easily broken later.
>
> That's a pretty big difference in my experience.
>
> b.g.
> --
>
> Bill Gatliff
> Embedded Linux training and consulting
> bgat(a)billgatliff.com
> (309) 453-3421
>
--
Martin Stadtler
Director, Enterprise Group
Linaro Ltd
mobile: +44.7492180779
IRC: martinst@#linaro on irc.freenode.ne <http://irc.freenode.net/>t
Hello,
Summary: Your SSH username on https://android-review.linaro.org will
now match Linaro username (first.last) after the next login. You local
repository clones need to be updated for new username.
Detailed description:
https://android-review.linaro.org was the first Gerrit server in
Linaro, when there were no central LDAP user database yet. As a result,
there were free-form SSH usernames used, instead of the later standard
first.last as used on all the other Gerrit servers. This inconsistency
was a subject of background concern for Systems team, but of course not
something having enough priority to "fix". However, Gerrit 2.12.2
upgrade started tonight uncovered following issue: Gerrit tries to
synchronize its SSH username setting with LDAP, and fails, as Gerrits
own rules disallow changing of username. The symptom of this is error
"Authentication temporary unavailable" when a user with "old" SSH
username tries to login via browser.
While this can be classified as a Gerrit bug (and previous versions
were smart enough), we'll unlikely find any timely solution to keep
things as they are. So, it makes sense to take a chance of cleaning up
username and making this server follow standard username conventions.
Consequently:
1. All usernames which don't match "first.last" pattern are reset.
2. If you are affected, you won't be able to perform SSH operations
(like git clone/push) until you login via web interface.
3. On the next login via web interface, it will be set to LDAP's
"first.last" value.
4. You will need to update remotes of your existing git clones to new
username (or alternatively, re-clone).
5. If you already use "first.last" SSH username, you're unaffected.
The list of users affected is given below. While it seems long,
majority of enrties there are for people no longer at Linaro, or
for community accounts (which didn't work since we switched to LDAP
anyway). If you need further assistance, please open a ticket at
https://servicedesk.linaro.org/servicedesk/customer/portal/4 .
Thanks,
Paul
username:Ng
username:pfefferz
username:asac
username:james_w
username:fgiff
username:jserv
username:mabac
username:deeptik
username:cyang
username:mpoirier
username:ndec
username:mwaddel
username:mansson
username:Sachin
username:vishalbhoj
username:suapapa
username:fabo
username:pabhishek
username:cnxsoft
username:amitdanielk
username:patrikryd
username:sangwook
username:ericm
username:pundiramit
username:glandium
username:eazyigz
username:tixy
username:danilo
username:john
username:nytowl
username:tony_tu
username:Sangwook
username:StefanEkenberg
username:uichi
username:plars
username:zyga
username:ruppi
username:ebenpor
username:jhkim
username:Annamalai
username:markoncomputer
username:Claude
username:tianhongwang
username:rchand00
username:omarrmz
username:aviksil
username:nvl1109
username:a0132810
username:pelya
username:krtaylor
username:developer4563
username:yusufbu
username:dzinman
username:arussello
username:mdupuy
username:williamcharles
username:aorth
username:lanaczko
username:rmcc
username:kcrudup
username:kelvin
username:angelsl
username:unixmanlinuxboy
username:Quiter
username:sourxsunny
username:pbeeler
username:anmar
username:wucongdonglai
username:bambi
username:rperier
username:sgt
username:roalex
username:vishveshwarbhat
username:therbom
username:rajagopalvenkat
username:winner00
username:ramesh
username:abelloni
username:sparksco
username:fahadkdi
username:ryanharkin
username:axelfagerstedt
username:nocoast
username:milo
username:fcarpenter
username:hongbozhang
username:fahadk
username:pelya2
username:janjic
username:dpervushin
username:qoowater
username:cerial
username:tinojantony
username:stephaneeee
username:kishoreboddu
username:nuclearmistake
username:SanjasdfsafsffaySinghsdfasfRawat
username:donvigo
username:tinojgit
username:afarah
username:mechmetal
username:akbennett
username:c
username:sikuner
username:wasungkim
username:stevanr
username:deqiang
username:anilkumar
username:willnewton
username:codeart
username:lrabiet
username:Bintian
username:bintian
username:vishalbhoj2
username:tusharbehera
username:jbergsag
username:tgall
username:amitkhare
username:neo
username:paramanands
username:sumit
username:danielt
username:help
username:XavierHsu
username:Serban
username:Z
username:koenkooi
(127 rows)
--
Best Regards,
Paul
Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linarohttp://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog
One of the goals of RPB is to follow that our changes go back
upstream. For the next RPB release, a more polished report is planned.
But to get things started, here's a sample - and to use as baseline to
compare progress for the 16.06 release. The list of packages is
collected from db410/alip image with distrodelta script[1]. This lists
the packages that do not become from Debian (Jessie release) or Debian
backports:
New packages not in Debian:
96boards-artwork: 0.6-0linaro1
96boards-tools: 0.5
optee-client: 1.0.1+git5+g89f25ce-1.linarojessie.1
glshim: 0.41+git20150911.42a7739-0.linarojessie.1
linaro-artwork: 0.6-0linaro1
linaro-default-settings: 0.3-0linaro1
linaro-overlay: 1112.8
wcnss-config: 1.8
Changed packages:
alsa-lib: 1.0.28-1+linaro1.linarojessie.1
* add UCM config file for HDMI and HiFi on DB410c
* add UCM config file for HDMI on IFC6410
chromium-browser: 47.0.2526.16-1.6.linarojessie.1
* seccomp and namespace fixes
* patches to compile on arm64/armhf
firmware-nonfree: 20160110-1.linarojessie.1
* ti-connectivity (updates for HiKey):
- Include wl18xx-fw-4 firmware
- Add TIInit_11.8.32.bts for bluetooth support
isc-dhcp: 4.3.3-8~linaro1
* Reverting requirement for debhelper >= 9.20151220
* Upload to make it work with latest systemd:
- https://bugs.96boards.org/show_bug.cgi?id=271
linux: 4.4.0.linaro.104-1.linarojessie.1
* See kernel changes report (TBD)
No-changes Backports:
apparmor: 2.10-2.linarojessie.1
binutils: 2.25.90.20160101-1.linarojessie.1
gdb: 7.10-1.linarojessie.1
gyp: 0.1+20150913git1f374df9-1.linarojessie.1
java-common: 2:1.8-53.linarojessie.1
nodejs: 4.2.2~dfsg-1.1.linarojessie.1
openssl: 1.0.2d-1.linarojessie.1
systemd: 228-2.4.linarojessie.3
sysvinit: 2.88dsf-59.2.linarojessie.1
util-linux: 2.27.1-1.linarojessie.1
x265: 1.7-4~bpo8+1.linarojessie.1
xorg: 1:7.7+11.linarojessie.1
xorg-server: 2:1.17.3-2.linarojessie.2
xserver-xorg-video-fbdev: 1:0.4.4-1.linarojessie.1
xserver-xorg-video-freedreno: 1.4.0-1.linarojessie.1
[1] https://github.com/suihkulokki/distrodelta with command
distrodelta -o Debian 'Debian Backports'
