Abner Silva <abner(a)collabora.co.uk> writes:
> Hi Michael,
>
> On Tue, 2012-06-12 at 17:07 +1200, Michael Hudson-Doyle wrote:
>> Those of you who have been around for a while will know that we used
>> to package LAVA as Debian packages. This had its problems and we
>> switched over to the world of pip and virtualenv and
>> lava-deployment-tool.
>>
>> Unfortunately, pip and virtualenv have not really given us the control
>> we need to be confident in our deployments, and so I'm proposing to
>> change again, this time to zc.buildout. This is less of a change than
>> the switch away from debs -- instances of LAVA will still be a concept
>> and the lava-deployment-tool script will still be used, at least for now.
>>
>> The way buildout works is that there is a config file that specifies
>> how -- reproducibly! -- to assemble various pieces into a deployment.
>> I have a branch of lava-server at lp:~mwhudson/lava-server/use-buildout
>> that contains such a config file (and various other related bits and
>> pieces), and a branch of lava-deployment-tool that creates a
>> buildout-based deployment.
>>
>> An instance on disk in the new world is similar to but not the same as
>> an instance is today. In particular it is not a virtual environment
>> -- although for compatibility, it looks like one. There is still a
>> $LAVA_INSTANCE/bin/activate script that you can source in bash to put
>> that instance's scripts at the front of your $PATH. Inside an
>> instance, the etc, run, tmp and var directories server the same
>> function as they do today. There is also a new directory, code. The
>> code directory contains a number of subdirectories, one for each revno
>> of lava-server that has been installed into that instance, and a
>> symlink 'current' that points to the currently active one.
>>
>> There are now two merge proposals on LP:
>>
>> https://code.launchpad.net/~mwhudson/lava-server/use-buildout/+merge/109768
>> https://code.launchpad.net/~mwhudson/lava-deployment-tool/use-buildout/+mer…
>>
>> I've made an effort to update the documentation in l-d-t's README, and
>> I've put an htmlified version of this file online at
>> http://people.linaro.org/~mwh/ldt.html so that might be a good place
>> to start.
>
> As I never worked with buildout before I was wondering if there was a
> discussion around this topic before deciding to adopt this new
> deployment approach.
There was some internal discussion, but this is the first time it's been
mentioned on a mailing list I guess.
> I would like to read that and see what was discussed and what are the
> main advantages of changing how to deploy Lava once again. Is it
> available somewhere?
No.
Basically, as I've been the only person to do major deployments to v.l.o
for several months now, I feel I'm in a pretty good position to make
decisions about this! And I hate using the current scripts.
I wanted to spend a little time with buildout before proposing that we
use it to make sure it was worth proposing, and probably spent a little
too long: I basically implemented everything except migration from
existing deployments.
> In case this change really happen, I believe that would be nice to have
> some sort of tutorial teaching how to migrate your current setup and
> deploy it with buildout, without loosing your database, etc.
_Of course_ we will do this sort of thing (we need it for v.l.o!).
Obviously, I don't know all the details of your deployment and so it's
going to be a slightly stressful time when the change comes to be made
-- I hope you can do lots of testing. I'm also happy to learn about
details of your deployment and advise you on how to make the transition.
> Also, I remember to have long conversations on IRC with you guys and
> IIRC there was an agreement that Lava would also be released as deb
> packages again,
I don't know which conversations you're referring to here, I don't think
I was involved. Certainly, in my position as owner of the technical
plan, "package server components as debs again" is not on there.
> so we would have an option to use pip or debs. How is that going?
It's not.
> Please, forgive me if I missed any important discussion.
Well, you weren't on the phone calls Andy and I had to talk about this.
I guess it's just part of the fun of open source: I am most motivated by
the problems _I_ see, and fragility of deployment was one of those. I
should say that nothing I am doing will prevent you from maintaining
your own version of lava-deployment-tool that uses virtualenv & pip but
as we won't be using this approach ourselves this might be more work for
you in the long run.
Cheers,
mwh
On Tue, Jun 12, 2012 at 7:10 PM, Zach Pfeffer <zach.pfeffer(a)linaro.org> wrote:
> On 12 June 2012 11:55, Alexander Sack <asac(a)linaro.org> wrote:
>> On Tue, Jun 12, 2012 at 6:54 PM, Zach Pfeffer <zach.pfeffer(a)linaro.org> wrote:
>>> On 12 June 2012 09:02, Alexander Sack <asac(a)linaro.org> wrote:
>>>> Hi,
>>>>
>>>> looking at origen and snowball lava results from our tip builds I
>>>> realized that mmtest's are failing:
>>>>
>>>> origen: https://android-build.linaro.org/builds/~linaro-android/origen-ics-gcc47-sa…
>>>> -> http://validation.linaro.org/lava-server/dashboard/streams/private/team/lin…
>>>
>>> I get a 404 error for this one ^^^^^^^^^^^^^
>>
>>
>> you have to be logged into validation.linaro.org to see any raw
>> results nowadays. try that.
>
> That worked better. How can I download the log? Every time I try and
> save it its giving me a download.html file?
>
Good question. Only thing I know is how to open the "inline" view ...
I guess there is a secret link to download attachment, but yes, I also
have a hard time spotting the download link for it (e.g. stdout.log)
:/
Andy? LAVA folks?
--
Alexander Sack
Technical Director, Linaro Platform Teams
http://www.linaro.org | Open source software for ARM SoCs
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog
lava-core will be very slow progress. lava-core only has untested
support for fast-models at the moment. We won't be porting over all our
other components in the near term.
On 06/12/2012 08:59 AM, Abner Silva wrote:
> Hi,
>
> I haven't heard about Lava Core effort for a while already. What is the
> current status? Any news on this front?
>
> BR,
>
> -Abner
>
> On Sat, 2012-04-14 at 20:14 +0200, Zygmunt Krynicki wrote:
>> Hi
>>
>> I'd like to announce a new project in the LAVA family.
>>
>> (README file follows)
>> http://xkcd.com/927/
>>
>> Seriously, lava has way to many tiny pieces. It all started when we
>> renamed launch-control to lava-dashboard and created lava-server in
>> the process. Now it's getting in the way.
>>
>> So here we are, lava-core is the start of the merge. Subsequent
>> projects will slowly merge into the core. Then one day we may just get
>> a package that is called 'lava', that ships a program called 'lava'
>> that does useful stuff.
>> (End of readme file)
>>
>> The project has been registered on Launchpad already. I plan on
>> proposing merge requests that move parts of the non-client pieces
>> there. Existing projects shall remain as backwards compatible code
>> imports from lava-core.
>>
>> Thanks
>> ZK
>> _______________________________________________
>> linaro-validation mailing list
>> linaro-validation(a)lists.linaro.org
>> http://lists.linaro.org/mailman/listinfo/linaro-validation
>
Hi,
if you check the android-build LAVA result table for
https://android-build.linaro.org/builds/~linaro-android/snowball-ics-gcc46-…
you will notice we see quite some details for the benchmarks.
However, we don't see the individual pass/fails for the "simple
pass/fail" tests.
For example:
1. lava, mmtest etc. - don't display what tests succeeded/failed -
just the aggregated number
2. glmark2 etc. - display all the individual measurements
What can we do to display details for all test runs on android-build?
I am in particular keen about seeing the individual "lava" results
inline on android-build.
--
Alexander Sack
Technical Director, Linaro Platform Teams
http://www.linaro.org | Open source software for ARM SoCs
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog
On 06/12/2012 09:10 AM, Abner Silva wrote:
> Hi Michael,
>
> On Tue, 2012-06-12 at 17:07 +1200, Michael Hudson-Doyle wrote:
>> Those of you who have been around for a while will know that we used
>> to package LAVA as Debian packages. This had its problems and we
>> switched over to the world of pip and virtualenv and
>> lava-deployment-tool.
>>
>> Unfortunately, pip and virtualenv have not really given us the control
>> we need to be confident in our deployments, and so I'm proposing to
>> change again, this time to zc.buildout. This is less of a change than
>> the switch away from debs -- instances of LAVA will still be a concept
>> and the lava-deployment-tool script will still be used, at least for now.
>>
>> The way buildout works is that there is a config file that specifies
>> how -- reproducibly! -- to assemble various pieces into a deployment.
>> I have a branch of lava-server at lp:~mwhudson/lava-server/use-buildout
>> that contains such a config file (and various other related bits and
>> pieces), and a branch of lava-deployment-tool that creates a
>> buildout-based deployment.
>>
>> An instance on disk in the new world is similar to but not the same as
>> an instance is today. In particular it is not a virtual environment
>> -- although for compatibility, it looks like one. There is still a
>> $LAVA_INSTANCE/bin/activate script that you can source in bash to put
>> that instance's scripts at the front of your $PATH. Inside an
>> instance, the etc, run, tmp and var directories server the same
>> function as they do today. There is also a new directory, code. The
>> code directory contains a number of subdirectories, one for each revno
>> of lava-server that has been installed into that instance, and a
>> symlink 'current' that points to the currently active one.
>>
>> There are now two merge proposals on LP:
>>
>> https://code.launchpad.net/~mwhudson/lava-server/use-buildout/+merge/109768
>> https://code.launchpad.net/~mwhudson/lava-deployment-tool/use-buildout/+mer…
>>
>> I've made an effort to update the documentation in l-d-t's README, and
>> I've put an htmlified version of this file online at
>> http://people.linaro.org/~mwh/ldt.html so that might be a good place
>> to start.
>
> As I never worked with buildout before I was wondering if there was a
> discussion around this topic before deciding to adopt this new
> deployment approach. I would like to read that and see what was
> discussed and what are the main advantages of changing how to deploy
> Lava once again. Is it available somewhere?
There hasn't been a big dialog. Michael and I had been talking about how
our current pip/virtualenv has been a ticking time bomb and isn't very
reproducible. Michael looked at buildout because its used by the
Launchpad team who has similar but more complex requirements.
As per the discussion - I think this is the place to have it and is why
Michael sent this email.
> In case this change really happen, I believe that would be nice to have
> some sort of tutorial teaching how to migrate your current setup and
> deploy it with buildout, without loosing your database, etc.
We'll get there, one requirement Michael has is to make this still work
with a "lava-deployment-tool upgrade".
> Also, I remember to have long conversations on IRC with you guys and
> IIRC there was an agreement that Lava would also be released as deb
> packages again, so we would have an option to use pip or debs. How is
> that going?
I don't think we've ever agreed to release the server components as
.debs. For the client stuff, we have and will continue to release them
this way.
> Please, forgive me if I missed any important discussion.
>
> Thanks in advance.
>
> BR,
>
> -Abner
>
>> Cheers,
>> mwh
>>
>> _______________________________________________
>> linaro-validation mailing list
>> linaro-validation(a)lists.linaro.org
>> http://lists.linaro.org/mailman/listinfo/linaro-validation
>
All,
Last week (mostly the weekend) I started a project to set up my own lava
instance. I use EC2 for this sort of thing so I don't trash my own systems.
Below is a walk through of my experience. Take it for what it is worth.
I did get something running at http://test3.arago-project.org/
I'll leave it running over the weekend.
I was hoping to do this all again clean but that does not look like it
will happen anytime soon. Rather than letting this get stale, I will
share it in its rather "raw" form.
0) What Ubuntu version?
I started with 12.04 but Lava seems to default to python2.6 and that is
not even an install candidate in 12.04 so I backed up to 11.10. Not
sure if it really matters or not as the virtualenv seems to have used
2.7 even though lava-deployment-tool hard codes 2.6. Did not figure out
what was going on there.
1) lava-tool, lava-scheduler-tool, do not work behind an http proxy
workaround: go home to play w/ lava
2) lava-deployment-tool does not handle uwsgi download correctly
tarball has moved to old/
some places fixed others not
workaround: patch to centralize definition
better fix: look in both "" and "/old"
better'ist fix: also move this to bundle
(otherwise this defeats the whole purpose of all the bundle overhead)
3) lava-deployment-tool wizards are a pain in the neck
better fix: have separate config step that creates values
install then takes bundle and config
4) no admin account and don't know how to install
instructions tell you to login to admin but no info on how to
lava-deployment-tool manage test1 createsuperuser
example should give URL for those not familiar with django conventions
5) deployment does not enable https:// protocol but examples use it
hack workaround: use http: for everything
shouldn't things that require authorization only work over https://??
at least as an option?
6) need to add schedule job permissions for users
prerequisite: login to site w/ your launchpad account
log in to site admin w/ superuser account
find your launchpad account in Users
add scheduler* permissions
better fix: create labuser and labadmin groups and give appropriate
permissions to each, then just add users to appropriate group
7) keyring.core needed but not available
source /srv/lava/instances/test1/bin/activate
pip install keyring
deactivate
8) no devicetypes
docs talk about adding device but does not say you need to add device
type as well
cant add a device w/o first adding a device type
dashboard has a device type also, do I need to add there as well?
[Seems like No]
what about kernel ci?
this all seams very manual
(tried to submit without any device or device type, did get 404 error)
in django admin panel under scheduler add device type panda
[can submit a job now]
9) added device config file to wrong directory, needs to go into
/srv/lava/instances/test1/etc/lava-dispatcher/devices/ not up one level
10) need to install conmux, & missing dependency liburi-perl
(for real HW this would be needed. I got it working but it was not
clear where they got the apc control app from and how it was configured.
need command hardreset)
[switched to qemu focus]
11) Add beagle-xm device type & beagle-xm01 device to scheduler
add devices/beagle-xm01.conf
device_type = beagle-xm
hostname = beagle-xm01
client_type = qemu
12) download of hwpack and rootfs not working
getting html for root dir of release server
wget of same url works ok on same machine
I suspect that url2 opener does not handle redirects that
releases.linaro.com make the user do (even thought wget works fine)
squid proxy probably hides this from dispatcher
result: squid is not really optional
workaround: put copy of images needed into my /images/ref dir and change
job to refer to those.
works
13) need to install qemu
installed qemu-user-static and qemu-system
supported -M beaglexm
however, EC2 tiny instance does not have enough memory to allocate 512M RAM
qemu-system-arm won't honor a -m 128 w/ a -M beaglexm in either order
(I consider that a bug)
installed 4GB swap file and got past this.
(booting linaro image on qemu did not cause any noticeable swapping)
14) qemu command does not work
default_qemu_command was qemu, set it to qemu-system-arm in
beagle-xm01.conf (should be in beagle-xm.conf)
15) qemu boot times out in udev or wait for root.
EC2 tiny instance is probably too underpowered to run this
Need to start over with a bigger machine
Bill
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi
I'd like to announce a new project in the LAVA family.
(README file follows)
http://xkcd.com/927/
Seriously, lava has way to many tiny pieces. It all started when we
renamed launch-control to lava-dashboard and created lava-server in
the process. Now it's getting in the way.
So here we are, lava-core is the start of the merge. Subsequent
projects will slowly merge into the core. Then one day we may just get
a package that is called 'lava', that ships a program called 'lava'
that does useful stuff.
(End of readme file)
The project has been registered on Launchpad already. I plan on
proposing merge requests that move parts of the non-client pieces
there. Existing projects shall remain as backwards compatible code
imports from lava-core.
Thanks
ZK
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQIcBAEBAgAGBQJPib6QAAoJEOvx/vtfL8ZXXgQP+QHhT/4beTTW7+yCFehDwe6i
iDNZVPAhjrXxeh8TFisfdnPP+8z9khkcQTyXOgJnC3T4vbmWNBTjDQTpkSg/0lzS
O8a0YBkcdQ5RPDr1NNiLG7PoAJDUts8yY2DXk2C1ykWmqOZBw7NcRVaX/gQ1Ym75
JODbrtcO1vT+E5rBNYiEgMEELStJZfMGbfGzFI+d4ONz702SJNVDHcY7FReKuTDc
nvOZ8zfW03fHFNobAAR5F57lGBz6oLZjz/nblsCJjA9tf20LPsQTUxkv6EkkEyd1
0CTkjQt60kHQB/FYoxLofJ76CR12gGGNLauAf+fQPBa4CG5MvMsd7fdDbMVa3iZh
MKzOv206SPTmB7GHRVTfTqoeFwsXNEc7CnFvEWA9GbCUboqBxuYnEw95m1137+/7
B3XHqsBmuJC1KJrwNIAFTl41oRuHchwEeZv72wBxoMU+4MKVpHFz6F0P3aR9da4s
Uty9z+Nk8lUyQBGMOy6GMzqlPnNNYPM/v3f0CBZkA+D9mtdD/pFJGkhm/xmyE23S
5NskNi7CXZ+m+kyONAzL3JagLnB7PE+L8tHbap3IKbxkPbB5kzz/3eWqK6Evq+3t
eQJZzuEq8gmIzyHVYjtnRvWXigv3m/vQrkcFC9Gs36tNXG0lEO07DbLpj3ZTlQ03
K+VSjvjsNhMvdhitIY7I
=DKzA
-----END PGP SIGNATURE-----
I have made a new installation from lava-deployment-tools
after that I have try tu use this command:
lava-tool auth-add http://novello@192.168.1.5/RPC2/
Paste token for http://novello@192.168.1.5/RPC2/:
ERROR: Token rejected by server for user novello.
I Have find this error.
sorry but what i have to do to solve it.
Best regards
Novello G.
I HAVE MADE THE INSTALLATION USING LAVA-DEPLOIMENT-TOOLS
thaN i WOULDLIKE TO INSTALL A NEW NEST.
===>
# download example test defintion:curl
http://people.linaro.org/~doanac/pass_fail.py >
lava_test/test_definitions/pass_fail.pyDONE OK# patch
lava_test/core/providers.py to include this test in the#
_builtin_tests arrayDONE OK# "install" the lava-test:lava-test install
pass_failtest)novello@qc:/srv/lava/instances/test/build/lava-android-test$
lava-test install pass_fail
<LAVA_TEST>2012-06-11 09:45:24 AM WARNING: Unable to load test
definition from u'pass_fail': ValueError(u'unknown url type:
pass_fail',)
<LAVA_TEST>2012-06-11 09:45:24 AM WARNING: Unable to load test
definition from u'pass_fail': ValueError(u'unknown url type:
pass_fail',)
ERROR: There is no test with the specified ID
AND HERE I HAVE STOPPED THE TEST.
THEN I HAVE TRY
lava-tool auth-add https://nicola@192.168.1.5/RPC2/
==> TOKEN REJECTED BY SERVER FROM USER NICOLA sorry how i can add a user?
I have made some superuser so i beleave that nicola that is a
superuser could be used
Best Regards
Novello G.
