linaro-validation

linaro-validation@lists.linaro.org

818 discussions

by Andy Doan

Hey Michael, I was trying to update our staging instance to use a new component today (lava-celery). I got it working, but I'm not sure if I followed the best practice for doing this. What I did: added entry to /srv/lava/instances/staging/code/current/buildout.cfg pulled in my local branch of the component to code/current/local ran ./bin/buildout So what's the proper way to do what I just hacked together? -andy

13 years, 2 months

[PATCH] Include image name and number in job metadata.

by Michael Hudson-Doyle

Include the name and number of the image that will be used for QA purposes in the job metadata so that these jobs and their results can be unambiguously identified in the LAVA database. --- I haven't tested this at all! lava-submit.py | 14 ++++++++++++-- submit-to-lava | 7 +++++-- 2 files changed, 17 insertions(+), 4 deletions(-) diff --git a/lava-submit.py b/lava-submit.py index 185759f..ff014a3 100755 --- a/lava-submit.py +++ b/lava-submit.py @@ -54,7 +54,7 @@ def get_test_actions(tests): }) return actions -def get_job(jobname, device_type, img_url, tests): +def get_job(jobname, device_type, img_url, tests, image_name, image_number): stream = '/private/team/linaro/pre-built-%s/' % DEVICE_STREAM[device_type] job = collections.OrderedDict() @@ -66,6 +66,10 @@ def get_job(jobname, device_type, img_url, tests): 'command': "deploy_linaro_image", 'parameters': { 'image': img_url + }, + 'metadata': { + 'ubuntu.name': image_name, + 'ubuntu.build': image_number, } }, { @@ -98,6 +102,10 @@ def main(): help='The device type to execute on. ie "panda"') p.add_argument('-t', dest='image_type', default='nano', help='The image type to execute on, nano (default) or ubuntu-desktop') + p.add_argument('-n', dest='image_name', required=True, + help='The name of the image for QA purposes, e.g. "lt-panda-x11-base"') + p.add_argument('-c', dest='image_number', required=True, + help='The build number for this image') args = p.parse_args() # test sets specific to images and boards @@ -126,7 +134,9 @@ def main(): except ValueError: pass - job = get_job(args.job_name, args.device_type, args.img_url, tests) + job = get_job( + args.job_name, args.device_type, args.img_url, tests, args.image_name, + args.image_number) user = os.environ.get('LAVA_USER') token = os.environ.get('LAVA_TOKEN') diff --git a/submit-to-lava b/submit-to-lava index b297b5b..5f763dd 100755 --- a/submit-to-lava +++ b/submit-to-lava @@ -29,8 +29,11 @@ SCRIPT_ROOT=$(dirname $0) BUILD_PAT="_${TYPE}_[[:digit:]]*-[[:digit:]]*\.img\.gz$" for img in `find $PREBUILT_DIR -type f | grep "${BUILD_PAT}"` ; do - url="http://snapshots.linaro.org/precise/pre-built/${img#${PREBUILT_DIR}/}" + relpath="${img#${PREBUILT_DIR}/}" + url="http://snapshots.linaro.org/precise/pre-built/${relpath}" info_file=$(dirname $img)/$(basename $img .img.gz).html + image_name="echo $relpath | cut -d/ -f 1" + image_number="echo $relpath | cut -d/ -f 2" if `echo "$url" | grep /lt-panda/ >/dev/null` ; then DEVICE="panda" elif `echo "$url" | grep /lt-panda-x11-base/ >/dev/null` ; then @@ -55,7 +58,7 @@ for img in `find $PREBUILT_DIR -type f | grep "${BUILD_PAT}"` ; do if [ -n "$DEVICE" ] ; then echo "submitting $img to LAVA" - jid=`${SCRIPT_ROOT}/lava-submit.py -t ${TYPE} -j ${JOB_NAME}-${DEVICE} -u $url -d ${DEVICE}` + jid=`${SCRIPT_ROOT}/lava-submit.py -t ${TYPE} -j ${JOB_NAME}-${DEVICE} -u $url -d ${DEVICE} -n ${image_name} -c ${image_number}` if [ $? -eq 0 ] ; then echo "job id: ${jid}" sed -i -e "s/lavaJobId = 0;/lavaJobId=${jid};/" ${info_file} -- 1.7.9.5

13 years, 2 months

About the boot of omapzoom and aosp of panda build in LAVA

by YongQin Liu

Hi, all I met a problem that the omapzoom and aosp build of panda can't be booted up in lava. here is the bug link: https://bugs.launchpad.net/lava-dispatcher/+bug/1012546 When we passed the bootcmd and bootargs via u-boot, and run boot to boot the images deployed on partition3, the u-boot load kernel image and initrd file successfully, and displayed the line: "Uncompressing Linux... done, booting the kernel." but after this line, it seems that the board rebooted and boot the master image as default again, not boot up the android image. I guess it has relation with uboot in master image because I have tried to use the MLO and u-boot.bin and u-boot.img in omapzoom build to replace the files on boot partition of master image, then I can boot the omapzoom and aosp build successfully. (also can boot the tracking build) What I did was as following: 1. deploy omapzoom build to sdcard with linaro-android-media-create command 2. copy the MLO and u-boot.bin and u-boot.img 3 file from sdcard to someplace in local 3. use "./lava-create-master -r 2G panda" to create the sdcard used for master image we can get the lava-create-master script from lp:lava-master-image-scripts 4. plug in the sdcard and boot the board 5. wait the board to complete the partition job. after this, the uImage and uInitrd file in boot partition has been change, and then the master image can be booted with the uboot files from omapzoom build but I don't know why and what change has been done. 6. copy the MLO and u-boot.bin and u-boot.img 3 file to the boot partition of sdcard 7. plug the sdcard to the board and boot it, then it can be use to boot the tracking/omapzoom/aosp 3 builds of panda. The version original u-boot for master image is: U-Boot 2011.12 (Feb 16 2012 - 21:46:14) The u-boot version of omapzoom build is: U-Boot 2011.06 (Aug 20 2011 - 18:00:16). The u-boot version of tracking build is : U-Boot SPL 2012.04.01 (Jun 14 2012 - 08:05:54) The tracking build works fine with the original uboot of master image. So I guess there are some differences between the original uboot for master image and the uboot build in omapzoom build. Anyone know the details why it can not boot the omapzoom and aosp build on the 3rd partition, and if I can boot the omapzoom and aosp build of panda with the original u-boot of master image? Thanks, Yongqin Liu

13 years, 2 months

anon bundlestream cleanup

by Alexander Sack

Hi,, i did a run through anon bundlestreams and identified the following that we want to remove: All anon ci- bundlestreams: ======================= /anonymous/ci-asac-linux-linaro-3_0/ not set 36 4 /anonymous/ci-asac-linux-linaro-3_0-build/ not set 4 4 /anonymous/ci-asac-linux-linaro-3_1/ not set 28 4 /anonymous/ci-asac-linux-linaro-3_1-build/ not set 4 4 /anonymous/ci--build/ not set 18 2 /anonymous/ci-build/ not set 15 7 /anonymous/ci-build-build/ not set 4 4 /anonymous/ci-build-with-ext4-3_0/ not set 41 9 /anonymous/ci-build-with-ext4-3_0-build/ not set 12 12 /anonymous/ci-lci-build-changes/ not set 3 3 /anonymous/ci-lci-build-changes-build/ not set 6 6 /anonymous/ci-linux-igloo-kernel-snowball/ not set 1 1 /anonymous/ci-linux-Igloo-Kernel-Snowball/ not set 5 4 /anonymous/ci-linux-igloo-kernel-snowball-build/ not set 1 1 /anonymous/ci-linux-linaro-snowball-cma-test/ not set 3 3 /anonymous/ci-linux-linaro-snowball-cma-test-build/ not set 5 5 /anonymous/ci-linux-maintainers-build/ not set 10 2 /anonymous/ci-linux-maintainers-build-build/ not set 2 2 /anonymous/ci-linux-maintainers-kernel/ not set 5 5 /anonymous/ci-linux-maintainers-kernel-build/ not set 18 18 /anonymous/ci-packaged-linux-linaro-3_0/ not set 62 10 /anonymous/ci-packaged-linux-linaro-3_0-build/ not set 24 24 /anonymous/ci-packaged-linux-linaro-3_1/ not set 190 51 /anonymous/ci-packaged-linux-linaro-3_1-build/ not set 144 144 /anonymous/ci-packaged-linux-linaro-build/ not set 1 1 /anonymous/ci-packaged-linux-linaro-build-build/ not set 1 1 /anonymous/ci-packaged-linux-linaro-lt-3_1/ not set 10 5 /anonymous/ci-packaged-linux-linaro-lt-3_1-build/ not set 30 30 /anonymous/ci-Test-TI-working-tree/ not set 21 19 /anonymous/ci-Test-TI-working-tree-build/ not set 28 28 /anonymous/ci-TI-working-tree/ not set 28 15 /anonymous/ci-TI-working-tree-build/ not set 18 18 /anonymous/ci-triage-build/ not set 3 3 /anonymous/ci-upstream/ not set 119 48 All anon lava-android-: =================== /anonymous/lava-android-leb-panda/ not set 910 770 /anonymous/lava-android-panda/ not set 13 13 /anonymous/lava-android-test/ not set 7 7 More random picks: ================= /anonymous/anonymous/ not set 8 5 /anonymous/bootchart/ not set 8 4 /anonymous/chander-kashyap/ /anonymous/lava-daily/ /anonymous/linux-linaro-tracking/ /anonymous/miscellaneous/ /anonymous/panda01-ltp/ Restored from backup 45 44 /anonymous/panda01-posixtestsuite/ Restored from backup 1 1 /anonymous/panda01-stream/ Restored from backup 1 1 /anonymous/qemu-testing-dumping-ground/ not set 12 7 /anonymous/smem/ smem 2 1 /anonymous/USERNAME/ not set 87 69 If we can just hide them that's also fine I guess. -- Alexander Sack Technical Director, Linaro Platform Teams http://www.linaro.org | Open source software for ARM SoCs http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog

13 years, 2 months

Monkey and long monkey trip over new test apks

by Zygmunt Krynicki

Hey everyone. It seems that monkey trips over new test apks that have been added as a side effect of building TARGET_BUILD_VARIANT=tests images. For example, see: https://pastebin.linaro.org/636/ We may need to blacklist some (all?) of them initially to get meanigful results. Best regards ZK -- Zygmunt Krynicki Linaro Validation Team s/Validation/Android/

13 years, 2 months

About the place to put wifi information

by YongQin Liu

Hi, All I have written a script used to connect the android with the wifi in our lava lab. Now we need to pass the ssid and password information to the script. But I think it's better to put the ssid and password in some place of our lava validation, then the script get the information from that place. Like put a file in /etc/lava/devices/wifi, and the content it contains like following: SSID=LAVA-WiFiTest01 PASSWD=PASSWORD SSID2=LAVA-5GWiFiTest01 PASSWD2=PASSWORD2 then in the shell script, we can simply use "source /etc/lava/devices/wifi" to get the SSID and password information. And when we put the script into lava-android-test, we can trigger this script when we need in the test. How do you think about the place to put the devices information? is /etc/lava/devices/wifi ok? bluetooth should have the same problem I think. Thanks, Yongqin Liu.

13 years, 2 months

Already updated the enable_network_after_boot_android definition in file /srv/lava/instances/production/etc/lava-dispatcher/device-defaults.conf for production

by YongQin Liu

Hi, All I have updated the enable_network_after_boot_android definition in file /srv/lava/instances/production/etc/lava-dispatcher/device-defaults.conf for production Just added the following two lines: # This is for android build where the network is not up by default. 1 or 0 enable_network_after_boot_android = 1 here just a notice. BTW, where is the lava source deployed? when I run find command in the /srv/lava/instances/production directory, I can't find the source files of lava-dispatcher Thanks, Yongqin Liu

13 years, 2 months

Health Job Failure Report: 2012-06-18->2012-07-01

by Andy Doan

We've seen a significant reduction in health job failures, but I still wanted to send out a report on these so people could see how things are still breaking. We've had 25 real health failures over the past 2 weeks. By device type: 6 snowball 1 imx53 1 vexpress 2 beagle 6 origen 9 panda By failure type: 2 SD cards died: (both on Origen) 7 Serial Console Related: - 5 connection never established at start of job - 1 connection dropped during test - 1 garbage over serial line 10 Network Related: - 3 network failed to come up - 3 ping unreachable error - 3 wget | tar type failure (kind of network we think) NOTE: A report on something like this is semi-subjective, given that its based on my interpretation of the failures. The raw information on this can be found at: <https://docs.google.com/a/linaro.org/spreadsheet/ccc?key=0AnxpY5uv-BlNdG9zY…>

13 years, 2 months

multi machine lava deployments

by Michael Hudson-Doyle

So we're reaching the point where we are going to want to have more than one machine involved in running LAVA production. The immediate cause is to avoid running fastmodels on the same machine as the database and web server and everything else, but I think as the system grows up we'll want to do this for more functions. I know there have been some thinking about this in the past, but not to my knowledge anything written down. I've spent a good while today thinking about this sort of thing, so here's a bit of a brain dump. The different parts of a LAVA instance ====================================== A LAVA instance is three things: 1) Code 2) Configuration 3) Data. Data comes in two kinds: a postgres database and "media" files (log files and bundles mostly for us). There are a few parts to the configuration: there is the dispatcher config, the server/django config and the apache config. TBH, it would be better if the dispatcher config was derived from the database somehow. I still think we don't do a good job of handling the apache config, but it's a messy problem. The Django configuration includes the information on how to reach the data, both media and db. I think we'll need to define an "instance set" concept -- a list of machines and instance on those machines that share code, configuration and data. Multi-machine code ================== This is easy IMHO: all machines should have the same code installed. With the appropriate ssh keys scattered around, it should be easy to write a fabric job or just plain bash script to run ldt update on each machine. Multi-machine data ================== Accessing postgres from another machine is a solved problem, to put it mildly :-) I don't have a good idea on how to access media files across the network. In an ideal world, we would have a Django storage backend that talked to something like http://ceph.com/ceph-storage/object-storage/ or http://hadoop.apache.org/hdfs/ -- we don't need anything like full file system semantics -- but for now, maybe just mounting the media files over NFS might be the quickest way to get things going. Multi-machine configuration =========================== I think by and large the configuration of each instance should be the same. This means we need a mechanism to distribute changes to the instances. One way would be to store the configuration in a branch, and have ldt update upgrade this branch too (I think it would even be fairly easy to have multiple copies of the configuration on disk, similar to the way we have multiple copies of the buildouts, and have each buildout point to a specific revision of the config). We could also have the revision of the config branch to use be specified in the lava-manifest branch but that doesn't sound friendly to third party deployments -- I think the config branch should be specified as a parameter of the instance set and updating an instance set should update each instance to the latest version of the config branch. This will require a certain discipline in making changes to the branch! All that said, we don't want the instances on each machine to be _completely_ identical, leading to... Differentiating instances ========================= The point of this excercise is not to purely scale horizontally; we want different instances to do different things. I think the primary way we will differentiate instances is by which services they run: do they run uwsgi or not, do they run the scheduler or not, do they run celeryd or not? We already have a limited form of this already, in that you can configure an instance to start the scheduler or not, but this will need systemizing. In addition one instance in a set will need to 'own' the database: when we upgrade an instance we want one and only one instance to run the migrations. I'd also like to push towards a model where we can do rolling upgrades but that's a different kettle of fish I think ... Setup issues ============ There will be a requirement to make sure ssh keys etc are set up on the various machines involved. Ideally this would be done via puppet but for now I think we can just do it by hand... Thoughts? After writing this email, I don't think that there is a huge amount of work to do a good job here; we shouldn't settle for hacks. Cheers, mwh

13 years, 2 months

LAVA 12.07 Release Planning

by Andy Doan

I wanted to send off a note talking about our 12.07 cycle plans for LAVA to describe what we are all working on and how it all fits in with our major team themes. As we've talked about in the past, we've got a few major themes we are focused on right now: * reliability * reporting/visualizations * fast models * external measurement sources (ie power measurement) By theme's 12.07 focus will be: = Reliability: Pre-built image job reliability (spring) <https://blueprints.launchpad.net/lava-dispatcher/+spec/pre-built-reliability> Monitoring Of Scheduler Queues: (andy d) <https://blueprints.launchpad.net/lava-scheduler/+spec/scheduler-queue-monit…> Squid Monitoring and Metrics: (spring) <https://blueprints.launchpad.net/lava-lab/+spec/squid-analytics> //kind of reliability Update Cloud Deployment in Lab: (dave p) <https://blueprints.launchpad.net/lava-lab/+spec/deploy-juju> Create a Proper Staging/Dogfood Server: (dave p) <https://blueprints.launchpad.net/lava-lab/+spec/proper-staging-server> Update the CTS test to use the google package: (yongqin) <https://blueprints.launchpad.net/lava-android-test/+spec/update-cts-test> Monitor Deployment in Lab: (dave p) <https://blueprints.launchpad.net/lava-lab/+spec/lab-monitors> Black Box Test Actions (yongqin) <https://blueprints.launchpad.net/lava-dispatcher/+spec/black-box-test-actio…> = FastModels: Support Fast Models: (amit and andy) <https://blueprints.launchpad.net/lava-dispatcher/+spec/fast-model-support> Ubuntu Fastmodel Support: (amit, maybe some andy) <https://blueprints.launchpad.net/lava-dispatcher/+spec/ubuntu-fastmodels> = Reporting / Visualizations: Build an image status view for the QA services team: (michael) <https://blueprints.launchpad.net/lava-dashboard/+spec/image-status-view-for…> Notify a user when a test fails in LAVA: (spring) <https://blueprints.launchpad.net/lava-dashboard/+spec/linaro-platforms-o-no…> As the power measurement stuff still hasn't arrived, we don't have anything queued up for the "external measurement" theme. However, I think this release will help us take another big stride in reliability and reports. ACTIONS needed from BP assignees: Please look at each of your BP's and make sure the descriptions and more importantly the work items reflect the work as you understand it needs to be done. -andy

13 years, 2 months

← Newer
1
...
64
65
66
67
68
69
70
...
82
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

linaro-validation