Tcwg-commits July 2015

tcwg-commits@lists.linaro.org

2 participants
1355 discussions

[toolchain/semiautobench] 01/01: Specify maindir for Benchmark.job

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a commit to branch inserted_src in repository toolchain/semiautobench. commit aa0f14e75e74df2c7f2fdaf35f699ba7bc8ebc85 Author: Bernard Ogden <bernie.ogden(a)linaro.org> Date: Thu Jul 23 09:30:07 2015 +0200 Specify maindir for Benchmark.job Change-Id: If0f9b9f3fbc2251d918c84f12a2c37c4c1c3d4ed --- invoke_session_debian | 3 +++ 1 file changed, 3 insertions(+) diff --git a/invoke_session_debian b/invoke_session_debian index 650b350..278ba51 100755 --- a/invoke_session_debian +++ b/invoke_session_debian @@ -122,6 +122,9 @@ for lava_name in `lava-group | grep -v '[[:blank:]]*host$' | aw [...] fi done export targets="${target_names[@]}" +export maindir="/${HOME}/bench" +mkdir "${maindir}" +chmod 700 "${maindir}" echo "*** DONE WITH AUTH SOCKET, YOU MAY CLOSE THE CONNECTION" echo "Running benchmark... I may be some time" -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

[gcc] branch trunk updated (9f0a16a -> 4e503e6)

by git

This is an automated email from the git hooks/post-receive script. unknown user pushed a change to branch trunk in repository gcc. from 9f0a16a * config/msp430/t-msp430 (MULTILIB_DIRNAMES): Remove trailing [...] new 43f467b 2015-07-23 Richard Biener <rguenther(a)suse.de> new 9304906 2015-07-23 Richard Biener <rguenther(a)suse.de> new 4e503e6 2015-07-23 Richard Biener <rguenther(a)suse.de> The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: gcc/ChangeLog | 19 +++++++++++++++++++ gcc/gimple.h | 8 ++++---- gcc/match.pd | 3 ++- gcc/testsuite/ChangeLog | 5 +++++ gcc/testsuite/gcc.dg/torture/pr66945.c | 12 ++++++++++++ gcc/tree-ssa-propagate.c | 26 +++++++++++++++++++++++--- 6 files changed, 65 insertions(+), 8 deletions(-) create mode 100644 gcc/testsuite/gcc.dg/torture/pr66945.c -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

[gcc] branch master updated (9f0a16a -> 4e503e6)

by git

This is an automated email from the git hooks/post-receive script. unknown user pushed a change to branch master in repository gcc. from 9f0a16a * config/msp430/t-msp430 (MULTILIB_DIRNAMES): Remove trailing [...] new 43f467b 2015-07-23 Richard Biener <rguenther(a)suse.de> new 9304906 2015-07-23 Richard Biener <rguenther(a)suse.de> new 4e503e6 2015-07-23 Richard Biener <rguenther(a)suse.de> The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: gcc/ChangeLog | 19 +++++++++++++++++++ gcc/gimple.h | 8 ++++---- gcc/match.pd | 3 ++- gcc/testsuite/ChangeLog | 5 +++++ gcc/testsuite/gcc.dg/torture/pr66945.c | 12 ++++++++++++ gcc/tree-ssa-propagate.c | 26 +++++++++++++++++++++++--- 6 files changed, 65 insertions(+), 8 deletions(-) create mode 100644 gcc/testsuite/gcc.dg/torture/pr66945.c -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

[toolchain/abe] branch bernie/jenkmarking-multinode updated: Let maindir be specified from the outside

by git

9 years, 4 months

[toolchain/semiautobench] branch inserted_src updated: Specify maindir for Benchmark.job

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a commit to branch inserted_src in repository toolchain/semiautobench. The following commit(s) were added to refs/heads/inserted_src by this push: new b94e85e Specify maindir for Benchmark.job b94e85e is described below commit b94e85ee234245f1acf0597b6739586883c27dd6 Author: Bernard Ogden <bernie.ogden(a)linaro.org> Date: Thu Jul 23 09:30:07 2015 +0200 Specify maindir for Benchmark.job Change-Id: If0f9b9f3fbc2251d918c84f12a2c37c4c1c3d4ed --- invoke_session_debian | 1 + 1 file changed, 1 insertion(+) diff --git a/invoke_session_debian b/invoke_session_debian index 650b350..e4d2346 100755 --- a/invoke_session_debian +++ b/invoke_session_debian @@ -122,6 +122,7 @@ for lava_name in `lava-group | grep -v '[[:blank:]]*host$' | aw [...] fi done export targets="${target_names[@]}" +export maindir="/${HOME}/bench" echo "*** DONE WITH AUTH SOCKET, YOU MAY CLOSE THE CONNECTION" echo "Running benchmark... I may be some time" -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

[toolchain/abe] branch bernie/jenkmarking-multinode updated: Strip out all references to lava

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a commit to branch bernie/jenkmarking-multinode in repository toolchain/abe. The following commit(s) were added to refs/heads/bernie/jenkmarking-multinode by th [...] new 9797214 Strip out all references to lava 9797214 is described below commit 9797214c97bb008f2660fda88b206206cda1fe24 Author: Bernard Ogden <bernie.ogden(a)linaro.org> Date: Wed Jul 22 14:28:03 2015 +0200 Strip out all references to lava Change-Id: Ic970aedc7f32a32d4ca052cf8da9f399e146d5f8 --- config/boards/bench/arndale.conf | 3 +- config/boards/bench/arndale.json | 40 ----- config/boards/bench/juno-a53.conf | 2 +- config/boards/bench/juno-a53.json | 41 ----- config/boards/bench/juno-a57.conf | 2 +- config/boards/bench/juno-a57.json | 41 ----- config/boards/bench/kvm.conf | 2 +- config/boards/bench/kvm.json | 39 ----- config/boards/bench/panda-es.conf | 3 +- config/boards/bench/panda-es.json | 41 ----- scripts/benchmark.sh | 10 +- scripts/benchutil.sh | 51 ------ scripts/lava.sh | 356 -------------------------------------- scripts/runbenchmark.sh | 103 +---------- 14 files changed, 12 insertions(+), 722 deletions(-) diff --git a/config/boards/bench/arndale.conf b/config/boards/bench/arndale.conf index 46b1fb4..00644a4 100644 --- a/config/boards/bench/arndale.conf +++ b/config/boards/bench/arndale.conf @@ -3,8 +3,7 @@ netctl=yes freq=900MHz benchcore=0 othercore=1 -ip=arndale.json -#ip=linaro(a)10.2.201.83 +ip= boot_timeout=90 #Avoid spaces within parameters, they aren't worth the heartache diff --git a/config/boards/bench/arndale.json b/config/boards/bench/arndale.json deleted file mode 100644 index 705e940..0000000 --- a/config/boards/bench/arndale.json +++ /dev/null @@ -1,40 +0,0 @@ -{ - "job_name": "", - "device_type": "arndale", - "timeout": 691200, - "actions": [ - { - "command": "deploy_linaro_image", - "parameters": { - "image": "http://people.linaro.org/~bernie.ogden/arndale/arndale.img" - } - }, - { - "command": "lava_test_shell", - "parameters": { - "testdef_repos": [ - { - "git-repo": "http://git.linaro.org/toolchain/lavabench.git", - "testdef": "bench-session-debian.yaml", - "parameters": { - "GATEWAY": "10.0.0.1", - "PUB_KEY": "", - "LISTENER_ADDR": "", - "LISTENER_PORT": "" - } - } - ], - "timeout": 691200 - } - }, - { - "command": "submit_results", - "parameters": - { - "server": "", - "stream": "" - } - } - - ] -} diff --git a/config/boards/bench/juno-a53.conf b/config/boards/bench/juno-a53.conf index 4a8dfcb..7512aed 100644 --- a/config/boards/bench/juno-a53.conf +++ b/config/boards/bench/juno-a53.conf @@ -3,7 +3,7 @@ netctl=yes freq= benchcore=1 othercore=0 -ip=juno-a53.json +ip= boot_timeout=90 #Avoid spaces within parameters, they aren't worth the heartache diff --git a/config/boards/bench/juno-a53.json b/config/boards/bench/juno-a53.json deleted file mode 100644 index 937dd10..0000000 --- a/config/boards/bench/juno-a53.json +++ /dev/null @@ -1,41 +0,0 @@ -{ - "job_name": "", - "device_type": "juno", - "tags": [ "serial-console-server" ], - "timeout": 691200, - "actions": [ - { - "command": "deploy_linaro_image", - "parameters": { - "hwpack": "http://people.linaro.org/~bernie.ogden/hwpack_linaro-lt [...] - "rootfs": "http://people.linaro.org/~bernie.ogden/linaro-utopic-de [...] - } - }, - { - "command": "lava_test_shell", - "parameters": { - "testdef_repos": [ - { - "git-repo": "git://git.linaro.org/toolchain/lavabench.git", - "testdef": "bench-session-debian.yaml", - "parameters": { - "GATEWAY": "10.0.0.1", - "PUB_KEY": "", - "LISTENER_ADDR": "", - "LISTENER_PORT": "" - } - } - ], - "timeout": 691200 - } - }, - { - "command": "submit_results", - "parameters": { - "server": "", - "stream": "" - } - } - ] -} - diff --git a/config/boards/bench/juno-a57.conf b/config/boards/bench/juno-a57.conf index 59b9c13..3798f05 100644 --- a/config/boards/bench/juno-a57.conf +++ b/config/boards/bench/juno-a57.conf @@ -3,7 +3,7 @@ netctl=yes freq= benchcore=5 othercore=0 -ip=juno-a57.json +ip= boot_timeout=90 #Avoid spaces within parameters, they aren't worth the heartache diff --git a/config/boards/bench/juno-a57.json b/config/boards/bench/juno-a57.json deleted file mode 100644 index 937dd10..0000000 --- a/config/boards/bench/juno-a57.json +++ /dev/null @@ -1,41 +0,0 @@ -{ - "job_name": "", - "device_type": "juno", - "tags": [ "serial-console-server" ], - "timeout": 691200, - "actions": [ - { - "command": "deploy_linaro_image", - "parameters": { - "hwpack": "http://people.linaro.org/~bernie.ogden/hwpack_linaro-lt [...] - "rootfs": "http://people.linaro.org/~bernie.ogden/linaro-utopic-de [...] - } - }, - { - "command": "lava_test_shell", - "parameters": { - "testdef_repos": [ - { - "git-repo": "git://git.linaro.org/toolchain/lavabench.git", - "testdef": "bench-session-debian.yaml", - "parameters": { - "GATEWAY": "10.0.0.1", - "PUB_KEY": "", - "LISTENER_ADDR": "", - "LISTENER_PORT": "" - } - } - ], - "timeout": 691200 - } - }, - { - "command": "submit_results", - "parameters": { - "server": "", - "stream": "" - } - } - ] -} - diff --git a/config/boards/bench/kvm.conf b/config/boards/bench/kvm.conf index 96f6337..7c00803 100644 --- a/config/boards/bench/kvm.conf +++ b/config/boards/bench/kvm.conf @@ -7,7 +7,7 @@ netctl=no freq= benchcore= othercore= -ip=kvm.json +ip= boot_timeout=60 #Avoid spaces within parameters, they aren't worth the heartache diff --git a/config/boards/bench/kvm.json b/config/boards/bench/kvm.json deleted file mode 100644 index 39e9170..0000000 --- a/config/boards/bench/kvm.json +++ /dev/null @@ -1,39 +0,0 @@ -{ - "job_name": "", - "device_type": "kvm", - "timeout": 1800, - "actions": [ - { - "command": "deploy_linaro_image", - "parameters": { - "image": "http://images.validation.linaro.org/ubuntu-14-04-server- [...] - } - }, - { - "command": "lava_test_shell", - "parameters": { - "testdef_repos": [ - { - "git-repo": "git://git.linaro.org/toolchain/lavabench.git", - "testdef": "bench-session-debian.yaml", - "parameters": { - "GATEWAY": "10.0.0.1", - "PUB_KEY": "", - "LISTENER_ADDR": "", - "LISTENER_PORT": "" - } - } - ], - "timeout": 691200 - } - }, - { - "command": "submit_results", - "parameters": { - "server": "", - "stream": "" - } - } - ] -} - diff --git a/config/boards/bench/panda-es.conf b/config/boards/bench/panda-es.conf index 73580d5..ffe26a9 100644 --- a/config/boards/bench/panda-es.conf +++ b/config/boards/bench/panda-es.conf @@ -3,8 +3,7 @@ netctl=yes freq= benchcore=0 othercore=1 -ip=panda-es.json -#ip=linaro(a)10.2.201.83 +ip= boot_timeout=90 #Avoid spaces within parameters, they aren't worth the heartache diff --git a/config/boards/bench/panda-es.json b/config/boards/bench/panda-es.json deleted file mode 100644 index 602ebb9..0000000 --- a/config/boards/bench/panda-es.json +++ /dev/null @@ -1,41 +0,0 @@ -{ - "job_name": "", - "device_type": "panda-es", - "timeout": 691200, - "actions": [ - { - "command": "deploy_linaro_image", - "parameters": { - "hwpack": "http://releases.linaro.org/14.05/ubuntu/panda/hwpack_linaro- [...] - "rootfs": "http://releases.linaro.org/14.05/ubuntu/panda/linaro-trusty- [...] - } - }, - { - "command": "lava_test_shell", - "parameters": { - "testdef_repos": [ - { - "git-repo": "http://git.linaro.org/toolchain/lavabench.git", - "testdef": "bench-session-debian.yaml", - "parameters": { - "GATEWAY": "10.0.0.1", - "PUB_KEY": "", - "LISTENER_ADDR": "", - "LISTENER_PORT": "" - } - } - ], - "timeout": 691200 - } - }, - { - "command": "submit_results", - "parameters": - { - "server": "", - "stream": "" - } - } - - ] -} diff --git a/scripts/benchmark.sh b/scripts/benchmark.sh index d09703a..6b10b70 100755 --- a/scripts/benchmark.sh +++ b/scripts/benchmark.sh @@ -10,7 +10,6 @@ set -o pipefail set -o nounset #Make sure that subscripts clean up - we must not leave benchmark sources or data [...] -#we should not leave lava targets reserved trap clean_top EXIT trap 'error=1; exit' TERM INT HUP QUIT @@ -48,12 +47,11 @@ $0 [-tckh] -b <benchmark> <board...> be treated as an error, even if recoverable. On by default. -h Show this help. -k Keep. If this is set, benchmark sources and results will be left on - target. LAVA targets will not be released. + target. <board...> may be anything that has a file in config/boards/bench, e.g. the existence of arndale.conf means that you can put arndale here. At least one - target may be specified. ssh targets must only be specified once. LAVA - targets can be specified as many times as you like. + target may be specified. Each target must only be specified once. If building natively, board is optional. If not given, the benchmark will run on localhost. @@ -129,9 +127,7 @@ while getopts g:f:a:i:b:kpchs flag; do esac done shift $((OPTIND - 1)) -devices=("$@") #Duplicate targets are fine for lava, they will resolve to differen [...] - #Duplicate targets not fine for ssh access, where they will just re [...] - #TODO: Check for multiple instances of a given non-lava target +devices=("$@") if test x"${benchmark:-}" = x; then echo "No benchmark given (-b)" 1>&2 diff --git a/scripts/benchutil.sh b/scripts/benchutil.sh index 66a6c6d..94cf75a 100644 --- a/scripts/benchutil.sh +++ b/scripts/benchutil.sh @@ -31,57 +31,6 @@ function get_addr echo "${listener_addr}" | sed 's/^[[:blank:]]*//' | sed 's/[[:blank:]]*$//' } -#Return 0 if we're inside the lava network -#Return 1 if we're outside the lava network -#Return 2 if we don't know where we are -#Typically, 2 will be returned if we can't ssh into the hackbox. This tends -#to mean we're not configured to do so and could happen both inside and outside -#the LAVA network. -function lava_network -{ - local hackbox_mac - - hackbox_mac="`ssh -F /dev/null -o PasswordAuthentication=no -o PubkeyAuthenticat [...] - if test $? -ne 0; then - echo "Failed to get hackbox mac" 1>&2 - echo "Tried: ssh -F /dev/null -o PasswordAuthentication=no -o PubkeyAuthentica [...] - return 2 #We couldn't get the mac, stop trying to figure out where we are - fi - /usr/sbin/arp 10.0.0.10 | grep -q "${hackbox_mac}"; - if test $? -eq 0; then - return 0 - else - return 1 - fi -} - -function lava_user -{ - local usr="${USER}" - if echo "$1" | grep -q '^http'; then - echo "LAVA URL must exclude protocol (e.g. http://, https://)" 1>&2 - return 1 - fi - if echo "$1" | grep -Eq '^.+@'; then - usr="${1/@*}" - usr="${usr/:*}" - fi - echo "${usr}" -} - -function lava_server -{ - if echo "$1" | grep -q '^http'; then - echo "LAVA URL must exclude protocol (e.g. http://, https://)" 1>&2 - return 1 - fi - if echo "$1" | grep -Eq '^.+@'; then - echo "$1" | sed 's/[^@]*@//' - else - echo "$1" - fi -} - #Attempt to use read to discover whether there is a record to read from the producer #If we time out, check to see whether the producer still seems to be alive. #We can check more than one pid, if we have visibility of some other process(s) that we diff --git a/scripts/lava.sh b/scripts/lava.sh deleted file mode 100755 index 8bdc639..0000000 --- a/scripts/lava.sh +++ /dev/null @@ -1,356 +0,0 @@ -#!/bin/bash - -#Deps: lava-tool, auth token for lava-tool -set -o pipefail -set -o nounset -# load the configure file produced by configure -if test -e "${PWD}/host.conf"; then - . "${PWD}/host.conf" -else - echo "ERROR: no host.conf file! Did you run configure?" 1>&2 - exit 1 -fi -topdir="${abe_path}" #abe global, but this should be the right value for abe -. "${topdir}"/scripts/benchutil.sh -if test $? -ne 0; then - echo "Unable to source ${topdir}/benchutil.sh" 1>&2 - exit 1 -fi -listener_pid= -waiter= -keep=1 -error=1 -trap 'keep=0; exit ${error}' USR1 -trap release EXIT -trap 'exit ${error}' TERM INT HUP QUIT - -function retrying_lava_tool -{ - local c - for c in {4..0}; do - lava-tool "$@" && return 0 - if test $c -eq 0; then - return 1 - else - echo "lava-tool $@ failed" 1>&2 - echo "May be spurious: ${c} retries remaining" 1>&2 - sleep 5 - fi - done - return 1 -} - -function job_status -{ - if test x"${id:-}" = x -o x"${lava_url:-}" = x; then - echo "Job ID or LAVA URL not defined" 1>&2 - return 1 - fi - - local jobstatus - jobstatus="`retrying_lava_tool job-status https://${lava_url} ${id}`" - if test $? -ne 0; then - echo "Job ${id} disappeared!" 1>&2 - return 1 - fi - echo "${jobstatus}" | grep "Job Status: $1" > /dev/null - if test $? -ne 0; then - echo "Job ${id} has surprising status" 1>&2 - echo -e "${jobstatus}" 1>&2 - return 1 - fi - - return 0 -} - -release() -{ - if test x"${waiter:-}" != x; then - kill "${waiter}" 2>/dev/null - wait "${waiter}" - fi - if test x"${listener_pid:-}" != x; then - kill "${listener_pid}" 2>/dev/null - wait "${listener_pid}" - fi - if test x"${temps:-}" != x; then - if test -d "${temps}"; then - exec {listener_handle}>&- - exec {listener_handle}<&- - rm -rf "${temps}" - if test $? -ne 0; then - echo "Failed to delete temporary file store ${temps}" 1>&2 - fi - fi - fi - if test x"${id:-}" != x; then - if test ${keep} -eq 0; then - retrying_lava_tool cancel-job https://"${lava_url}" "${id}" - if test $? -eq 0; then - echo "Cancelled job ${id}" - error=0 - else - echo "Failed to cancel job ${id}" 1>&2 - echo "Run 'lava-tool cancel-job https://"${lava_url}" "${id}"' to cancel" 1>&2 - error=1 - fi - else - echo "Did not cancel job ${id} - superior did not request cancellation." - echo "You probably have some cleanup to do." - echo "When you've finished, cancel by running:" - echo "lava-tool cancel-job https://${lava_url} ${id}" - error=0 - fi - echo "Getting present job status:" - retrying_lava_tool job-status "https://${lava_url}" "${id}" - if test $? -ne 0; then - echo "Was unable to get job status" - fi - lava_url=${lava_url%/RPC2/} - echo "Log should be at: https://${lava_url#*@}/scheduler/job/${id}/log_file#bottom" - fi - exit "${error}" -} - -lava_url="${LAVA_SERVER:-}" -lava_server= -lava_user= -lava_json= -lava_job_name="tcwg-benchmark" -boot_timeout="$((120*60))" #2 hours -key=${LAVA_SSH_KEYFILE:-} -while getopts s:j:b:p:g: flag; do - case "${flag}" in - g) lava_job_name="${OPTARG}";; - s) lava_url="${OPTARG}";; - j) lava_json="${OPTARG}";; - b) boot_timeout="$((OPTARG*60))";; - p) key="${OPTARG}";; - *) - echo 'Unknown option' 1>&2 - exit 1 - ;; - esac -done - -expr ${lava_url:?'Must give a LAVA URL (-s) or set $LAVA_SERVER'} > /dev/null -lava_user="`lava_user ${lava_url}`" -if test $? -ne 0; then - echo "Unable to find username from ${lavaserver}" 1>&2 - exit 1 -fi -lava_server="`lava_server ${lava_url}`" -if test $? -ne 0; then - echo "Unable to find username from ${lavaserver}" 1>&2 - exit 1 -fi - -if ! test -f ${lava_json:?'Must give a json file (-j)'}; then - echo "JSON file ${lava_json} not a file" - exit 1 -fi - -shift $((OPTIND - 1)) -if test ${#@} -ne 0; then - echo -n "Unknown option(s): " 1>&2 - for opt in "$@"; do - echo -n " '${opt}'" 1>&2 - done - echo 1>&2 - exit 1 -fi - -#Store the public key in key - if no public key file given on CLI, try a few -#sensible defaults. (We only ever end up sharing a public key, so there's no -#security issue here.) -if test x"${key:-}" = x; then - if ssh-add -l; then - key="`ssh-add -L | head -n1 2>/dev/null`" - if test $? -ne 0; then - echo "Failed to get public key from ssh-agent" 1>&2 - exit 1 - fi - elif test -f ~/.ssh/id_rsa; then - key="`ssh-keygen -y -f ~/.ssh/id_rsa`" - if test $? -ne 0; then - echo "Failed to get public key from private key file" 1>&2 - exit 1 - fi - else - echo "Could not find a key to authenticate with target (tried ssh-agent, ~/.ss [...] - exit 1 - fi -else - if test -f "${key}"; then - #Build in a little protection against accidental private key publication - if head -n 1 "${key}" | grep PRIVATE > /dev/null; then - echo "Given key file appears to be a private key, will generate public key" 1>&2 - key="`ssh-keygen -y -f ${key}`" - if test $? -ne 0; then - echo "Failed to get public key from private key file" 1>&2 - exit 1 - fi - elif head -n 1 "${key}" | grep -v ^ssh-; then - echo "Given key file does not look like an ssh public key" 1>&2 - exit 1 - else - key="`cat ${key}`" - fi - else - echo "Public key file ${key} does not exist or is not a file" 1>&2 - exit 1 - fi -fi - -#Convert key into a sed-friendly replacement string (i.e. escape chars that are sp [...] -#Danger - Don't change the sed runes to backticks, they resolve differently and re [...] -key="$(set -f; echo ${key} | sed 's/[\/&]/\\&/g')" - -temps="`mktemp -dt XXXXXXXXX`" || exit 1 -listener_fifo="${temps}/listener_fifo" -mkfifo "${listener_fifo}" || exit 1 -exec {listener_handle}<>"${listener_fifo}" -json_copy="${temps}/job.json" -cp "${lava_json}" "${json_copy}" -sed -i "s/^$.*\"PUB_KEY\":$[^\"]*\".*\"[^,]*$,\?$[[:blank:]]*$/\1 \"${key}\"\2 [...] -if test $? -ne 0; then - echo "Failed to populate json file with public key" 1>&2 - exit 1 -fi -sed -i "s+^$.*\"job_name\":$[^\"]*\".*\"[^,]*$,\?$[[:blank:]]*\$+\1 \"${lava_j [...] -sed -i "s+^$.*\"server\":$[^\"]*\".*\"[^,]*$,\?$[[:blank:]]*\$+\1 \"https://${ [...] -sed -i "s+^$.*\"stream\":$[^\"]*\".*\"[^,]*$,\?$[[:blank:]]*\$+\1 \"/private/p [...] - -if test -z "${LAVA_IN_LAB}"; then - lava_network "${lava_user}" - in_lab=$? -else - in_lab=0 -fi -if test ${in_lab} -eq 2; then - echo "Unable to determine whether I am inside the LAVA lab, assuming that I am n [...] -fi - -listener_addr="`get_addr`" -if test $? -ne 0; then - echo "Unable to get IP for listener" 1>&2 - exit 1 -fi -if test ${in_lab} -eq 0; then - "${topdir}"/scripts/establish_listener.sh ${listener_addr} 4200 5200 >&"${listen [...] -else - "${topdir}"/scripts/establish_listener.sh -f 10.0.0.10:${lava_user}@lab.validati [...] -fi -listener_pid=$! -listener_addr="`bgread -T 60 ${listener_pid} <&${listener_handle}`" -if test $? -ne 0; then - echo "Failed to read listener address" 1>&2 - exit 1 -fi -listener_port="`bgread -T 60 ${listener_pid} <&${listener_handle}`" -if test $? -ne 0; then - echo "Failed to read listener port" 1>&2 - exit 1 -fi -echo "Listener ${listener_addr}:${listener_port}" - -sed -i "s/^$.*\"LISTENER_ADDR\":$[^\"]*\".*\"[^,]*$,\?$[[:blank:]]*$/\1 \"${li [...] -if test $? -ne 0; then - echo "Failed to populate json file with listener ip" 1>&2 - exit 1 -fi -sed -i "s/^$.*\"LISTENER_PORT\":$[^\"]*\".*\"[^,]*$,\?$[[:blank:]]*$/\1 \"${li [...] -if test $? -ne 0; then - echo "Failed to populate json file with listener port" 1>&2 - exit 1 -fi - -id="`retrying_lava_tool submit-job https://${lava_url} ${json_copy}`" -if test $? -ne 0; then - echo "Failed to submit job" 1>&2 - echo "URL: https://${lava_url}" 1>&2 - echo "JSON: " 1>&2 - cat "${json_copy}" 1>&2 - exit 1 -fi - -#TODO: Should be able to use cut at the end of this pipe, but when lava-tool -# is invoked through expect wrapper this line ends up with a carriage return -# at the end. Should be fixed on the expect side, or expect script should -# be discarded, but hack it here for now. -id="`echo ${id} | grep 'submitted as job id: [[:digit:]]\+' | grep -o '[[:digit:]]\+'`" -if test $? -ne 0; then - echo "Failed to read job id" 1>&2 - echo "Input string was: ${id}" 1>&2 - echo "JSON: " 1>&2 - cat "${json_copy}" 1>&2 - exit 1 -fi -echo "Dispatched LAVA job ${id}" - -sleep 15& #A short delay here is handy when debugging (if the LAVA queues are empt [...] -wait $! - -#Monitor job status until it starts running or fails -#TODO: This block assumes that lava_tool doesn't return until the job is in 'Submi [...] -#TODO: In principle we want a timeout here, but we could be queued for a very long [...] -while true; do - jobstatus="`retrying_lava_tool job-status https://${lava_url} ${id}`" - if test $? -ne 0; then - echo "Job ${id} disappeared!" 1>&2 - exit 1 - fi - echo "${jobstatus}" | grep 'Job Status: Running' > /dev/null - if test $? -eq 0; then - break - fi - echo "${jobstatus}" | grep 'Job Status: Submitted' > /dev/null - if test $? -ne 0; then - echo "Job ${id} has surprising status, giving up" 1>&2 - echo -e "${jobstatus}" 1>&2 - exit 1 - fi - sleep 60& - wait $! -done - -echo "Job ${id} is running, waiting for boot" - -deadline=$((`date +%s` + boot_timeout)) -while test `date +%s` -lt ${deadline}; do - read -t 60 user_ip <&"${listener_handle}" - err=$? - if test ${err} -eq 0; then - break - elif test ${err} -le 128; then - echo "Non-timeout error code ${err} while trying to read user_ip" 1>&2 - exit 1 - fi - if ! job_status 'Running'; then - echo "Bad status, giving up" 1>&2 - exit 1 - fi -done - -#We've finished with the listener -kill ${listener_pid} 2>/dev/null -#Don't wait on it - that way, when we kill and wait on exit, we know we're -#still dealing with the same process. (Killing it more than once is harmless.) - -if test x"${user_ip:-}" = x; then - echo "LAVA boot abandoned after $((boot_timeout/60)) minutes" 1>&2 - exit 1 -fi - -echo "LAVA target ready at ${user_ip}" - -#Wait to be killed, at which point we cancel the job -while true; do - sleep 600 & - waiter=$! - wait ${waiter} - if ! job_status Running; then - echo "Job ${id} has bad status, giving up" 1>&2 - exit 1 - fi -done diff --git a/scripts/runbenchmark.sh b/scripts/runbenchmark.sh index e966952..6e3c487 100755 --- a/scripts/runbenchmark.sh +++ b/scripts/runbenchmark.sh @@ -1,8 +1,6 @@ #!/bin/bash #This script takes a build of a benchmark and transfers it to, and runs it on -#a single target. The target may be LAVA or non-LAVA. The script is mostly -#LAVA-agnostic - apart from the section that initiates the LAVA session, there -#is some awareness of it in the exit handler, and that's all. +#a single target. set -o pipefail set -o nounset @@ -11,14 +9,12 @@ trap 'exit ${error}' TERM INT HUP QUIT tag= session_pid= -lava_pid= listener_pid= benchmark= device= keep= cautious='' build_dir= -lava_target= run_benchargs= while getopts g:b:d:t:a:kpc flag; do case "${flag}" in @@ -54,13 +50,6 @@ else fi topdir="${abe_path}" #abe global, but this should be the right value for abe confdir="${topdir}/config/boards/bench" -if test x"${LAVA_SERVER:-}" != x; then - lava_url="${LAVA_SERVER}" -else - lava_url="${USER}(a)validation.linaro.org/RPC2/" - echo "Environment variable LAVA_SERVER not set, defaulted to ${lava_url}" 1>&2 -fi - benchlog="`. ${abe_top}/host.conf && . ${topdir}/lib/common.sh && read_config ${be [...] if test $? -ne 0; then echo "Unable to read benchmark config file for ${benchmark}" 1>&2 @@ -86,14 +75,10 @@ fi temps="`mktemp -dt XXXXXXXXX`" || exit 1 listener_file="${temps}/listener_file" listener_fifo="${temps}/listener_fifo" -lava_fifo="${temps}/lava_fifo" mkfifo "${listener_fifo}" || exit 1 exec {listener_handle}<>${listener_fifo} -mkfifo "${lava_fifo}" || exit 1 -exec {lava_handle}<>"${lava_fifo}" #Make sure that subscripts clean up - we must not leave benchmark sources or data [...] -#we should not leave lava targets reserved clean_benchmark() { error=$? @@ -131,26 +116,9 @@ clean_benchmark() wait "${listener_pid}" fi - if test x"${lava_pid:-}" != x; then - if (test x"${keep}" = x && test ${error} -ne 0) || test x"${keep}" = 'x-k'; then - echo "Not killing lava session, to ensure session remains open for investiga [...] - kill "${lava_pid}" 2>/dev/null - wait "${lava_pid}" - else - kill -USR1 "${lava_pid}" 2>/dev/null - wait "${lava_pid}" - fi - - #Make sure we see any messages from the lava.sh handlers - dd iflag=nonblock <&${lava_handle} 2>/dev/null | awk "{print \"${lava_target}: [...] - fi - - #Delete these last so that we can still get messages through the lava fifo if test -d "${temps}"; then exec {listener_handle}>&- exec {listener_handle}<&- - exec {lava_handle}>&- - exec {lava_handle}<&- rm -rf "${temps}" if test $? -ne 0; then echo "Failed to delete ${temps}" 1>&2 @@ -161,66 +129,9 @@ clean_benchmark() exit "${error}" } -ssh_opts="-F /dev/null ${LAVA_SSH_KEYFILE:+-o IdentityFile=${LAVA_SSH_KEYFILE}}" +ssh_opts="-F /dev/null" establish_listener_opts= -#Handle LAVA case -echo "${ip}" | grep '\.json$' > /dev/null -if test $? -eq 0; then - lava_user="`lava_user ${lava_url}`" - if test $? -ne 0; then - echo "Unable to find username from ${lava_url}" 1>&2 - exit 1 - fi - if test -z "${LAVA_IN_LAB}"; then - lava_network "${lava_user}" - case $? in - 2) echo "Unable to determing location w.r.t. lava lab: assuming outside" 1>&2 ;; - 1) - gateway=lab.validation.linaro.org - ssh_opts="${ssh_opts} ProxyCommand='ssh ${lava_user}@${gateway} nc -q0 %h %p'" - establish_listener_opts="-f 10.0.0.10:${lava_user}@${gateway}" - esac - fi - lava_target="${ip}" - ip='' - tee_output=/dev/console - echo "Acquiring LAVA target ${lava_target}" - echo "${topdir}/scripts/lava.sh ${tag:+-g "${tag}"} -s ${lava_url} -j ${confdir} [...] - - ${topdir}/scripts/lava.sh ${tag:+-g "${tag}"} -s "${lava_url}" -j "${confdir}/${ [...] - if test $? -ne 0; then - echo "+++ Failed to acquire LAVA target ${lava_target}" 1>&2 - exit 1 - fi - lava_pid=$! - while true; do - line="`bgread -t 5 ${lava_pid} <&${lava_handle}`" - if test $? -ne 0; then - echo "${lava_target}: Failed to read lava output" 1>&2 - exit 1 - fi - echo "${lava_target}: $line" - if echo "${line}" | grep '^LAVA target ready at ' > /dev/null; then - ip="`echo ${line} | cut -d ' ' -f 5 | sed 's/\s*$//'`" - break - fi - done - #After this point, lava.sh should produce no output until we reach the exit handlers. - #Our exit handler checks the pipe from lava.sh before closing down. - - if test x"${ip:-}" = x; then - echo "+++ Failed to acquire LAVA target ${lava_target}" 1>&2 - exit 1 - fi -fi - -if test -z "${gateway:-}"; then - gateway="${ip/*@}" -fi -#LAVA-agnostic from here, apart from a section in the exit handler, and bgread -#monitoring of the LAVA process while we're waiting for the benchmark to end - #Set up our listener listener_addr="`get_addr`" if test $? -ne 0; then @@ -354,16 +265,10 @@ fi )& session_pid=$! -#lava_pid will expand to empty if we're not using lava -#lava_pid will expand to empty if we're not using lava #No sense in setting a deadline on this one, it's order of days for many cases -ip="`bgread ${listener_pid} ${lava_pid} <&${listener_handle}`" +ip="`bgread ${listener_pid} <&${listener_handle}`" if test $? -ne 0; then - if test x"${lava_pid:-}" = x; then - echo "Failed to read post-benchmark-run IP" 1>&2 - else - echo "LAVA process died, or otherwise failed while waiting to read post-benchm [...] - fi + echo "Failed to read post-benchmark-run IP" 1>&2 exit 1 fi -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

[toolchain/abe] 01/01: Do not delete results on exit

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a commit to branch bernie/jenkmarking-multinode in repository toolchain/abe. commit 8022cda231ac6b049876a56742cdcde15447f069 Author: Bernard Ogden <bernie.ogden(a)linaro.org> Date: Thu Jul 23 08:40:55 2015 +0200 Do not delete results on exit This script used to send results to a fileserver. This responsibility lies elsewhere in the system (at least for now) so stop it from deleting the results. Also lets us get rid of some mild complexity that had been required to support results-removal-via-exit-trap. Change-Id: I1a45dba71393fcb6c42ae678a07d21d131974eb5 --- scripts/Benchmark.job | 9 --------- 1 file changed, 9 deletions(-) diff --git a/scripts/Benchmark.job b/scripts/Benchmark.job index e8bd00d..b03ac99 100755 --- a/scripts/Benchmark.job +++ b/scripts/Benchmark.job @@ -7,9 +7,6 @@ set -o pipefail benchgccpath= targets=(${targets}) maindir="`mktemp --tmpdir=${HOME} -d bench-XXX`" -error=1 - -trap "rm -rf ${maindir}; exit \${error}" EXIT #A little parameter checking if test x"${toolchain:-}" = x; then @@ -58,10 +55,4 @@ fi cd "${maindir}" "${ABE_DIR}"/configure --with-fileserver=148.251.136.42 --with-remote-snapshots=/s [...] -set +e "${ABE_DIR}"/scripts/benchmark.sh ${run_flags:+-a "${run_flags//\"/\\\"}"} ${compi [...] -tmperr=$? -set -e -if test ${tmperr} -eq 0; then - error=0 -fi -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

[toolchain/abe] branch bernie/jenkmarking-multinode updated (21e3754 -> 8022cda)

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a change to branch bernie/jenkmarking-multinode in repository toolchain/abe. discards 21e3754 Strip out all references to lava new 8022cda Do not delete results on exit This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (21e3754) \ N -- N -- N refs/heads/bernie/jenkmarking-multinode (8022cda) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omits" are not gone; other references still refer to them. Any revisions marked "discards" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: config/boards/bench/arndale.conf | 3 +- config/boards/bench/arndale.json | 40 +++++ config/boards/bench/juno-a53.conf | 2 +- config/boards/bench/juno-a53.json | 41 +++++ config/boards/bench/juno-a57.conf | 2 +- config/boards/bench/juno-a57.json | 41 +++++ config/boards/bench/kvm.conf | 2 +- config/boards/bench/kvm.json | 39 +++++ config/boards/bench/panda-es.conf | 3 +- config/boards/bench/panda-es.json | 41 +++++ scripts/Benchmark.job | 9 - scripts/benchmark.sh | 10 +- scripts/benchutil.sh | 51 ++++++ scripts/lava.sh | 356 ++++++++++++++++++++++++++++++++++++++ scripts/runbenchmark.sh | 103 ++++++++++- 15 files changed, 722 insertions(+), 21 deletions(-) create mode 100644 config/boards/bench/arndale.json create mode 100644 config/boards/bench/juno-a53.json create mode 100644 config/boards/bench/juno-a57.json create mode 100644 config/boards/bench/kvm.json create mode 100644 config/boards/bench/panda-es.json create mode 100755 scripts/lava.sh -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

[toolchain/abe] branch bernie/jenkmarking-multinode updated: Strip out all references to lava

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a commit to branch bernie/jenkmarking-multinode in repository toolchain/abe. The following commit(s) were added to refs/heads/bernie/jenkmarking-multinode by th [...] new 21e3754 Strip out all references to lava 21e3754 is described below commit 21e3754ec46129f43cbcafad2b7cd37513965425 Author: Bernard Ogden <bernie.ogden(a)linaro.org> Date: Wed Jul 22 14:28:03 2015 +0200 Strip out all references to lava Change-Id: Ic970aedc7f32a32d4ca052cf8da9f399e146d5f8 --- config/boards/bench/arndale.conf | 3 +- config/boards/bench/arndale.json | 40 ----- config/boards/bench/juno-a53.conf | 2 +- config/boards/bench/juno-a53.json | 41 ----- config/boards/bench/juno-a57.conf | 2 +- config/boards/bench/juno-a57.json | 41 ----- config/boards/bench/kvm.conf | 2 +- config/boards/bench/kvm.json | 39 ----- config/boards/bench/panda-es.conf | 3 +- config/boards/bench/panda-es.json | 41 ----- scripts/benchmark.sh | 10 +- scripts/benchutil.sh | 51 ------ scripts/lava.sh | 356 -------------------------------------- scripts/runbenchmark.sh | 103 +---------- 14 files changed, 12 insertions(+), 722 deletions(-) diff --git a/config/boards/bench/arndale.conf b/config/boards/bench/arndale.conf index 46b1fb4..00644a4 100644 --- a/config/boards/bench/arndale.conf +++ b/config/boards/bench/arndale.conf @@ -3,8 +3,7 @@ netctl=yes freq=900MHz benchcore=0 othercore=1 -ip=arndale.json -#ip=linaro(a)10.2.201.83 +ip= boot_timeout=90 #Avoid spaces within parameters, they aren't worth the heartache diff --git a/config/boards/bench/arndale.json b/config/boards/bench/arndale.json deleted file mode 100644 index 705e940..0000000 --- a/config/boards/bench/arndale.json +++ /dev/null @@ -1,40 +0,0 @@ -{ - "job_name": "", - "device_type": "arndale", - "timeout": 691200, - "actions": [ - { - "command": "deploy_linaro_image", - "parameters": { - "image": "http://people.linaro.org/~bernie.ogden/arndale/arndale.img" - } - }, - { - "command": "lava_test_shell", - "parameters": { - "testdef_repos": [ - { - "git-repo": "http://git.linaro.org/toolchain/lavabench.git", - "testdef": "bench-session-debian.yaml", - "parameters": { - "GATEWAY": "10.0.0.1", - "PUB_KEY": "", - "LISTENER_ADDR": "", - "LISTENER_PORT": "" - } - } - ], - "timeout": 691200 - } - }, - { - "command": "submit_results", - "parameters": - { - "server": "", - "stream": "" - } - } - - ] -} diff --git a/config/boards/bench/juno-a53.conf b/config/boards/bench/juno-a53.conf index 4a8dfcb..7512aed 100644 --- a/config/boards/bench/juno-a53.conf +++ b/config/boards/bench/juno-a53.conf @@ -3,7 +3,7 @@ netctl=yes freq= benchcore=1 othercore=0 -ip=juno-a53.json +ip= boot_timeout=90 #Avoid spaces within parameters, they aren't worth the heartache diff --git a/config/boards/bench/juno-a53.json b/config/boards/bench/juno-a53.json deleted file mode 100644 index 937dd10..0000000 --- a/config/boards/bench/juno-a53.json +++ /dev/null @@ -1,41 +0,0 @@ -{ - "job_name": "", - "device_type": "juno", - "tags": [ "serial-console-server" ], - "timeout": 691200, - "actions": [ - { - "command": "deploy_linaro_image", - "parameters": { - "hwpack": "http://people.linaro.org/~bernie.ogden/hwpack_linaro-lt [...] - "rootfs": "http://people.linaro.org/~bernie.ogden/linaro-utopic-de [...] - } - }, - { - "command": "lava_test_shell", - "parameters": { - "testdef_repos": [ - { - "git-repo": "git://git.linaro.org/toolchain/lavabench.git", - "testdef": "bench-session-debian.yaml", - "parameters": { - "GATEWAY": "10.0.0.1", - "PUB_KEY": "", - "LISTENER_ADDR": "", - "LISTENER_PORT": "" - } - } - ], - "timeout": 691200 - } - }, - { - "command": "submit_results", - "parameters": { - "server": "", - "stream": "" - } - } - ] -} - diff --git a/config/boards/bench/juno-a57.conf b/config/boards/bench/juno-a57.conf index 59b9c13..3798f05 100644 --- a/config/boards/bench/juno-a57.conf +++ b/config/boards/bench/juno-a57.conf @@ -3,7 +3,7 @@ netctl=yes freq= benchcore=5 othercore=0 -ip=juno-a57.json +ip= boot_timeout=90 #Avoid spaces within parameters, they aren't worth the heartache diff --git a/config/boards/bench/juno-a57.json b/config/boards/bench/juno-a57.json deleted file mode 100644 index 937dd10..0000000 --- a/config/boards/bench/juno-a57.json +++ /dev/null @@ -1,41 +0,0 @@ -{ - "job_name": "", - "device_type": "juno", - "tags": [ "serial-console-server" ], - "timeout": 691200, - "actions": [ - { - "command": "deploy_linaro_image", - "parameters": { - "hwpack": "http://people.linaro.org/~bernie.ogden/hwpack_linaro-lt [...] - "rootfs": "http://people.linaro.org/~bernie.ogden/linaro-utopic-de [...] - } - }, - { - "command": "lava_test_shell", - "parameters": { - "testdef_repos": [ - { - "git-repo": "git://git.linaro.org/toolchain/lavabench.git", - "testdef": "bench-session-debian.yaml", - "parameters": { - "GATEWAY": "10.0.0.1", - "PUB_KEY": "", - "LISTENER_ADDR": "", - "LISTENER_PORT": "" - } - } - ], - "timeout": 691200 - } - }, - { - "command": "submit_results", - "parameters": { - "server": "", - "stream": "" - } - } - ] -} - diff --git a/config/boards/bench/kvm.conf b/config/boards/bench/kvm.conf index 96f6337..7c00803 100644 --- a/config/boards/bench/kvm.conf +++ b/config/boards/bench/kvm.conf @@ -7,7 +7,7 @@ netctl=no freq= benchcore= othercore= -ip=kvm.json +ip= boot_timeout=60 #Avoid spaces within parameters, they aren't worth the heartache diff --git a/config/boards/bench/kvm.json b/config/boards/bench/kvm.json deleted file mode 100644 index 39e9170..0000000 --- a/config/boards/bench/kvm.json +++ /dev/null @@ -1,39 +0,0 @@ -{ - "job_name": "", - "device_type": "kvm", - "timeout": 1800, - "actions": [ - { - "command": "deploy_linaro_image", - "parameters": { - "image": "http://images.validation.linaro.org/ubuntu-14-04-server- [...] - } - }, - { - "command": "lava_test_shell", - "parameters": { - "testdef_repos": [ - { - "git-repo": "git://git.linaro.org/toolchain/lavabench.git", - "testdef": "bench-session-debian.yaml", - "parameters": { - "GATEWAY": "10.0.0.1", - "PUB_KEY": "", - "LISTENER_ADDR": "", - "LISTENER_PORT": "" - } - } - ], - "timeout": 691200 - } - }, - { - "command": "submit_results", - "parameters": { - "server": "", - "stream": "" - } - } - ] -} - diff --git a/config/boards/bench/panda-es.conf b/config/boards/bench/panda-es.conf index 73580d5..ffe26a9 100644 --- a/config/boards/bench/panda-es.conf +++ b/config/boards/bench/panda-es.conf @@ -3,8 +3,7 @@ netctl=yes freq= benchcore=0 othercore=1 -ip=panda-es.json -#ip=linaro(a)10.2.201.83 +ip= boot_timeout=90 #Avoid spaces within parameters, they aren't worth the heartache diff --git a/config/boards/bench/panda-es.json b/config/boards/bench/panda-es.json deleted file mode 100644 index 602ebb9..0000000 --- a/config/boards/bench/panda-es.json +++ /dev/null @@ -1,41 +0,0 @@ -{ - "job_name": "", - "device_type": "panda-es", - "timeout": 691200, - "actions": [ - { - "command": "deploy_linaro_image", - "parameters": { - "hwpack": "http://releases.linaro.org/14.05/ubuntu/panda/hwpack_linaro- [...] - "rootfs": "http://releases.linaro.org/14.05/ubuntu/panda/linaro-trusty- [...] - } - }, - { - "command": "lava_test_shell", - "parameters": { - "testdef_repos": [ - { - "git-repo": "http://git.linaro.org/toolchain/lavabench.git", - "testdef": "bench-session-debian.yaml", - "parameters": { - "GATEWAY": "10.0.0.1", - "PUB_KEY": "", - "LISTENER_ADDR": "", - "LISTENER_PORT": "" - } - } - ], - "timeout": 691200 - } - }, - { - "command": "submit_results", - "parameters": - { - "server": "", - "stream": "" - } - } - - ] -} diff --git a/scripts/benchmark.sh b/scripts/benchmark.sh index d09703a..6b10b70 100755 --- a/scripts/benchmark.sh +++ b/scripts/benchmark.sh @@ -10,7 +10,6 @@ set -o pipefail set -o nounset #Make sure that subscripts clean up - we must not leave benchmark sources or data [...] -#we should not leave lava targets reserved trap clean_top EXIT trap 'error=1; exit' TERM INT HUP QUIT @@ -48,12 +47,11 @@ $0 [-tckh] -b <benchmark> <board...> be treated as an error, even if recoverable. On by default. -h Show this help. -k Keep. If this is set, benchmark sources and results will be left on - target. LAVA targets will not be released. + target. <board...> may be anything that has a file in config/boards/bench, e.g. the existence of arndale.conf means that you can put arndale here. At least one - target may be specified. ssh targets must only be specified once. LAVA - targets can be specified as many times as you like. + target may be specified. Each target must only be specified once. If building natively, board is optional. If not given, the benchmark will run on localhost. @@ -129,9 +127,7 @@ while getopts g:f:a:i:b:kpchs flag; do esac done shift $((OPTIND - 1)) -devices=("$@") #Duplicate targets are fine for lava, they will resolve to differen [...] - #Duplicate targets not fine for ssh access, where they will just re [...] - #TODO: Check for multiple instances of a given non-lava target +devices=("$@") if test x"${benchmark:-}" = x; then echo "No benchmark given (-b)" 1>&2 diff --git a/scripts/benchutil.sh b/scripts/benchutil.sh index 66a6c6d..94cf75a 100644 --- a/scripts/benchutil.sh +++ b/scripts/benchutil.sh @@ -31,57 +31,6 @@ function get_addr echo "${listener_addr}" | sed 's/^[[:blank:]]*//' | sed 's/[[:blank:]]*$//' } -#Return 0 if we're inside the lava network -#Return 1 if we're outside the lava network -#Return 2 if we don't know where we are -#Typically, 2 will be returned if we can't ssh into the hackbox. This tends -#to mean we're not configured to do so and could happen both inside and outside -#the LAVA network. -function lava_network -{ - local hackbox_mac - - hackbox_mac="`ssh -F /dev/null -o PasswordAuthentication=no -o PubkeyAuthenticat [...] - if test $? -ne 0; then - echo "Failed to get hackbox mac" 1>&2 - echo "Tried: ssh -F /dev/null -o PasswordAuthentication=no -o PubkeyAuthentica [...] - return 2 #We couldn't get the mac, stop trying to figure out where we are - fi - /usr/sbin/arp 10.0.0.10 | grep -q "${hackbox_mac}"; - if test $? -eq 0; then - return 0 - else - return 1 - fi -} - -function lava_user -{ - local usr="${USER}" - if echo "$1" | grep -q '^http'; then - echo "LAVA URL must exclude protocol (e.g. http://, https://)" 1>&2 - return 1 - fi - if echo "$1" | grep -Eq '^.+@'; then - usr="${1/@*}" - usr="${usr/:*}" - fi - echo "${usr}" -} - -function lava_server -{ - if echo "$1" | grep -q '^http'; then - echo "LAVA URL must exclude protocol (e.g. http://, https://)" 1>&2 - return 1 - fi - if echo "$1" | grep -Eq '^.+@'; then - echo "$1" | sed 's/[^@]*@//' - else - echo "$1" - fi -} - #Attempt to use read to discover whether there is a record to read from the producer #If we time out, check to see whether the producer still seems to be alive. #We can check more than one pid, if we have visibility of some other process(s) that we diff --git a/scripts/lava.sh b/scripts/lava.sh deleted file mode 100755 index 8bdc639..0000000 --- a/scripts/lava.sh +++ /dev/null @@ -1,356 +0,0 @@ -#!/bin/bash - -#Deps: lava-tool, auth token for lava-tool -set -o pipefail -set -o nounset -# load the configure file produced by configure -if test -e "${PWD}/host.conf"; then - . "${PWD}/host.conf" -else - echo "ERROR: no host.conf file! Did you run configure?" 1>&2 - exit 1 -fi -topdir="${abe_path}" #abe global, but this should be the right value for abe -. "${topdir}"/scripts/benchutil.sh -if test $? -ne 0; then - echo "Unable to source ${topdir}/benchutil.sh" 1>&2 - exit 1 -fi -listener_pid= -waiter= -keep=1 -error=1 -trap 'keep=0; exit ${error}' USR1 -trap release EXIT -trap 'exit ${error}' TERM INT HUP QUIT - -function retrying_lava_tool -{ - local c - for c in {4..0}; do - lava-tool "$@" && return 0 - if test $c -eq 0; then - return 1 - else - echo "lava-tool $@ failed" 1>&2 - echo "May be spurious: ${c} retries remaining" 1>&2 - sleep 5 - fi - done - return 1 -} - -function job_status -{ - if test x"${id:-}" = x -o x"${lava_url:-}" = x; then - echo "Job ID or LAVA URL not defined" 1>&2 - return 1 - fi - - local jobstatus - jobstatus="`retrying_lava_tool job-status https://${lava_url} ${id}`" - if test $? -ne 0; then - echo "Job ${id} disappeared!" 1>&2 - return 1 - fi - echo "${jobstatus}" | grep "Job Status: $1" > /dev/null - if test $? -ne 0; then - echo "Job ${id} has surprising status" 1>&2 - echo -e "${jobstatus}" 1>&2 - return 1 - fi - - return 0 -} - -release() -{ - if test x"${waiter:-}" != x; then - kill "${waiter}" 2>/dev/null - wait "${waiter}" - fi - if test x"${listener_pid:-}" != x; then - kill "${listener_pid}" 2>/dev/null - wait "${listener_pid}" - fi - if test x"${temps:-}" != x; then - if test -d "${temps}"; then - exec {listener_handle}>&- - exec {listener_handle}<&- - rm -rf "${temps}" - if test $? -ne 0; then - echo "Failed to delete temporary file store ${temps}" 1>&2 - fi - fi - fi - if test x"${id:-}" != x; then - if test ${keep} -eq 0; then - retrying_lava_tool cancel-job https://"${lava_url}" "${id}" - if test $? -eq 0; then - echo "Cancelled job ${id}" - error=0 - else - echo "Failed to cancel job ${id}" 1>&2 - echo "Run 'lava-tool cancel-job https://"${lava_url}" "${id}"' to cancel" 1>&2 - error=1 - fi - else - echo "Did not cancel job ${id} - superior did not request cancellation." - echo "You probably have some cleanup to do." - echo "When you've finished, cancel by running:" - echo "lava-tool cancel-job https://${lava_url} ${id}" - error=0 - fi - echo "Getting present job status:" - retrying_lava_tool job-status "https://${lava_url}" "${id}" - if test $? -ne 0; then - echo "Was unable to get job status" - fi - lava_url=${lava_url%/RPC2/} - echo "Log should be at: https://${lava_url#*@}/scheduler/job/${id}/log_file#bottom" - fi - exit "${error}" -} - -lava_url="${LAVA_SERVER:-}" -lava_server= -lava_user= -lava_json= -lava_job_name="tcwg-benchmark" -boot_timeout="$((120*60))" #2 hours -key=${LAVA_SSH_KEYFILE:-} -while getopts s:j:b:p:g: flag; do - case "${flag}" in - g) lava_job_name="${OPTARG}";; - s) lava_url="${OPTARG}";; - j) lava_json="${OPTARG}";; - b) boot_timeout="$((OPTARG*60))";; - p) key="${OPTARG}";; - *) - echo 'Unknown option' 1>&2 - exit 1 - ;; - esac -done - -expr ${lava_url:?'Must give a LAVA URL (-s) or set $LAVA_SERVER'} > /dev/null -lava_user="`lava_user ${lava_url}`" -if test $? -ne 0; then - echo "Unable to find username from ${lavaserver}" 1>&2 - exit 1 -fi -lava_server="`lava_server ${lava_url}`" -if test $? -ne 0; then - echo "Unable to find username from ${lavaserver}" 1>&2 - exit 1 -fi - -if ! test -f ${lava_json:?'Must give a json file (-j)'}; then - echo "JSON file ${lava_json} not a file" - exit 1 -fi - -shift $((OPTIND - 1)) -if test ${#@} -ne 0; then - echo -n "Unknown option(s): " 1>&2 - for opt in "$@"; do - echo -n " '${opt}'" 1>&2 - done - echo 1>&2 - exit 1 -fi - -#Store the public key in key - if no public key file given on CLI, try a few -#sensible defaults. (We only ever end up sharing a public key, so there's no -#security issue here.) -if test x"${key:-}" = x; then - if ssh-add -l; then - key="`ssh-add -L | head -n1 2>/dev/null`" - if test $? -ne 0; then - echo "Failed to get public key from ssh-agent" 1>&2 - exit 1 - fi - elif test -f ~/.ssh/id_rsa; then - key="`ssh-keygen -y -f ~/.ssh/id_rsa`" - if test $? -ne 0; then - echo "Failed to get public key from private key file" 1>&2 - exit 1 - fi - else - echo "Could not find a key to authenticate with target (tried ssh-agent, ~/.ss [...] - exit 1 - fi -else - if test -f "${key}"; then - #Build in a little protection against accidental private key publication - if head -n 1 "${key}" | grep PRIVATE > /dev/null; then - echo "Given key file appears to be a private key, will generate public key" 1>&2 - key="`ssh-keygen -y -f ${key}`" - if test $? -ne 0; then - echo "Failed to get public key from private key file" 1>&2 - exit 1 - fi - elif head -n 1 "${key}" | grep -v ^ssh-; then - echo "Given key file does not look like an ssh public key" 1>&2 - exit 1 - else - key="`cat ${key}`" - fi - else - echo "Public key file ${key} does not exist or is not a file" 1>&2 - exit 1 - fi -fi - -#Convert key into a sed-friendly replacement string (i.e. escape chars that are sp [...] -#Danger - Don't change the sed runes to backticks, they resolve differently and re [...] -key="$(set -f; echo ${key} | sed 's/[\/&]/\\&/g')" - -temps="`mktemp -dt XXXXXXXXX`" || exit 1 -listener_fifo="${temps}/listener_fifo" -mkfifo "${listener_fifo}" || exit 1 -exec {listener_handle}<>"${listener_fifo}" -json_copy="${temps}/job.json" -cp "${lava_json}" "${json_copy}" -sed -i "s/^$.*\"PUB_KEY\":$[^\"]*\".*\"[^,]*$,\?$[[:blank:]]*$/\1 \"${key}\"\2 [...] -if test $? -ne 0; then - echo "Failed to populate json file with public key" 1>&2 - exit 1 -fi -sed -i "s+^$.*\"job_name\":$[^\"]*\".*\"[^,]*$,\?$[[:blank:]]*\$+\1 \"${lava_j [...] -sed -i "s+^$.*\"server\":$[^\"]*\".*\"[^,]*$,\?$[[:blank:]]*\$+\1 \"https://${ [...] -sed -i "s+^$.*\"stream\":$[^\"]*\".*\"[^,]*$,\?$[[:blank:]]*\$+\1 \"/private/p [...] - -if test -z "${LAVA_IN_LAB}"; then - lava_network "${lava_user}" - in_lab=$? -else - in_lab=0 -fi -if test ${in_lab} -eq 2; then - echo "Unable to determine whether I am inside the LAVA lab, assuming that I am n [...] -fi - -listener_addr="`get_addr`" -if test $? -ne 0; then - echo "Unable to get IP for listener" 1>&2 - exit 1 -fi -if test ${in_lab} -eq 0; then - "${topdir}"/scripts/establish_listener.sh ${listener_addr} 4200 5200 >&"${listen [...] -else - "${topdir}"/scripts/establish_listener.sh -f 10.0.0.10:${lava_user}@lab.validati [...] -fi -listener_pid=$! -listener_addr="`bgread -T 60 ${listener_pid} <&${listener_handle}`" -if test $? -ne 0; then - echo "Failed to read listener address" 1>&2 - exit 1 -fi -listener_port="`bgread -T 60 ${listener_pid} <&${listener_handle}`" -if test $? -ne 0; then - echo "Failed to read listener port" 1>&2 - exit 1 -fi -echo "Listener ${listener_addr}:${listener_port}" - -sed -i "s/^$.*\"LISTENER_ADDR\":$[^\"]*\".*\"[^,]*$,\?$[[:blank:]]*$/\1 \"${li [...] -if test $? -ne 0; then - echo "Failed to populate json file with listener ip" 1>&2 - exit 1 -fi -sed -i "s/^$.*\"LISTENER_PORT\":$[^\"]*\".*\"[^,]*$,\?$[[:blank:]]*$/\1 \"${li [...] -if test $? -ne 0; then - echo "Failed to populate json file with listener port" 1>&2 - exit 1 -fi - -id="`retrying_lava_tool submit-job https://${lava_url} ${json_copy}`" -if test $? -ne 0; then - echo "Failed to submit job" 1>&2 - echo "URL: https://${lava_url}" 1>&2 - echo "JSON: " 1>&2 - cat "${json_copy}" 1>&2 - exit 1 -fi - -#TODO: Should be able to use cut at the end of this pipe, but when lava-tool -# is invoked through expect wrapper this line ends up with a carriage return -# at the end. Should be fixed on the expect side, or expect script should -# be discarded, but hack it here for now. -id="`echo ${id} | grep 'submitted as job id: [[:digit:]]\+' | grep -o '[[:digit:]]\+'`" -if test $? -ne 0; then - echo "Failed to read job id" 1>&2 - echo "Input string was: ${id}" 1>&2 - echo "JSON: " 1>&2 - cat "${json_copy}" 1>&2 - exit 1 -fi -echo "Dispatched LAVA job ${id}" - -sleep 15& #A short delay here is handy when debugging (if the LAVA queues are empt [...] -wait $! - -#Monitor job status until it starts running or fails -#TODO: This block assumes that lava_tool doesn't return until the job is in 'Submi [...] -#TODO: In principle we want a timeout here, but we could be queued for a very long [...] -while true; do - jobstatus="`retrying_lava_tool job-status https://${lava_url} ${id}`" - if test $? -ne 0; then - echo "Job ${id} disappeared!" 1>&2 - exit 1 - fi - echo "${jobstatus}" | grep 'Job Status: Running' > /dev/null - if test $? -eq 0; then - break - fi - echo "${jobstatus}" | grep 'Job Status: Submitted' > /dev/null - if test $? -ne 0; then - echo "Job ${id} has surprising status, giving up" 1>&2 - echo -e "${jobstatus}" 1>&2 - exit 1 - fi - sleep 60& - wait $! -done - -echo "Job ${id} is running, waiting for boot" - -deadline=$((`date +%s` + boot_timeout)) -while test `date +%s` -lt ${deadline}; do - read -t 60 user_ip <&"${listener_handle}" - err=$? - if test ${err} -eq 0; then - break - elif test ${err} -le 128; then - echo "Non-timeout error code ${err} while trying to read user_ip" 1>&2 - exit 1 - fi - if ! job_status 'Running'; then - echo "Bad status, giving up" 1>&2 - exit 1 - fi -done - -#We've finished with the listener -kill ${listener_pid} 2>/dev/null -#Don't wait on it - that way, when we kill and wait on exit, we know we're -#still dealing with the same process. (Killing it more than once is harmless.) - -if test x"${user_ip:-}" = x; then - echo "LAVA boot abandoned after $((boot_timeout/60)) minutes" 1>&2 - exit 1 -fi - -echo "LAVA target ready at ${user_ip}" - -#Wait to be killed, at which point we cancel the job -while true; do - sleep 600 & - waiter=$! - wait ${waiter} - if ! job_status Running; then - echo "Job ${id} has bad status, giving up" 1>&2 - exit 1 - fi -done diff --git a/scripts/runbenchmark.sh b/scripts/runbenchmark.sh index e966952..6e3c487 100755 --- a/scripts/runbenchmark.sh +++ b/scripts/runbenchmark.sh @@ -1,8 +1,6 @@ #!/bin/bash #This script takes a build of a benchmark and transfers it to, and runs it on -#a single target. The target may be LAVA or non-LAVA. The script is mostly -#LAVA-agnostic - apart from the section that initiates the LAVA session, there -#is some awareness of it in the exit handler, and that's all. +#a single target. set -o pipefail set -o nounset @@ -11,14 +9,12 @@ trap 'exit ${error}' TERM INT HUP QUIT tag= session_pid= -lava_pid= listener_pid= benchmark= device= keep= cautious='' build_dir= -lava_target= run_benchargs= while getopts g:b:d:t:a:kpc flag; do case "${flag}" in @@ -54,13 +50,6 @@ else fi topdir="${abe_path}" #abe global, but this should be the right value for abe confdir="${topdir}/config/boards/bench" -if test x"${LAVA_SERVER:-}" != x; then - lava_url="${LAVA_SERVER}" -else - lava_url="${USER}(a)validation.linaro.org/RPC2/" - echo "Environment variable LAVA_SERVER not set, defaulted to ${lava_url}" 1>&2 -fi - benchlog="`. ${abe_top}/host.conf && . ${topdir}/lib/common.sh && read_config ${be [...] if test $? -ne 0; then echo "Unable to read benchmark config file for ${benchmark}" 1>&2 @@ -86,14 +75,10 @@ fi temps="`mktemp -dt XXXXXXXXX`" || exit 1 listener_file="${temps}/listener_file" listener_fifo="${temps}/listener_fifo" -lava_fifo="${temps}/lava_fifo" mkfifo "${listener_fifo}" || exit 1 exec {listener_handle}<>${listener_fifo} -mkfifo "${lava_fifo}" || exit 1 -exec {lava_handle}<>"${lava_fifo}" #Make sure that subscripts clean up - we must not leave benchmark sources or data [...] -#we should not leave lava targets reserved clean_benchmark() { error=$? @@ -131,26 +116,9 @@ clean_benchmark() wait "${listener_pid}" fi - if test x"${lava_pid:-}" != x; then - if (test x"${keep}" = x && test ${error} -ne 0) || test x"${keep}" = 'x-k'; then - echo "Not killing lava session, to ensure session remains open for investiga [...] - kill "${lava_pid}" 2>/dev/null - wait "${lava_pid}" - else - kill -USR1 "${lava_pid}" 2>/dev/null - wait "${lava_pid}" - fi - - #Make sure we see any messages from the lava.sh handlers - dd iflag=nonblock <&${lava_handle} 2>/dev/null | awk "{print \"${lava_target}: [...] - fi - - #Delete these last so that we can still get messages through the lava fifo if test -d "${temps}"; then exec {listener_handle}>&- exec {listener_handle}<&- - exec {lava_handle}>&- - exec {lava_handle}<&- rm -rf "${temps}" if test $? -ne 0; then echo "Failed to delete ${temps}" 1>&2 @@ -161,66 +129,9 @@ clean_benchmark() exit "${error}" } -ssh_opts="-F /dev/null ${LAVA_SSH_KEYFILE:+-o IdentityFile=${LAVA_SSH_KEYFILE}}" +ssh_opts="-F /dev/null" establish_listener_opts= -#Handle LAVA case -echo "${ip}" | grep '\.json$' > /dev/null -if test $? -eq 0; then - lava_user="`lava_user ${lava_url}`" - if test $? -ne 0; then - echo "Unable to find username from ${lava_url}" 1>&2 - exit 1 - fi - if test -z "${LAVA_IN_LAB}"; then - lava_network "${lava_user}" - case $? in - 2) echo "Unable to determing location w.r.t. lava lab: assuming outside" 1>&2 ;; - 1) - gateway=lab.validation.linaro.org - ssh_opts="${ssh_opts} ProxyCommand='ssh ${lava_user}@${gateway} nc -q0 %h %p'" - establish_listener_opts="-f 10.0.0.10:${lava_user}@${gateway}" - esac - fi - lava_target="${ip}" - ip='' - tee_output=/dev/console - echo "Acquiring LAVA target ${lava_target}" - echo "${topdir}/scripts/lava.sh ${tag:+-g "${tag}"} -s ${lava_url} -j ${confdir} [...] - - ${topdir}/scripts/lava.sh ${tag:+-g "${tag}"} -s "${lava_url}" -j "${confdir}/${ [...] - if test $? -ne 0; then - echo "+++ Failed to acquire LAVA target ${lava_target}" 1>&2 - exit 1 - fi - lava_pid=$! - while true; do - line="`bgread -t 5 ${lava_pid} <&${lava_handle}`" - if test $? -ne 0; then - echo "${lava_target}: Failed to read lava output" 1>&2 - exit 1 - fi - echo "${lava_target}: $line" - if echo "${line}" | grep '^LAVA target ready at ' > /dev/null; then - ip="`echo ${line} | cut -d ' ' -f 5 | sed 's/\s*$//'`" - break - fi - done - #After this point, lava.sh should produce no output until we reach the exit handlers. - #Our exit handler checks the pipe from lava.sh before closing down. - - if test x"${ip:-}" = x; then - echo "+++ Failed to acquire LAVA target ${lava_target}" 1>&2 - exit 1 - fi -fi - -if test -z "${gateway:-}"; then - gateway="${ip/*@}" -fi -#LAVA-agnostic from here, apart from a section in the exit handler, and bgread -#monitoring of the LAVA process while we're waiting for the benchmark to end - #Set up our listener listener_addr="`get_addr`" if test $? -ne 0; then @@ -354,16 +265,10 @@ fi )& session_pid=$! -#lava_pid will expand to empty if we're not using lava -#lava_pid will expand to empty if we're not using lava #No sense in setting a deadline on this one, it's order of days for many cases -ip="`bgread ${listener_pid} ${lava_pid} <&${listener_handle}`" +ip="`bgread ${listener_pid} <&${listener_handle}`" if test $? -ne 0; then - if test x"${lava_pid:-}" = x; then - echo "Failed to read post-benchmark-run IP" 1>&2 - else - echo "LAVA process died, or otherwise failed while waiting to read post-benchm [...] - fi + echo "Failed to read post-benchmark-run IP" 1>&2 exit 1 fi -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

[toolchain/release-notes] branch releases/linaro-4.9-2015.05 created (now 004d0fd)

by git

This is an automated email from the git hooks/post-receive script. ryan.arnold pushed a change to branch releases/linaro-4.9-2015.05 in repository toolchain/release-notes. at 004d0fd Create release notes for release linaro-4.9-2015.05. This branch includes the following new commits: new 004d0fd Create release notes for release linaro-4.9-2015.05. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

Tcwg-commits July 2015