Tcwg-commits July 2015

tcwg-commits@lists.linaro.org

2 participants
1355 discussions

Start a nNew thread

[toolchain/abe] 02/02: More output when we trap errors for debug

by git

9 years, 3 months

[gcc] branch trunk updated (9d8e046 -> aa0bf07)

by git

This is an automated email from the git hooks/post-receive script. unknown user pushed a change to branch trunk in repository gcc. from 9d8e046 [AArch64] Properly handle simple arith+extend ops in rtx costs new 39e21ca 2015-07-28 Richard Biener <rguenther(a)suse.de> new aa0bf07 2015-07-28 Richard Biener <rguenther(a)suse.de> The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: gcc/ChangeLog | 9 +++++++++ gcc/match.pd | 52 ++++++++++++++++++++++++++++++++++++++++++++++------ 2 files changed, 55 insertions(+), 6 deletions(-) -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 3 months

[gcc] branch master updated (9d8e046 -> aa0bf07)

by git

This is an automated email from the git hooks/post-receive script. unknown user pushed a change to branch master in repository gcc. from 9d8e046 [AArch64] Properly handle simple arith+extend ops in rtx costs new 39e21ca 2015-07-28 Richard Biener <rguenther(a)suse.de> new aa0bf07 2015-07-28 Richard Biener <rguenther(a)suse.de> The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: gcc/ChangeLog | 9 +++++++++ gcc/match.pd | 52 ++++++++++++++++++++++++++++++++++++++++++++++------ 2 files changed, 55 insertions(+), 6 deletions(-) -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 3 months

[gcc] branch trunk updated (66ed189 -> 9d8e046)

by git

This is an automated email from the git hooks/post-receive script. unknown user pushed a change to branch trunk in repository gcc. from 66ed189 2015-07-28 Paolo Carlini <paolo.carlini(a)oracle.com> new 9d8e046 [AArch64] Properly handle simple arith+extend ops in rtx costs The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: gcc/ChangeLog | 7 +++++++ gcc/config/aarch64/aarch64.c | 11 +++++++++-- 2 files changed, 16 insertions(+), 2 deletions(-) -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 3 months

[toolchain/abe] 01/02: LAVA configuration

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a commit to branch bernie/jenkmarking-multinode in repository toolchain/abe. commit 10136947d340a9120e6d8172b7f4dea4e44811cb Author: Bernard Ogden <bernie.ogden(a)linaro.org> Date: Tue Jul 28 11:57:18 2015 +0200 LAVA configuration Based on git.linaro.org/toolchain/semiautobench (host side) and git.linaro.org/toolchain/lavabench (target side), with alterations to clean out remaining hacking-session-isms. git.linaro.org/toolchain/semiautobench: c90fe30dbe0af4a41349bcaf1e7b051cc5c11546 git.linaro.org/toolchain/lavabench: c48e5f8503b61d8af7fa738c1fd1ac46ca2250b4 Change-Id: I827d445d4d3436654627bdaf76ab6778beb66e7b --- config/bench/lava/host-session | 142 ++++++++++++++++++++++++++++++++++ config/bench/lava/host-session.yaml | 39 ++++++++++ config/bench/lava/setup | 10 +++ config/bench/lava/target-session | 3 + config/bench/lava/target-session.yaml | 24 ++++++ 5 files changed, 218 insertions(+) diff --git a/config/bench/lava/host-session b/config/bench/lava/host-session new file mode 100755 index 0000000..b50fe56 --- /dev/null +++ b/config/bench/lava/host-session @@ -0,0 +1,142 @@ +#!/bin/bash +set -u +set -o pipefail + +error=1 + +function exitfunc { + if test ${error} -ne 0; then + if test x"${BENCH_DEBUG}" != x && test x"${BENCH_DEBUG}" != xNone; then + sleep infinity + fi + fi + rm -rf ~/* + exit ${error} +} + +trap exitfunc EXIT + +abe_branch="$1" + +#these parameters used by Benchmark.job +export benchmark="${2:?Must set a benchmark}" +export toolchain="${3:?Must set a toolchain}" +if test x"$4" = xNone; then + export run_flags= +else + export run_flags="$4" +fi +if test x"$5" = xNone; then + export compiler_flags= +else + export compiler_flags="$5" +fi + +#Create directory to store src +if ! (mkdir ~/benchsrc && chmod 700 ~/benchsrc); then + echo "Failed to create directory for source" >&2 + exit 1 +fi + +#Initialize git-new-workdir - sadly not just an apt-get call +ln /usr/share/doc/git/contrib/workdir/git-new-workdir /usr/local/bin +chmod 755 /usr/local/bin/git-new-workdir + +#Get abe +export ABE_DIR=~/src/abe +mkdir -p "${ABE_DIR}" || exit 1 +if ! git clone ${abe_branch:+-b ${abe_branch}} http://git.linaro.org/toolchain/abe [...] + rm -rf "${ABE_DIR}" + exit 1 +fi + +#Generate one-time key +mkdir ~/data || exit 1 +if ! ssh-keygen -P '' -f ~/data/onetime > /dev/null < /dev/null; then + rm -rf ~/data + exit 1 +fi +eval `ssh-agent` +ssh-add ~/data/onetime + +#Need to be able to ssh to self to get local sources +#Can reuse the onetime key for this +cat ~/data/onetime.pub >> ~/.ssh/authorized_keys +if ! ssh -o NoHostAuthenticationForLocalhost=yes localhost true; then + echo "Failed to ssh to self" >&2 + exit 1 +fi + +#Generate config file for each target in multinode job +lava-network broadcast eth0 +lava-network collect eth0 +ip_addr="$(lava-network query `lava-group | grep '[[:blank:]]\+host$' | awk '{prin [...] + +#Request agent just before we need it +#This would be two keys in one agent if we wanted one for source, one for job +echo "*** WAITING FOR AGENT: ${ip_addr}" +echo "ssh -o ControlMaster=no -o ControlPath=/dev/null -A ${ip_addr}.lab 'ln -sf \ [...] +while ! test -e ~/data/socket; do + inotifywait -e create ~/data +done +#Best practice would be to use abe to do this, but I hope this is temporary +SSH_AUTH_SOCK=~/data/socket git clone "`grep ${benchmark} ${ABE_DIR}/config/source [...] +#An alternative to the following would be to pass ${benchmark}=ssh://... to abe.sh +sed -i "s#^\$${benchmark}[[:blank:]]\\+\$.*#\\1 ssh://localhost/~/benchsrc/#" "$ [...] + +target_names=() +for lava_name in `lava-group | grep -v '[[:blank:]]*host$' | awk '{print $1}'`; do + target_ip="$(lava-network query ${lava_name} ipv4)" + if test $? -ne 0; then echo "Failed to find IP for ${lava_name}"; exit 1; fi + if test -z "${target_ip}"; then echo "Failed to find IP for ${lava_name}"; exit 1; fi + + #Check that target is accessible, add it to our known_hosts, + #add our one-time key to its authorized_keys + if ! cat ~/data/onetime.pub | SSH_AUTH_SOCK=~/data/socket ssh -o StrictHostKeyCh [...] + echo "Could not access target ${target_ip} (${lava_name})" >&2 + exit 1 + fi + + #Generate configuration files for target + target_type="$(echo ${lava_name} | sed 's/[[:digit:]]*$//')" #TODO: HACK + if test $? -ne 0; then echo "Failed to find target type from ${lava_name}"; exit 1; fi + sed "s/^ip=.*/ip=${target_ip}/" "${ABE_DIR}/config/boards/bench/${target_type}.c [...] + if test $? -ne 0; then echo "Failed to generate config file for target ${lava_na [...] + + cp "${ABE_DIR}/config/boards/bench/${target_type}.services" "${ABE_DIR}/config/b [...] + if test $? -ne 0; then echo "Failed to generate services file for target ${lava_ [...] + + #Add target to list for Benchmark.job + if test "${#target_names[@]}" -eq 0; then + target_names=("${lava_name}") + else + target_names=("${target_names[@]}" "${lava_name}") + fi +done + +#Don't need the agent any more, kill the socket +rm ~/data/socket + +export targets="${target_names[@]}" +export maindir="/${HOME}/bench" +export post_target_cmd="shutdown -h 0" +mkdir "${maindir}" +chmod 700 "${maindir}" + +echo "Running benchmark... I may be some time" +if ! "${ABE_DIR}"/scripts/Benchmark.job; then + echo "Benchmark run failed" >&2 + echo "ssh -o ControlMaster=no -o ControlPath=/dev/null ${ip_addr}.lab" + error=1 + exit +fi + +echo "*** RESULTS READY" +echo "ssh -o ControlMaster=no -o ControlPath=/dev/null ${ip_addr}.lab" +echo "ssh -o ControlMaster=no -o ControlPath=/dev/null ${ip_addr}.lab 'touch ~/dat [...] +while ! test -e ~/data/resultsgone; do + inotifywait -e create ~/data +done + +error=0 +exit diff --git a/config/bench/lava/host-session.yaml b/config/bench/lava/host-session.yaml new file mode 100644 index 0000000..5372321 --- /dev/null +++ b/config/bench/lava/host-session.yaml @@ -0,0 +1,39 @@ +metadata: + name: bench-session-debian + format: "Lava-Test-Shell Test Definition 1.0" + description: "Benchmarking Session" + version: 1.0 + +params: + PUB_KEY: None + ABE_BRANCH: benchmarking + BENCHMARK: fakebench + TOOLCHAIN: https://releases.linaro.org/14.11/components/toolchain/binaries/arm [...] + RUN_FLAGS: None + COMPILER_FLAGS: None + BENCH_DEBUG: None + +install: + deps: + - autogen + - binutils + - bison + - dejagnu + - flex + - gawk + - gcc + - gcc-multilib + - g++-multilib + - git + - inotify-tools + - libgfortran3 + - libncurses5-dev + - make + - openssh-server + - rsync + - texinfo + - wget +run: + steps: + - ./config/bench/lava/setup "$PUB_KEY" + - env BENCH_DEBUG="$BENCH_DEBUG" ./config/bench/lava/host-session "$ABE_BR [...] diff --git a/config/bench/lava/setup b/config/bench/lava/setup new file mode 100755 index 0000000..633a9b7 --- /dev/null +++ b/config/bench/lava/setup @@ -0,0 +1,10 @@ +#!/bin/bash +echo "Hostname for early debugging: $(cat /etc/hostname)" +echo "IP address for early debugging: $(hostname -I)" + +mkdir -p ~/.ssh/ +echo "$1" >> ~/.ssh/authorized_keys +echo "Public Key Installed: $1" + +/etc/init.d/ssh restart +echo "sshd re-started" diff --git a/config/bench/lava/target-session b/config/bench/lava/target-session new file mode 100755 index 0000000..f042d82 --- /dev/null +++ b/config/bench/lava/target-session @@ -0,0 +1,3 @@ +#!/bin/bash +lava-network broadcast eth0 +sleep infinity diff --git a/config/bench/lava/target-session.yaml b/config/bench/lava/target-session.yaml new file mode 100644 index 0000000..6b737ab --- /dev/null +++ b/config/bench/lava/target-session.yaml @@ -0,0 +1,24 @@ +metadata: + name: bench-session + format: "Lava-Test-Shell Test Definition 1.0" + description: "Benchmarking Session" + version: 1.0 + +params: + PUB_KEY: None + +install: + deps: + - openssh-server + - wget + - make + - cpufrequtils + - binutils + - gawk + - libgfortran3 + - rsync + +run: + steps: + - ./config/bench/lava/setup "$PUB_KEY" + - ./config/bench/lava/target-session -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 3 months

[toolchain/abe] branch bernie/jenkmarking-multinode updated (68a090f -> d2d1903)

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a change to branch bernie/jenkmarking-multinode in repository toolchain/abe. discards 68a090f LAVA configuration new 1013694 LAVA configuration new d2d1903 More output when we trap errors for debug This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (68a090f) \ N -- N -- N refs/heads/bernie/jenkmarking-multinode (d2d1903) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omits" are not gone; other references still refer to them. Any revisions marked "discards" are gone forever. The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: config/bench/lava/host-session | 6 +++--- config/bench/lava/host-session.yaml | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 3 months

[toolchain/abe] 02/02: More output when we trap errors for debug

by git

9 years, 3 months

[toolchain/abe] 01/01: LAVA configuration

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a commit to branch bernie/jenkmarking-multinode in repository toolchain/abe. commit 68a090ffd213c49807e8bd185dd4ad0bacf284f8 Author: Bernard Ogden <bernie.ogden(a)linaro.org> Date: Tue Jul 28 11:57:18 2015 +0200 LAVA configuration Based on git.linaro.org/toolchain/semiautobench (host side) and git.linaro.org/toolchain/lavabench (target side), with alterations to clean out remaining hacking-session-isms. git.linaro.org/toolchain/semiautobench: c90fe30dbe0af4a41349bcaf1e7b051cc5c11546 git.linaro.org/toolchain/lavabench: c48e5f8503b61d8af7fa738c1fd1ac46ca2250b4 Change-Id: I827d445d4d3436654627bdaf76ab6778beb66e7b --- config/bench/lava/host-session | 145 ++++++++++++++++++++++++++++++++++ config/bench/lava/host-session.yaml | 39 +++++++++ config/bench/lava/setup | 10 +++ config/bench/lava/target-session | 3 + config/bench/lava/target-session.yaml | 24 ++++++ 5 files changed, 221 insertions(+) diff --git a/config/bench/lava/host-session b/config/bench/lava/host-session new file mode 100755 index 0000000..9c0d48f --- /dev/null +++ b/config/bench/lava/host-session @@ -0,0 +1,145 @@ +#!/bin/bash +set -u +set -o pipefail + +error=1 + +function exitfunc { + if test ${error} -ne 0; then + if test x"${BENCH_DEBUG}" != x && test x"${BENCH_DEBUG}" != xNone; then + sleep infinity + fi + fi + rm -rf ~/* + exit ${error} +} + +trap exitfunc EXIT + +abe_branch="$1" + +#these parameters used by Benchmark.job +export benchmark="${2:?Must set a benchmark}" +export toolchain="${3:?Must set a toolchain}" +if test x"$4" = xNone; then + export run_flags= +else + export run_flags="$4" +fi +if test x"$5" = xNone; then + export compiler_flags= +else + export compiler_flags="$5" +fi + +# Set the PATH to use the LAVA api +echo "export PATH=/lava/bin/:$PATH" > ~/.bashrc + +#Create directory to store src +if ! (mkdir ~/benchsrc && chmod 700 ~/benchsrc); then + echo "Failed to create directory for source" >&2 + exit 1 +fi + +#Initialize git-new-workdir - sadly not just an apt-get call +ln /usr/share/doc/git/contrib/workdir/git-new-workdir /usr/local/bin +chmod 755 /usr/local/bin/git-new-workdir + +#Get abe +export ABE_DIR=~/src/abe +mkdir -p "${ABE_DIR}" || exit 1 +if ! git clone ${abe_branch:+-b ${abe_branch}} http://git.linaro.org/toolchain/abe [...] + rm -rf "${ABE_DIR}" + exit 1 +fi + +#Generate one-time key +mkdir ~/data || exit 1 +if ! ssh-keygen -P '' -f ~/data/onetime > /dev/null < /dev/null; then + rm -rf ~/data + exit 1 +fi +eval `ssh-agent` +ssh-add ~/data/onetime + +#Need to be able to ssh to self to get local sources +#Can reuse the onetime key for this +cat ~/data/onetime.pub >> ~/.ssh/authorized_keys +if ! ssh -o NoHostAuthenticationForLocalhost=yes localhost true; then + echo "Failed to ssh to self" >&2 + exit 1 +fi + +#Generate config file for each target in multinode job +lava-network broadcast eth0 +lava-network collect eth0 +ip_addr="$(lava-network query `lava-group | grep '[[:blank:]]\+host$' | awk '{prin [...] + +#Request agent just before we need it +#This would be two keys in one agent if we wanted one for source, one for job +echo "*** WAITING FOR AGENT: ${ip_addr}" +echo "ssh -o ControlMaster=no -o ControlPath=/dev/null -A ${ip_addr}.lab 'ln -sf \ [...] +while ! test -e ~/data/socket; do + inotifywait -e create ~/data +done +#Best practice would be to use abe to do this, but I hope this is temporary +SSH_AUTH_SOCK=~/data/socket git clone "`grep ${benchmark} ${ABE_DIR}/config/source [...] +#An alternative to the following would be to pass ${benchmark}=ssh://... to abe.sh +sed -i "s#^\$${benchmark}[[:blank:]]\\+\$.*#\\1 ssh://localhost/~/benchsrc/#" "$ [...] + +target_names=() +for lava_name in `lava-group | grep -v '[[:blank:]]*host$' | awk '{print $1}'`; do + target_ip="$(lava-network query ${lava_name} ipv4)" + if test $? -ne 0; then echo "Failed to find IP for ${lava_name}"; exit 1; fi + if test -z "${target_ip}"; then echo "Failed to find IP for ${lava_name}"; exit 1; fi + + #Check that target is accessible, add it to our known_hosts, + #add our one-time key to its authorized_keys + if ! cat ~/data/onetime.pub | SSH_AUTH_SOCK=~/data/socket ssh -o StrictHostKeyCh [...] + echo "Could not access target ${target_ip} (${lava_name})" >&2 + exit 1 + fi + + #Generate configuration files for target + target_type="$(echo ${lava_name} | sed 's/[[:digit:]]*$//')" #TODO: HACK + if test $? -ne 0; then echo "Failed to find target type from ${lava_name}"; exit 1; fi + sed "s/^ip=.*/ip=${target_ip}/" "${ABE_DIR}/config/boards/bench/${target_type}.c [...] + if test $? -ne 0; then echo "Failed to generate config file for target ${lava_na [...] + + cp "${ABE_DIR}/config/boards/bench/${target_type}.services" "${ABE_DIR}/config/b [...] + if test $? -ne 0; then echo "Failed to generate services file for target ${lava_ [...] + + #Add target to list for Benchmark.job + if test "${#target_names[@]}" -eq 0; then + target_names=("${lava_name}") + else + target_names=("${target_names[@]}" "${lava_name}") + fi +done + +#Don't need the agent any more, kill the socket +rm ~/data/socket + +export targets="${target_names[@]}" +export maindir="/${HOME}/bench" +export post_target_cmd="shutdown -h 0" +mkdir "${maindir}" +chmod 700 "${maindir}" + +echo "Running benchmark... I may be some time" +if ! "${ABE_DIR}"/scripts/Benchmark.job; then + echo "Benchmark run failed" >&2 + echo "ssh -o ControlMaster=no -o ControlPath=/dev/null ${ip_addr}.lab" + error=1 + exit +fi + +echo "*** RESULTS READY" +echo "ssh -o ControlMaster=no -o ControlPath=/dev/null ${ip_addr}.lab" +echo "ssh -o ControlMaster=no -o ControlPath=/dev/null ${ip_addr}.lab 'touch ~/dat [...] +while ! test -e ~/data/resultsgone; do + inotifywait -e create ~/data +done + +error=0 +exit diff --git a/config/bench/lava/host-session.yaml b/config/bench/lava/host-session.yaml new file mode 100644 index 0000000..7f90a66 --- /dev/null +++ b/config/bench/lava/host-session.yaml @@ -0,0 +1,39 @@ +metadata: + name: bench-session-debian + format: "Lava-Test-Shell Test Definition 1.0" + description: "Benchmarking Session" + version: 1.0 + +params: + PUB_KEY: None + ABE_BRANCH: benchmarking + BENCHMARK: fakebench + TOOLCHAIN: https://releases.linaro.org/14.11/components/toolchain/binaries/arm [...] + RUN_FLAGS: None + COMPILER_FLAGS: None + BENCH_DEBUG: None + +install: + deps: + - autogen + - binutils + - bison + - dejagnu + - flex + - gawk + - gcc + - gcc-multilib + - g++-multilib + - git + - inotify-tools + - libgfortran3 + - libncurses5-dev + - make + - openssh-server + - rsync + - texinfo + - wget +run: + steps: + - ./config/bench/lava/setup + - env BENCH_DEBUG="$BENCH_DEBUG" ./config/bench/lava/host-session "$PUB_KE [...] diff --git a/config/bench/lava/setup b/config/bench/lava/setup new file mode 100755 index 0000000..633a9b7 --- /dev/null +++ b/config/bench/lava/setup @@ -0,0 +1,10 @@ +#!/bin/bash +echo "Hostname for early debugging: $(cat /etc/hostname)" +echo "IP address for early debugging: $(hostname -I)" + +mkdir -p ~/.ssh/ +echo "$1" >> ~/.ssh/authorized_keys +echo "Public Key Installed: $1" + +/etc/init.d/ssh restart +echo "sshd re-started" diff --git a/config/bench/lava/target-session b/config/bench/lava/target-session new file mode 100755 index 0000000..f042d82 --- /dev/null +++ b/config/bench/lava/target-session @@ -0,0 +1,3 @@ +#!/bin/bash +lava-network broadcast eth0 +sleep infinity diff --git a/config/bench/lava/target-session.yaml b/config/bench/lava/target-session.yaml new file mode 100644 index 0000000..6b737ab --- /dev/null +++ b/config/bench/lava/target-session.yaml @@ -0,0 +1,24 @@ +metadata: + name: bench-session + format: "Lava-Test-Shell Test Definition 1.0" + description: "Benchmarking Session" + version: 1.0 + +params: + PUB_KEY: None + +install: + deps: + - openssh-server + - wget + - make + - cpufrequtils + - binutils + - gawk + - libgfortran3 + - rsync + +run: + steps: + - ./config/bench/lava/setup "$PUB_KEY" + - ./config/bench/lava/target-session -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 3 months

[toolchain/abe] branch bernie/jenkmarking-multinode updated (90b9597 -> 68a090f)

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a change to branch bernie/jenkmarking-multinode in repository toolchain/abe. discards 90b9597 LAVA configuration new 68a090f LAVA configuration This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (90b9597) \ N -- N -- N refs/heads/bernie/jenkmarking-multinode (68a090f) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omits" are not gone; other references still refer to them. Any revisions marked "discards" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: config/bench/lava/host-session.yaml | 4 ++-- config/bench/lava/target-session.yaml | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 3 months

[toolchain/abe] 02/02: LAVA configuration

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a commit to branch bernie/jenkmarking-multinode in repository toolchain/abe. commit 90b9597f391c04a15948dc8535ea4bf3e9b3e4b2 Author: Bernard Ogden <bernie.ogden(a)linaro.org> Date: Tue Jul 28 11:57:18 2015 +0200 LAVA configuration Based on git.linaro.org/toolchain/semiautobench (host side) and git.linaro.org/toolchain/lavabench (target side), with alterations to clean out remaining hacking-session-isms. git.linaro.org/toolchain/semiautobench: c90fe30dbe0af4a41349bcaf1e7b051cc5c11546 git.linaro.org/toolchain/lavabench: c48e5f8503b61d8af7fa738c1fd1ac46ca2250b4 Change-Id: I827d445d4d3436654627bdaf76ab6778beb66e7b --- config/bench/lava/host-session | 145 ++++++++++++++++++++++++++++++++++ config/bench/lava/host-session.yaml | 39 +++++++++ config/bench/lava/setup | 10 +++ config/bench/lava/target-session | 3 + config/bench/lava/target-session.yaml | 24 ++++++ 5 files changed, 221 insertions(+) diff --git a/config/bench/lava/host-session b/config/bench/lava/host-session new file mode 100755 index 0000000..9c0d48f --- /dev/null +++ b/config/bench/lava/host-session @@ -0,0 +1,145 @@ +#!/bin/bash +set -u +set -o pipefail + +error=1 + +function exitfunc { + if test ${error} -ne 0; then + if test x"${BENCH_DEBUG}" != x && test x"${BENCH_DEBUG}" != xNone; then + sleep infinity + fi + fi + rm -rf ~/* + exit ${error} +} + +trap exitfunc EXIT + +abe_branch="$1" + +#these parameters used by Benchmark.job +export benchmark="${2:?Must set a benchmark}" +export toolchain="${3:?Must set a toolchain}" +if test x"$4" = xNone; then + export run_flags= +else + export run_flags="$4" +fi +if test x"$5" = xNone; then + export compiler_flags= +else + export compiler_flags="$5" +fi + +# Set the PATH to use the LAVA api +echo "export PATH=/lava/bin/:$PATH" > ~/.bashrc + +#Create directory to store src +if ! (mkdir ~/benchsrc && chmod 700 ~/benchsrc); then + echo "Failed to create directory for source" >&2 + exit 1 +fi + +#Initialize git-new-workdir - sadly not just an apt-get call +ln /usr/share/doc/git/contrib/workdir/git-new-workdir /usr/local/bin +chmod 755 /usr/local/bin/git-new-workdir + +#Get abe +export ABE_DIR=~/src/abe +mkdir -p "${ABE_DIR}" || exit 1 +if ! git clone ${abe_branch:+-b ${abe_branch}} http://git.linaro.org/toolchain/abe [...] + rm -rf "${ABE_DIR}" + exit 1 +fi + +#Generate one-time key +mkdir ~/data || exit 1 +if ! ssh-keygen -P '' -f ~/data/onetime > /dev/null < /dev/null; then + rm -rf ~/data + exit 1 +fi +eval `ssh-agent` +ssh-add ~/data/onetime + +#Need to be able to ssh to self to get local sources +#Can reuse the onetime key for this +cat ~/data/onetime.pub >> ~/.ssh/authorized_keys +if ! ssh -o NoHostAuthenticationForLocalhost=yes localhost true; then + echo "Failed to ssh to self" >&2 + exit 1 +fi + +#Generate config file for each target in multinode job +lava-network broadcast eth0 +lava-network collect eth0 +ip_addr="$(lava-network query `lava-group | grep '[[:blank:]]\+host$' | awk '{prin [...] + +#Request agent just before we need it +#This would be two keys in one agent if we wanted one for source, one for job +echo "*** WAITING FOR AGENT: ${ip_addr}" +echo "ssh -o ControlMaster=no -o ControlPath=/dev/null -A ${ip_addr}.lab 'ln -sf \ [...] +while ! test -e ~/data/socket; do + inotifywait -e create ~/data +done +#Best practice would be to use abe to do this, but I hope this is temporary +SSH_AUTH_SOCK=~/data/socket git clone "`grep ${benchmark} ${ABE_DIR}/config/source [...] +#An alternative to the following would be to pass ${benchmark}=ssh://... to abe.sh +sed -i "s#^\$${benchmark}[[:blank:]]\\+\$.*#\\1 ssh://localhost/~/benchsrc/#" "$ [...] + +target_names=() +for lava_name in `lava-group | grep -v '[[:blank:]]*host$' | awk '{print $1}'`; do + target_ip="$(lava-network query ${lava_name} ipv4)" + if test $? -ne 0; then echo "Failed to find IP for ${lava_name}"; exit 1; fi + if test -z "${target_ip}"; then echo "Failed to find IP for ${lava_name}"; exit 1; fi + + #Check that target is accessible, add it to our known_hosts, + #add our one-time key to its authorized_keys + if ! cat ~/data/onetime.pub | SSH_AUTH_SOCK=~/data/socket ssh -o StrictHostKeyCh [...] + echo "Could not access target ${target_ip} (${lava_name})" >&2 + exit 1 + fi + + #Generate configuration files for target + target_type="$(echo ${lava_name} | sed 's/[[:digit:]]*$//')" #TODO: HACK + if test $? -ne 0; then echo "Failed to find target type from ${lava_name}"; exit 1; fi + sed "s/^ip=.*/ip=${target_ip}/" "${ABE_DIR}/config/boards/bench/${target_type}.c [...] + if test $? -ne 0; then echo "Failed to generate config file for target ${lava_na [...] + + cp "${ABE_DIR}/config/boards/bench/${target_type}.services" "${ABE_DIR}/config/b [...] + if test $? -ne 0; then echo "Failed to generate services file for target ${lava_ [...] + + #Add target to list for Benchmark.job + if test "${#target_names[@]}" -eq 0; then + target_names=("${lava_name}") + else + target_names=("${target_names[@]}" "${lava_name}") + fi +done + +#Don't need the agent any more, kill the socket +rm ~/data/socket + +export targets="${target_names[@]}" +export maindir="/${HOME}/bench" +export post_target_cmd="shutdown -h 0" +mkdir "${maindir}" +chmod 700 "${maindir}" + +echo "Running benchmark... I may be some time" +if ! "${ABE_DIR}"/scripts/Benchmark.job; then + echo "Benchmark run failed" >&2 + echo "ssh -o ControlMaster=no -o ControlPath=/dev/null ${ip_addr}.lab" + error=1 + exit +fi + +echo "*** RESULTS READY" +echo "ssh -o ControlMaster=no -o ControlPath=/dev/null ${ip_addr}.lab" +echo "ssh -o ControlMaster=no -o ControlPath=/dev/null ${ip_addr}.lab 'touch ~/dat [...] +while ! test -e ~/data/resultsgone; do + inotifywait -e create ~/data +done + +error=0 +exit diff --git a/config/bench/lava/host-session.yaml b/config/bench/lava/host-session.yaml new file mode 100644 index 0000000..1288f6c --- /dev/null +++ b/config/bench/lava/host-session.yaml @@ -0,0 +1,39 @@ +metadata: + name: bench-session-debian + format: "Lava-Test-Shell Test Definition 1.0" + description: "Benchmarking Session" + version: 1.0 + +params: + PUB_KEY: None + ABE_BRANCH: benchmarking + BENCHMARK: fakebench + TOOLCHAIN: https://releases.linaro.org/14.11/components/toolchain/binaries/arm [...] + RUN_FLAGS: None + COMPILER_FLAGS: None + BENCH_DEBUG: None + +install: + deps: + - autogen + - binutils + - bison + - dejagnu + - flex + - gawk + - gcc + - gcc-multilib + - g++-multilib + - git + - inotify-tools + - libgfortran3 + - libncurses5-dev + - make + - openssh-server + - rsync + - texinfo + - wget +run: + steps: + - ./setup + - env BENCH_DEBUG="$BENCH_DEBUG" ./host-session "$PUB_KEY" "$ABE_BRANCH" " [...] diff --git a/config/bench/lava/setup b/config/bench/lava/setup new file mode 100755 index 0000000..633a9b7 --- /dev/null +++ b/config/bench/lava/setup @@ -0,0 +1,10 @@ +#!/bin/bash +echo "Hostname for early debugging: $(cat /etc/hostname)" +echo "IP address for early debugging: $(hostname -I)" + +mkdir -p ~/.ssh/ +echo "$1" >> ~/.ssh/authorized_keys +echo "Public Key Installed: $1" + +/etc/init.d/ssh restart +echo "sshd re-started" diff --git a/config/bench/lava/target-session b/config/bench/lava/target-session new file mode 100755 index 0000000..f042d82 --- /dev/null +++ b/config/bench/lava/target-session @@ -0,0 +1,3 @@ +#!/bin/bash +lava-network broadcast eth0 +sleep infinity diff --git a/config/bench/lava/target-session.yaml b/config/bench/lava/target-session.yaml new file mode 100644 index 0000000..5c9e6bd --- /dev/null +++ b/config/bench/lava/target-session.yaml @@ -0,0 +1,24 @@ +metadata: + name: bench-session + format: "Lava-Test-Shell Test Definition 1.0" + description: "Benchmarking Session" + version: 1.0 + +params: + PUB_KEY: None + +install: + deps: + - openssh-server + - wget + - make + - cpufrequtils + - binutils + - gawk + - libgfortran3 + - rsync + +run: + steps: + - ./setup "$PUB_KEY" + - ./target-session -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 3 months

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

Tcwg-commits July 2015