Tcwg-commits July 2015

tcwg-commits@lists.linaro.org

2 participants
1355 discussions

[gcc] branch hjl/iamcu/gcc-5-branch updated (64b03b2 -> 850fa4a)

by git

This is an automated email from the git hooks/post-receive script. unknown user pushed a change to branch hjl/iamcu/gcc-5-branch in repository gcc. discards 64b03b2 Cleanup FPmode splitters a bit discards d1b9ab5 More fix for PR rtl-optimization/66626 discards dba25d1 Fix PR rtl-optimization/66626 discards 2355c67 Replicate static chain on the stack discards fd885f0 Add a testcase for PR target/66824 discards cea782e Fix PR target/66824 discards 1a5931a Allow indirect sibcall with register arguments discards 605380e Fix PR target/65753 discards 1d55c12 Adjust variable shift costs for IA MCU discards 31a26c8 Update config/i386/iamcu.h discards be29017 Define ATTRIBUTE_ALIGNED_VALUE to 32 for IA MCU discards 894db30 Check int_size_in_bytes in ix86_return_in_memory discards 8c27a3d Allow <adxintrin.h> for IA MCU discards 0a36965 Don't pass/return vectors in registers for IAMCU discards fbd7c54 Optimize i?86-*-elfiamcu for iamcu by default discards 104e858 Add -march=iamcu to optimize for IA MCU discards e1e2597 Add IA MCU support to x86intrin.h discards a1e2339 IA MCU psABI support: testsuite discards a95cebd IA MCU psABI support: changes to libraries discards 6a00ee0 IA MCU psABI support: GCC changes adds 52c80e2 Daily bump. adds 35b2f10 Daily bump. adds dfe71bc [ARM][testsuite] Fix FAIL: gcc.target/arm/macro_defs0.c and m [...] adds 5efb961 2015-07-21 Alex Velenko <Alex.Velenko(a)arm.com> adds 95a45a0 2015-07-21 Alex Velenko <Alex.Velenko(a)arm.com> adds 4aef292 gcc/fortran/ChangeLog: adds 52ae5e5 2015-07-21 Richard Biener <rguenther(a)suse.de> adds 8aa6d5f 2015-07-21 Alex Velenko <Alex.Velenko(a)arm.com> adds b94cc67 Backport from 2015-07-21 trunk r226046. PR target/66956 * [...] adds 90336eb Daily bump. adds a0cc8dc Daily bump. adds 1c9597b 2015-07-23 Chung-Lin Tang <cltang(a)codesourcery.com> adds b115f3b gcc/ adds 6e64f22 gcc/ adds 9e32ae3 gcc/ adds 1b6fcbf gcc/ adds 3e7fcef gcc/ adds e949f47 gcc/ adds 2a713ca gcc/ new e79a240 IA MCU psABI support: GCC changes new 80d5cc5 IA MCU psABI support: changes to libraries new d9c8b80 IA MCU psABI support: testsuite new 2f92abd Add IA MCU support to x86intrin.h new 533008e Add -march=iamcu to optimize for IA MCU new ba77467 Optimize i?86-*-elfiamcu for iamcu by default new 33b9f67 Don't pass/return vectors in registers for IAMCU new 8eea771 Allow <adxintrin.h> for IA MCU new ce8896b Check int_size_in_bytes in ix86_return_in_memory new d109dec Define ATTRIBUTE_ALIGNED_VALUE to 32 for IA MCU new 36ed612 Update config/i386/iamcu.h new 2f1e0b4 Adjust variable shift costs for IA MCU new f1e7e12 Fix PR target/65753 new 6a7d6c9 Allow indirect sibcall with register arguments new d858d43 Fix PR target/66824 new c6eded9 Add a testcase for PR target/66824 new 09fe1fe Replicate static chain on the stack new cead75e Fix PR rtl-optimization/66626 new 45074b7 More fix for PR rtl-optimization/66626 new 850fa4a Cleanup FPmode splitters a bit This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (64b03b2) \ N -- N -- N refs/heads/hjl/iamcu/gcc-5-branch (850fa4a) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omits" are not gone; other references still refer to them. Any revisions marked "discards" are gone forever. The 20 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: gcc/ChangeLog | 88 +++++++++++++++++++++ gcc/DATESTAMP | 2 +- gcc/cfgexpand.c | 92 ++++++++++++++-------- gcc/config/avr/avr-dimode.md | 6 +- gcc/config/i386/i386.c | 3 +- gcc/fortran/ChangeLog | 12 +++ gcc/fortran/trans-expr.c | 26 +++++- gcc/function.c | 35 ++++---- gcc/genmatch.c | 25 ++++-- gcc/ipa-chkp.c | 87 +++++++++++++------- gcc/ipa-chkp.h | 1 + gcc/lto-cgraph.c | 12 ++- gcc/testsuite/ChangeLog | 91 +++++++++++++++++++++ gcc/testsuite/gcc.dg/lto/chkp-removed-alias_0.c | 28 +++++++ gcc/testsuite/gcc.dg/lto/chkp-wrap-asm-name_0.c | 20 +++++ gcc/testsuite/gcc.target/arm/macro_defs0.c | 1 + gcc/testsuite/gcc.target/arm/macro_defs1.c | 1 + gcc/testsuite/gcc.target/arm/thumb-bitfld1.c | 2 + gcc/testsuite/gcc.target/arm/thumb-ltu.c | 3 + gcc/testsuite/gcc.target/arm/thumb1-far-jump-2.c | 2 +- gcc/testsuite/gcc.target/i386/mpx/pr66048.cc | 16 ++++ gcc/testsuite/gcc.target/i386/mpx/pr66134.c | 17 ++++ gcc/testsuite/gcc.target/i386/mpx/pr66567.c | 16 ++++ gcc/testsuite/gcc.target/i386/mpx/pr66568.c | 11 +++ gcc/testsuite/gcc.target/i386/mpx/pr66569.c | 14 ++++ .../gfortran.dg/structure_constructor_13.f03 | 28 +++++++ gcc/tree-chkp.c | 37 +++++++-- gcc/tree-chkp.h | 1 + libgcc/ChangeLog | 8 ++ libgcc/config/nios2/linux-atomic.c | 5 -- 30 files changed, 579 insertions(+), 111 deletions(-) create mode 100644 gcc/testsuite/gcc.dg/lto/chkp-removed-alias_0.c create mode 100644 gcc/testsuite/gcc.dg/lto/chkp-wrap-asm-name_0.c create mode 100644 gcc/testsuite/gcc.target/i386/mpx/pr66048.cc create mode 100644 gcc/testsuite/gcc.target/i386/mpx/pr66134.c create mode 100644 gcc/testsuite/gcc.target/i386/mpx/pr66567.c create mode 100644 gcc/testsuite/gcc.target/i386/mpx/pr66568.c create mode 100644 gcc/testsuite/gcc.target/i386/mpx/pr66569.c create mode 100644 gcc/testsuite/gfortran.dg/structure_constructor_13.f03 -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

[toolchain/semiautobench] branch inserted_src updated (aa0f14e -> dff3152)

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a change to branch inserted_src in repository toolchain/semiautobench. from aa0f14e Specify maindir for Benchmark.job new c3960cc Put onetime key into agent new dff3152 Use more specific option for localhost connection The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: invoke_session_debian | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

[toolchain/semiautobench] 01/02: Put onetime key into agent

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a commit to branch inserted_src in repository toolchain/semiautobench. commit c3960cc80c1a5b5e5ccd3805a3e3b3e5d96aa1a3 Author: Bernard Ogden <bernie.ogden(a)linaro.org> Date: Thu Jul 23 15:05:52 2015 +0200 Put onetime key into agent Change-Id: I492cfb87a7b545e5640d4d09bdec39d6f1609746 --- invoke_session_debian | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/invoke_session_debian b/invoke_session_debian index 278ba51..ee1d249 100755 --- a/invoke_session_debian +++ b/invoke_session_debian @@ -68,11 +68,13 @@ if ! ssh-keygen -P '' -f ~/data/onetime > /dev/null < /dev/null; then rm -rf ~/data exit 1 fi +eval `ssh-agent` +ssh-add ~/data/onetime #Need to be able to ssh to self to get local sources #Can reuse the onetime key for this cat ~/data/onetime.pub >> ~/.ssh/authorized_keys -if ! ssh -i ~/data/onetime -o StrictHostKeyChecking=no localhost true; then +if ! ssh -o StrictHostKeyChecking=no localhost true; then echo "Failed to ssh to self" >&2 exit 1 fi -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

[toolchain/semiautobench] 02/02: Use more specific option for localhost connection

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a commit to branch inserted_src in repository toolchain/semiautobench. commit dff31528eff91d0e95f41265894b9a2e5aef28e2 Author: Bernard Ogden <bernie.ogden(a)linaro.org> Date: Thu Jul 23 15:06:43 2015 +0200 Use more specific option for localhost connection Change-Id: I650eec8e9b8cdf5a4da928bcac1ee3fdb5b30f4e --- invoke_session_debian | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/invoke_session_debian b/invoke_session_debian index ee1d249..7040dcd 100755 --- a/invoke_session_debian +++ b/invoke_session_debian @@ -74,7 +74,7 @@ ssh-add ~/data/onetime #Need to be able to ssh to self to get local sources #Can reuse the onetime key for this cat ~/data/onetime.pub >> ~/.ssh/authorized_keys -if ! ssh -o StrictHostKeyChecking=no localhost true; then +if ! ssh -o NoHostAuthenticationForLocalhost=yes localhost true; then echo "Failed to ssh to self" >&2 exit 1 fi -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

[toolchain/abe] branch bernie/jenkmarking-multinode updated: wip: more output

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a commit to branch bernie/jenkmarking-multinode in repository toolchain/abe. The following commit(s) were added to refs/heads/bernie/jenkmarking-multinode by th [...] new 58f88a3 wip: more output 58f88a3 is described below commit 58f88a33808630e7e190043975b09faa39b05325 Author: Bernard Ogden <bernie.ogden(a)linaro.org> Date: Thu Jul 23 14:56:45 2015 +0200 wip: more output Change-Id: Ia6c2de4fded3c55810f57e078b9856a75662c8ab --- scripts/runbenchmark.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/runbenchmark.sh b/scripts/runbenchmark.sh index f6ff8a4..40ca21a 100755 --- a/scripts/runbenchmark.sh +++ b/scripts/runbenchmark.sh @@ -105,7 +105,7 @@ clean_benchmark() exit "${error}" } -if ! (. "${topdir}"/lib/common.sh; remote_exec "${ip}" true ${ssh_opts}) > /dev/nu [...] +if ! (. "${topdir}"/lib/common.sh; remote_exec "${ip}" true ${ssh_opts}); then echo "Unable to connect to target ${ip:-(unknown)}" 1>&2 exit 1 fi -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

[gcc] branch trunk updated (10adbe6 -> b826233)

by git

This is an automated email from the git hooks/post-receive script. unknown user pushed a change to branch trunk in repository gcc. from 10adbe6 2015-07-23 Richard Biener <rguenther(a)suse.de> new b826233 Check TYPE_OVERFLOW_WRAPS for parloops reductions The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: gcc/ChangeLog | 12 ++++++ gcc/testsuite/ChangeLog | 13 ++++++ gcc/testsuite/gcc.dg/autopar/outer-4.c | 2 +- gcc/testsuite/gcc.dg/autopar/outer-5.c | 2 +- gcc/testsuite/gcc.dg/autopar/outer-6.c | 4 +- gcc/testsuite/gcc.dg/autopar/reduc-2.c | 4 +- gcc/testsuite/gcc.dg/autopar/reduc-2char.c | 4 +- gcc/testsuite/gcc.dg/autopar/reduc-2short.c | 4 +- gcc/testsuite/gcc.dg/autopar/reduc-8.c | 4 +- .../gcc.dg/autopar/{outer-4.c => uns-outer-4.c} | 21 +++++----- gcc/testsuite/gcc.dg/autopar/uns-outer-5.c | 49 ++++++++++++++++++++++ .../gcc.dg/autopar/{outer-6.c => uns-outer-6.c} | 42 ++++++++++--------- gcc/tree-parloops.c | 6 +-- gcc/tree-vect-loop.c | 44 ++++++++++++------- gcc/tree-vectorizer.h | 3 +- 15 files changed, 154 insertions(+), 60 deletions(-) copy gcc/testsuite/gcc.dg/autopar/{outer-4.c => uns-outer-4.c} (61%) create mode 100644 gcc/testsuite/gcc.dg/autopar/uns-outer-5.c copy gcc/testsuite/gcc.dg/autopar/{outer-6.c => uns-outer-6.c} (57%) -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

[toolchain/abe] 01/02: Remove last listener

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a commit to branch bernie/jenkmarking-multinode in repository toolchain/abe. commit dcaa216b9a5f0f975b458949cd94d52ea08f5619 Author: Bernard Ogden <bernie.ogden(a)linaro.org> Date: Thu Jul 23 13:49:58 2015 +0200 Remove last listener Remove a bunch of support code that was only there for the listener, too. We replace this with code that sends a ping when the benchmark run finishes on a target. On receipt of ping, the host will try to get the benchmark data from the target that sent the ping (it assumes that the hostname (or IP address, for nameless networks) is stable). For protection against connecting to the wrong target, we require that the public key for each target is in known_hosts before benchmarking begins, and we set StrictHostKeyAuthentication to yes. On the plus side, this is less complicated and thus less error-prone and more maintainable. It's also more secure, in that our only service is now ssh - not ssh + netcat. On the minus side, it is less capable. We confine the scripts to only working: 1) Within a single network (all machines must be able to ping one another) 2) With reliable addresses (stable IP or hostname) If these conditions are not met, failure is reasonably visible: the host will either not notice that the target completed its benchmark (but the user might), or the host will get a message that a target completed the benchmark from a wrong IP address and thus fail to connect to it. Either way, the target will stay up, and so data can be collected 'by hand', provided the necessary private key is availble. Change-Id: Ieb9393a2b9d665eea268dab1ab24232e213360cc --- scripts/benchutil.sh | 107 ------------------------- scripts/establish_listener.sh | 181 ------------------------------------------ scripts/runbenchmark.sh | 96 ++++------------------ 3 files changed, 14 insertions(+), 370 deletions(-) diff --git a/scripts/benchutil.sh b/scripts/benchutil.sh deleted file mode 100644 index 94cf75a..0000000 --- a/scripts/benchutil.sh +++ /dev/null @@ -1,107 +0,0 @@ -set -o pipefail -set -o nounset - -function get_addr -{ - local hostname="${1:-localhost}" - local listener_addr - if test x"${hostname}" = xlocalhost; then - listener_addr="`hostname -I`" - else - listener_addr="`ssh -F /dev/null -o PasswordAuthentication=no -o PubkeyAuthent [...] - fi - if test $? -ne 0; then - echo "Failed to run 'hostname -I' on ${hostname}" 1>&2 - return 1 - fi - - #We only try to support IPv4 for now - listener_addr="`echo ${listener_addr} | tr ' ' '\n' | grep '^\([[:digit:]]\{1,3\ [...] - if test $? -ne 0; then - echo "No IPv4 address found, aborting" 1>&2 - return 1 - fi - - #We don't try to figure out what'll happen if we've got multiple IPv4 interfaces - if test "`echo ${listener_addr} | wc -w`" -ne 1; then - echo "Multiple IPv4 addresses found, aborting" 1>&2 - return 1 - fi - - echo "${listener_addr}" | sed 's/^[[:blank:]]*//' | sed 's/[[:blank:]]*$//' -} - -#Attempt to use read to discover whether there is a record to read from the producer -#If we time out, check to see whether the producer still seems to be alive. -#We can check more than one pid, if we have visibility of some other process(s) that we -#would also like to monitor while we wait to read something. If any of these -#processes seem dead then return 2, otherwise keep trying to read. -#Once we've established that there seems to be a record, try to read it with a -#read timeout. If we fail to read within read_timeout, return 1 to indicate -#read failure - but the producer may still be alive in this case. -#Can also set a deadline - bgread will return 3 if it hasn't seen any new output -#before deadline expires. deadline is only checked at read_timeout intervals. -#Typical invocation: foo="`bgread ${child_pid} <&3`" -#Invocation with read checks every 5 seconds, failure after 2 minutes and two -#pids to monitor: -#foo="`bgread -T 120 -t 5 ${child_pid} ${other_pid} <&3`" -function bgread -{ - OPTIND=1 - local read_timeout=60 - local deadline= - local pid - - while getopts T:t: flag; do - case "${flag}" in - t) read_timeout="${OPTARG}";; - T) deadline="$((${OPTARG} + `date +%s`))";; - *) - echo "Bad arg" 1>&2 - return 1 - ;; - esac - done - shift $((OPTIND - 1)) - - if test $# -eq 0; then - echo "No pid(s) to poll!" 1>&2 - return 1 - fi - local buffer='' - local line='' - - #We have to be careful here. If the read times out when there was a partial - #record on the fifo then the part that has been read just gets discarded. We - #can avoid this problem by using -N to ensure that we read the minimal amount - #and DO NOT discard it. -N 0 might be intuited to do the right thing, but is - #arguably undefined behaviour and empirically doesn't work. - #Read 1 char then timeout if it isn't a delimiter: buffer is the char, exit code 0 OR - #Read the delimiter, don't timeout: buffer is empty, exit code 0 OR - #Fail to read any chars coz there aren't any, then timeout: buffer is empty, exi [...] - while ! read -N 1 -t "${read_timeout}" buffer; do - for pid in "$@"; do - kill -0 "${pid}" > /dev/null 2>&1 || return 2 - done - if test x"${deadline:-}" != x; then - if test `date +%s` -ge ${deadline}; then - echo "bgread timed out" 1>&2 - return 3 - fi - fi - done - - #If we get here, we managed to read 1 char. If we have a null string just - #return it (the record was empty). Otherwise, assume the rest of the record is r [...] - #especially within the generous timeout that we allow. - if test x"${buffer:-}" != x; then - read -t 60 line - if test $? -ne 0; then - echo "Record did not complete" 1>&2 - return 1 - fi - fi - echo "${buffer}${line}" - return 0 -} - diff --git a/scripts/establish_listener.sh b/scripts/establish_listener.sh deleted file mode 100755 index 5266faa..0000000 --- a/scripts/establish_listener.sh +++ /dev/null @@ -1,181 +0,0 @@ -#!/bin/bash -set -o pipefail -set -o nounset - -# load the configure file produced by configure -if test -e "${PWD}/host.conf"; then - . "${PWD}/host.conf" -else - echo "ERROR: no host.conf file! Did you run configure?" 1>&2 - exit 1 -fi -topdir="${abe_path}" #abe global, but this should be the right value for abe -. "${topdir}"/scripts/benchutil.sh -if test $? -ne 0; then - echo "Unable to source ${topdir}/benchutil.sh" 1>&2 - exit 1 -fi - -trap cleanup EXIT -trap 'exit ${error}' TERM INT HUP QUIT - -error=1 -listener_pid= -forward_pid= -pseudofifo_pid= -temps="`mktemp -dt XXXXXXXXX`" || exit 1 -listener_file="${temps}/listener_file" -gateway= - -function cleanup -{ - error=$? - if test x"${listener_pid:-}" != x; then - kill "${listener_pid}" 2>/dev/null - wait "${listener_pid}" - fi - if test x"${pseudofifo_pid:-}" != x; then - kill "${pseudofifo_pid}" 2>/dev/null - #Substituted process is not our child and cannot be waited on. Fortunately, - #it doesn't matter too much when it dies. - fi - if test x"${forward_pid:-}" != x; then - kill "${forward_pid}" 2>/dev/null - wait "${forward_pid}" - fi - if test -d "${temps}"; then - exec {pseudofifo_handle}<&- - rm -rf ${temps} - if test $? -ne 0; then - echo "Failed to delete temporary file store ${temps}" 1>&2 - fi - error=1 - fi - exit "${error}" -} - -#A fifo would make much more sense, but nc doesn't like it -touch "${listener_file}" -if test $? -ne 0; then - echo "Failed to create listener file '${listener_file}'" 1>&2 - exit 1 -fi - -#The trap is just to suppress the 'Terminated' message -exec {pseudofifo_handle}< <(trap 'exit' TERM; tail -f "${listener_file}"& echo $! [...] -read -t 60 pseudofifo_pid <&"${pseudofifo_handle}" -if test $? -ne 0; then - echo "Failed to read pseudofifo pid" 1>&2 - exit 1 -fi - -forward_fifo="${temps}/forward_fifo" -mkfifo "${forward_fifo}" || exit 1 - -while getopts f: flag; do - case "${flag}" in - f) gateway="${OPTARG}" ;; - *) - echo "Bad arg" 1>&2 - exit 1 - ;; - esac -done -shift $((OPTIND - 1)) -if test $# -ne 3; then - echo "establish_listener needs 3 args, got $#" 1>&2 - for arg in "$@"; do echo "${arg}" 1>&2; done - exit 1 -fi -if test x"${gateway:-}" != x; then - if ! echo "${gateway}" | grep -q '.\+:.\+'; then - echo "If specifying a gateway to forward through, must be in format 'internal_ [...] - echo "Got: ${gateway}" 1>&2 - exit 1 - fi -fi - -listener_addr="$1" -ping -c 1 "${listener_addr}" > /dev/null -if test $? -ne 0; then - echo "Unable to ping host ${listener_addr}" 1>&2 - exit 1 -fi -start_port="$2" -end_port="$3" - -for ((listener_port=${start_port}; listener_port < ${end_port}; listener_port++)); do - #Try to listen on the port. nc will fail if something has snatched it. - echo "Attempting to establish listener on ${listener_addr}:${listener_port}" 1>&2 - - if test -e /bin/nc.traditional; then - /bin/nc.traditional -l -s "${listener_addr}" -p "${listener_port}" >> "${liste [...] - elif test -e /bin/nc.openbsd; then - /bin/nc.openbsd -l "${listener_addr}" "${listener_port}" >> "${liste [...] - else - echo "Unable to identify netcat" 1>&2 - exit 1 - fi - listener_pid=$! - - #nc doesn't confirm that it's got the port, so we spin until either: - #1) We see that the port has been taken by our process - #2) We see our process exit (implying that the port was taken) - #3) We've waited long enough - #(listener_pid can't be reused until we wait on it) - for j in {1..5}; do - if test x"`lsof -i tcp@${listener_addr}:${listener_port} | sed 1d | awk '{prin [...] - break 2; #success, exit outer loop - elif ! ps -p "${listener_pid}" > /dev/null; then - #listener has exited, reap it and go back to start of outer loop - wait "${listener_pid}" - listener_pid= - continue 2 - else - sleep 1 - fi - done - - #We gave up waiting, kill and reap the nc process - kill "${listener_pid}" - wait "${listener_pid}" - listener_pid= -done - -if test x"${listener_pid:-}" = x; then - echo "Failed to find a free port in range ${start_port}-${end_port}" 1>&2 - exit 1 -fi - -if test x"${gateway:-}" != x; then - internal_interface="${gateway/%:*}" - external_interface="${gateway/#*:}" - ssh -F /dev/null -o PasswordAuthentication=no -o PubkeyAuthentication=yes -NR ${ [...] - forward_pid=$! - read -t 30 line < "${forward_fifo}" - if test $? -ne 0; then - echo "Timeout while establishing port forward" 1>&2 - exit 1 - fi - if echo ${line} | grep -q "^Allocated port [[:digit:]]\\+ for remote forward to [...] - listener_port="`echo ${line} | cut -d ' ' -f 3`" - else - echo "Unable to get port forwarded for listener" 1>&2 - echo "Tried: ssh -F /dev/null -o PasswordAuthentication=no -o PubkeyAuthentica [...] - echo "Got: $line" 1>&2 - exit 1 - fi - listener_addr="`get_addr ${external_interface}`" || exit 1 -fi - -echo "${listener_addr}" -echo "${listener_port}" - -while true; do - line="`bgread ${pseudofifo_pid} <&${pseudofifo_handle}`" - if test $? -ne 0; then - echo "Failed to read pseudofifo pid" 1>&2 - exit 1 - fi - echo $line -done diff --git a/scripts/runbenchmark.sh b/scripts/runbenchmark.sh index 6e3c487..e92f0be 100755 --- a/scripts/runbenchmark.sh +++ b/scripts/runbenchmark.sh @@ -7,9 +7,11 @@ set -o nounset trap clean_benchmark EXIT trap 'exit ${error}' TERM INT HUP QUIT +#Precondition: the target is in known_hosts +ssh_opts="-F /dev/null -o StrictHostKeyAuthentication=yes -o CheckHostIP=yes" + tag= session_pid= -listener_pid= benchmark= device= keep= @@ -61,23 +63,12 @@ if test $? -ne 0; then exit 1 fi -. "${topdir}"/scripts/benchutil.sh -if test $? -ne 0; then - echo "+++ Unable to source ${topdir}/benchutil.sh" 1>&2 - exit 1 -fi . "${confdir}/${device}.conf" #We can't use abe's source_config here as it require [...] if test $? -ne 0; then echo "+++ Failed to source ${confdir}/${device}.conf" 1>&2 exit 1 fi -temps="`mktemp -dt XXXXXXXXX`" || exit 1 -listener_file="${temps}/listener_file" -listener_fifo="${temps}/listener_fifo" -mkfifo "${listener_fifo}" || exit 1 -exec {listener_handle}<>${listener_fifo} - #Make sure that subscripts clean up - we must not leave benchmark sources or data [...] clean_benchmark() { @@ -111,47 +102,9 @@ clean_benchmark() wait "${session_pid}" fi - if test x"${listener_pid:-}" != x; then - kill "${listener_pid}" 2>/dev/null - wait "${listener_pid}" - fi - - if test -d "${temps}"; then - exec {listener_handle}>&- - exec {listener_handle}<&- - rm -rf "${temps}" - if test $? -ne 0; then - echo "Failed to delete ${temps}" 1>&2 - error=1 - fi - fi - exit "${error}" } -ssh_opts="-F /dev/null" -establish_listener_opts= - -#Set up our listener -listener_addr="`get_addr`" -if test $? -ne 0; then - echo "Unable to get IP for listener" 1>&2 - exit 1 -fi -"${topdir}"/scripts/establish_listener.sh ${establish_listener_opts} "${listener_a [...] -listener_pid=$! -listener_addr="`bgread -T 60 ${listener_pid} <&${listener_handle}`" -if test $? -ne 0; then - echo "Failed to read listener address" 1>&2 - exit 1 -fi -listener_port="`bgread -T 60 ${listener_pid} <&${listener_handle}`" -if test $? -ne 0; then - echo "Failed to read listener port" 1>&2 - exit 1 -fi -echo "Listener ${listener_addr}:${listener_port}" - if ! (. "${topdir}"/lib/common.sh; remote_exec "${ip}" true ${ssh_opts}) > /dev/nu [...] echo "Unable to connect to target ${ip:-(unknown)} after boot" 1>&2 exit 1 @@ -234,23 +187,13 @@ fi cd `tar tjf ${buildtar} | head -n1` && \ ../controlledrun.sh ${cautious} ${flags} -l ${tee_output} -- ./linarobench.sh [...] ret=\\\$?; \ - listener_found=0; \ - for i in {1..10}; do \ - if ping -c 1 ${listener_addr}; then \ - listener_found=1; \ + while true; do + if ping -i 11 -c 1 `hostname -I`; then \ break; \ fi \ done; \ - if test \\\${listener_found} -eq 1; then \ - echo \"\\\${USER}@\\\`/sbin/ifconfig eth0 | grep 'inet addr' | sed 's/[^:]* [...] - if test \\\${ret} -eq 0; then \ - true; \ - else \ - false; \ - fi \ - else \ - false; \ - fi" \ + echo \\\${ret} > ${target_dir}/RETCODE \ + exit \\\${ret}" \ "${target_dir}/stdout" "${target_dir}/stderr" \ ${ssh_opts} if test $? -ne 0; then @@ -265,28 +208,17 @@ fi )& session_pid=$! -#No sense in setting a deadline on this one, it's order of days for many cases -ip="`bgread ${listener_pid} <&${listener_handle}`" -if test $? -ne 0; then - echo "Failed to read post-benchmark-run IP" 1>&2 - exit 1 -fi - -error="`echo ${ip} | sed 's/.*://'`" +#Wait for a ping from the target +#This assumes that the target's identifier does not change +#This should hold for name in a DNS network, but not necessarily for IP +#Today LAVA lab does not provide DNS, but IP seems stable in practice +#Rather than work around lack of DNS, just make sure we notice if the IP changes +while ! tcpdump -c 1 -i eth0 'icmp and icmp[icmptype]=imcp-echo' | grep -q "${ip} [...] +error="`(. ${topdir}/lib/common.sh; remote_exec "${ip}" 'cat ${target_dir}/RETCODE [...] if test $? -ne 0; then echo "Unable to determine exit code, assuming the worst." 1>&2 error=1 fi -ip="`echo ${ip} | sed 's/:.*//' | sed 's/\s*$//'`" -if test $? -ne 0; then - echo "Unable to determine IP, giving up." 1>&2 - exit 1 -fi - -if ! (. "${topdir}"/lib/common.sh; remote_exec "${ip}" true ${ssh_opts}) > /dev/nu [...] - echo "Unable to connect to target after ${ip:-(unknown)} benchmark run" 1>&2 - exit 1 -fi if test ${error} -ne 0; then echo "Command failed: will try to get logs" 1>&2 -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

[toolchain/abe] branch bernie/jenkmarking-multinode updated (6c6a351 -> 33c2d3a)

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a change to branch bernie/jenkmarking-multinode in repository toolchain/abe. discards 6c6a351 Make an error message more sensible discards d9f7f8a Remove last listener new dcaa216 Remove last listener new 33c2d3a Make an error message more sensible This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (6c6a351) \ N -- N -- N refs/heads/bernie/jenkmarking-multinode (33c2d3a) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omits" are not gone; other references still refer to them. Any revisions marked "discards" are gone forever. The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: scripts/runbenchmark.sh | 5 ----- 1 file changed, 5 deletions(-) -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

[toolchain/abe] 02/02: Make an error message more sensible

by git

9 years, 4 months

[toolchain/abe] 01/02: Remove last listener

by git

This is an automated email from the git hooks/post-receive script. bernie.ogden pushed a commit to branch bernie/jenkmarking-multinode in repository toolchain/abe. commit d9f7f8a6f7b79593047fbdf2c260053f191a5331 Author: Bernard Ogden <bernie.ogden(a)linaro.org> Date: Thu Jul 23 13:49:58 2015 +0200 Remove last listener Remove a bunch of support code that was only there for the listener, too. We replace this with code that sends a ping when the benchmark run finishes on a target. On receipt of ping, the host will try to get the benchmark data from the target that sent the ping (it assumes that the hostname (or IP address, for nameless networks) is stable). For protection against connecting to the wrong target, we require that the public key for each target is in known_hosts before benchmarking begins, and we set StrictHostKeyAuthentication to yes. On the plus side, this is less complicated and thus less error-prone and more maintainable. It's also more secure, in that our only service is now ssh - not ssh + netcat. On the minus side, it is less capable. We confine the scripts to only working: 1) Within a single network (all machines must be able to ping one another) 2) With reliable addresses (stable IP or hostname) If these conditions are not met, failure is reasonably visible: the host will either not notice that the target completed its benchmark (but the user might), or the host will get a message that a target completed the benchmark from a wrong IP address and thus fail to connect to it. Either way, the target will stay up, and so data can be collected 'by hand', provided the necessary private key is availble. Change-Id: Ieb9393a2b9d665eea268dab1ab24232e213360cc --- scripts/benchutil.sh | 107 ------------------------- scripts/establish_listener.sh | 181 ------------------------------------------ scripts/runbenchmark.sh | 91 ++++----------------- 3 files changed, 14 insertions(+), 365 deletions(-) diff --git a/scripts/benchutil.sh b/scripts/benchutil.sh deleted file mode 100644 index 94cf75a..0000000 --- a/scripts/benchutil.sh +++ /dev/null @@ -1,107 +0,0 @@ -set -o pipefail -set -o nounset - -function get_addr -{ - local hostname="${1:-localhost}" - local listener_addr - if test x"${hostname}" = xlocalhost; then - listener_addr="`hostname -I`" - else - listener_addr="`ssh -F /dev/null -o PasswordAuthentication=no -o PubkeyAuthent [...] - fi - if test $? -ne 0; then - echo "Failed to run 'hostname -I' on ${hostname}" 1>&2 - return 1 - fi - - #We only try to support IPv4 for now - listener_addr="`echo ${listener_addr} | tr ' ' '\n' | grep '^\([[:digit:]]\{1,3\ [...] - if test $? -ne 0; then - echo "No IPv4 address found, aborting" 1>&2 - return 1 - fi - - #We don't try to figure out what'll happen if we've got multiple IPv4 interfaces - if test "`echo ${listener_addr} | wc -w`" -ne 1; then - echo "Multiple IPv4 addresses found, aborting" 1>&2 - return 1 - fi - - echo "${listener_addr}" | sed 's/^[[:blank:]]*//' | sed 's/[[:blank:]]*$//' -} - -#Attempt to use read to discover whether there is a record to read from the producer -#If we time out, check to see whether the producer still seems to be alive. -#We can check more than one pid, if we have visibility of some other process(s) that we -#would also like to monitor while we wait to read something. If any of these -#processes seem dead then return 2, otherwise keep trying to read. -#Once we've established that there seems to be a record, try to read it with a -#read timeout. If we fail to read within read_timeout, return 1 to indicate -#read failure - but the producer may still be alive in this case. -#Can also set a deadline - bgread will return 3 if it hasn't seen any new output -#before deadline expires. deadline is only checked at read_timeout intervals. -#Typical invocation: foo="`bgread ${child_pid} <&3`" -#Invocation with read checks every 5 seconds, failure after 2 minutes and two -#pids to monitor: -#foo="`bgread -T 120 -t 5 ${child_pid} ${other_pid} <&3`" -function bgread -{ - OPTIND=1 - local read_timeout=60 - local deadline= - local pid - - while getopts T:t: flag; do - case "${flag}" in - t) read_timeout="${OPTARG}";; - T) deadline="$((${OPTARG} + `date +%s`))";; - *) - echo "Bad arg" 1>&2 - return 1 - ;; - esac - done - shift $((OPTIND - 1)) - - if test $# -eq 0; then - echo "No pid(s) to poll!" 1>&2 - return 1 - fi - local buffer='' - local line='' - - #We have to be careful here. If the read times out when there was a partial - #record on the fifo then the part that has been read just gets discarded. We - #can avoid this problem by using -N to ensure that we read the minimal amount - #and DO NOT discard it. -N 0 might be intuited to do the right thing, but is - #arguably undefined behaviour and empirically doesn't work. - #Read 1 char then timeout if it isn't a delimiter: buffer is the char, exit code 0 OR - #Read the delimiter, don't timeout: buffer is empty, exit code 0 OR - #Fail to read any chars coz there aren't any, then timeout: buffer is empty, exi [...] - while ! read -N 1 -t "${read_timeout}" buffer; do - for pid in "$@"; do - kill -0 "${pid}" > /dev/null 2>&1 || return 2 - done - if test x"${deadline:-}" != x; then - if test `date +%s` -ge ${deadline}; then - echo "bgread timed out" 1>&2 - return 3 - fi - fi - done - - #If we get here, we managed to read 1 char. If we have a null string just - #return it (the record was empty). Otherwise, assume the rest of the record is r [...] - #especially within the generous timeout that we allow. - if test x"${buffer:-}" != x; then - read -t 60 line - if test $? -ne 0; then - echo "Record did not complete" 1>&2 - return 1 - fi - fi - echo "${buffer}${line}" - return 0 -} - diff --git a/scripts/establish_listener.sh b/scripts/establish_listener.sh deleted file mode 100755 index 5266faa..0000000 --- a/scripts/establish_listener.sh +++ /dev/null @@ -1,181 +0,0 @@ -#!/bin/bash -set -o pipefail -set -o nounset - -# load the configure file produced by configure -if test -e "${PWD}/host.conf"; then - . "${PWD}/host.conf" -else - echo "ERROR: no host.conf file! Did you run configure?" 1>&2 - exit 1 -fi -topdir="${abe_path}" #abe global, but this should be the right value for abe -. "${topdir}"/scripts/benchutil.sh -if test $? -ne 0; then - echo "Unable to source ${topdir}/benchutil.sh" 1>&2 - exit 1 -fi - -trap cleanup EXIT -trap 'exit ${error}' TERM INT HUP QUIT - -error=1 -listener_pid= -forward_pid= -pseudofifo_pid= -temps="`mktemp -dt XXXXXXXXX`" || exit 1 -listener_file="${temps}/listener_file" -gateway= - -function cleanup -{ - error=$? - if test x"${listener_pid:-}" != x; then - kill "${listener_pid}" 2>/dev/null - wait "${listener_pid}" - fi - if test x"${pseudofifo_pid:-}" != x; then - kill "${pseudofifo_pid}" 2>/dev/null - #Substituted process is not our child and cannot be waited on. Fortunately, - #it doesn't matter too much when it dies. - fi - if test x"${forward_pid:-}" != x; then - kill "${forward_pid}" 2>/dev/null - wait "${forward_pid}" - fi - if test -d "${temps}"; then - exec {pseudofifo_handle}<&- - rm -rf ${temps} - if test $? -ne 0; then - echo "Failed to delete temporary file store ${temps}" 1>&2 - fi - error=1 - fi - exit "${error}" -} - -#A fifo would make much more sense, but nc doesn't like it -touch "${listener_file}" -if test $? -ne 0; then - echo "Failed to create listener file '${listener_file}'" 1>&2 - exit 1 -fi - -#The trap is just to suppress the 'Terminated' message -exec {pseudofifo_handle}< <(trap 'exit' TERM; tail -f "${listener_file}"& echo $! [...] -read -t 60 pseudofifo_pid <&"${pseudofifo_handle}" -if test $? -ne 0; then - echo "Failed to read pseudofifo pid" 1>&2 - exit 1 -fi - -forward_fifo="${temps}/forward_fifo" -mkfifo "${forward_fifo}" || exit 1 - -while getopts f: flag; do - case "${flag}" in - f) gateway="${OPTARG}" ;; - *) - echo "Bad arg" 1>&2 - exit 1 - ;; - esac -done -shift $((OPTIND - 1)) -if test $# -ne 3; then - echo "establish_listener needs 3 args, got $#" 1>&2 - for arg in "$@"; do echo "${arg}" 1>&2; done - exit 1 -fi -if test x"${gateway:-}" != x; then - if ! echo "${gateway}" | grep -q '.\+:.\+'; then - echo "If specifying a gateway to forward through, must be in format 'internal_ [...] - echo "Got: ${gateway}" 1>&2 - exit 1 - fi -fi - -listener_addr="$1" -ping -c 1 "${listener_addr}" > /dev/null -if test $? -ne 0; then - echo "Unable to ping host ${listener_addr}" 1>&2 - exit 1 -fi -start_port="$2" -end_port="$3" - -for ((listener_port=${start_port}; listener_port < ${end_port}; listener_port++)); do - #Try to listen on the port. nc will fail if something has snatched it. - echo "Attempting to establish listener on ${listener_addr}:${listener_port}" 1>&2 - - if test -e /bin/nc.traditional; then - /bin/nc.traditional -l -s "${listener_addr}" -p "${listener_port}" >> "${liste [...] - elif test -e /bin/nc.openbsd; then - /bin/nc.openbsd -l "${listener_addr}" "${listener_port}" >> "${liste [...] - else - echo "Unable to identify netcat" 1>&2 - exit 1 - fi - listener_pid=$! - - #nc doesn't confirm that it's got the port, so we spin until either: - #1) We see that the port has been taken by our process - #2) We see our process exit (implying that the port was taken) - #3) We've waited long enough - #(listener_pid can't be reused until we wait on it) - for j in {1..5}; do - if test x"`lsof -i tcp@${listener_addr}:${listener_port} | sed 1d | awk '{prin [...] - break 2; #success, exit outer loop - elif ! ps -p "${listener_pid}" > /dev/null; then - #listener has exited, reap it and go back to start of outer loop - wait "${listener_pid}" - listener_pid= - continue 2 - else - sleep 1 - fi - done - - #We gave up waiting, kill and reap the nc process - kill "${listener_pid}" - wait "${listener_pid}" - listener_pid= -done - -if test x"${listener_pid:-}" = x; then - echo "Failed to find a free port in range ${start_port}-${end_port}" 1>&2 - exit 1 -fi - -if test x"${gateway:-}" != x; then - internal_interface="${gateway/%:*}" - external_interface="${gateway/#*:}" - ssh -F /dev/null -o PasswordAuthentication=no -o PubkeyAuthentication=yes -NR ${ [...] - forward_pid=$! - read -t 30 line < "${forward_fifo}" - if test $? -ne 0; then - echo "Timeout while establishing port forward" 1>&2 - exit 1 - fi - if echo ${line} | grep -q "^Allocated port [[:digit:]]\\+ for remote forward to [...] - listener_port="`echo ${line} | cut -d ' ' -f 3`" - else - echo "Unable to get port forwarded for listener" 1>&2 - echo "Tried: ssh -F /dev/null -o PasswordAuthentication=no -o PubkeyAuthentica [...] - echo "Got: $line" 1>&2 - exit 1 - fi - listener_addr="`get_addr ${external_interface}`" || exit 1 -fi - -echo "${listener_addr}" -echo "${listener_port}" - -while true; do - line="`bgread ${pseudofifo_pid} <&${pseudofifo_handle}`" - if test $? -ne 0; then - echo "Failed to read pseudofifo pid" 1>&2 - exit 1 - fi - echo $line -done diff --git a/scripts/runbenchmark.sh b/scripts/runbenchmark.sh index 6e3c487..a277805 100755 --- a/scripts/runbenchmark.sh +++ b/scripts/runbenchmark.sh @@ -7,9 +7,11 @@ set -o nounset trap clean_benchmark EXIT trap 'exit ${error}' TERM INT HUP QUIT +#Precondition: the target is in known_hosts +ssh_opts="-F /dev/null -o StrictHostKeyAuthentication=yes -o CheckHostIP=yes" + tag= session_pid= -listener_pid= benchmark= device= keep= @@ -72,12 +74,6 @@ if test $? -ne 0; then exit 1 fi -temps="`mktemp -dt XXXXXXXXX`" || exit 1 -listener_file="${temps}/listener_file" -listener_fifo="${temps}/listener_fifo" -mkfifo "${listener_fifo}" || exit 1 -exec {listener_handle}<>${listener_fifo} - #Make sure that subscripts clean up - we must not leave benchmark sources or data [...] clean_benchmark() { @@ -111,47 +107,9 @@ clean_benchmark() wait "${session_pid}" fi - if test x"${listener_pid:-}" != x; then - kill "${listener_pid}" 2>/dev/null - wait "${listener_pid}" - fi - - if test -d "${temps}"; then - exec {listener_handle}>&- - exec {listener_handle}<&- - rm -rf "${temps}" - if test $? -ne 0; then - echo "Failed to delete ${temps}" 1>&2 - error=1 - fi - fi - exit "${error}" } -ssh_opts="-F /dev/null" -establish_listener_opts= - -#Set up our listener -listener_addr="`get_addr`" -if test $? -ne 0; then - echo "Unable to get IP for listener" 1>&2 - exit 1 -fi -"${topdir}"/scripts/establish_listener.sh ${establish_listener_opts} "${listener_a [...] -listener_pid=$! -listener_addr="`bgread -T 60 ${listener_pid} <&${listener_handle}`" -if test $? -ne 0; then - echo "Failed to read listener address" 1>&2 - exit 1 -fi -listener_port="`bgread -T 60 ${listener_pid} <&${listener_handle}`" -if test $? -ne 0; then - echo "Failed to read listener port" 1>&2 - exit 1 -fi -echo "Listener ${listener_addr}:${listener_port}" - if ! (. "${topdir}"/lib/common.sh; remote_exec "${ip}" true ${ssh_opts}) > /dev/nu [...] echo "Unable to connect to target ${ip:-(unknown)} after boot" 1>&2 exit 1 @@ -234,23 +192,13 @@ fi cd `tar tjf ${buildtar} | head -n1` && \ ../controlledrun.sh ${cautious} ${flags} -l ${tee_output} -- ./linarobench.sh [...] ret=\\\$?; \ - listener_found=0; \ - for i in {1..10}; do \ - if ping -c 1 ${listener_addr}; then \ - listener_found=1; \ + while true; do + if ping -i 11 -c 1 `hostname -I`; then \ break; \ fi \ done; \ - if test \\\${listener_found} -eq 1; then \ - echo \"\\\${USER}@\\\`/sbin/ifconfig eth0 | grep 'inet addr' | sed 's/[^:]* [...] - if test \\\${ret} -eq 0; then \ - true; \ - else \ - false; \ - fi \ - else \ - false; \ - fi" \ + echo \\\${ret} > ${target_dir}/RETCODE \ + exit \\\${ret}" \ "${target_dir}/stdout" "${target_dir}/stderr" \ ${ssh_opts} if test $? -ne 0; then @@ -265,28 +213,17 @@ fi )& session_pid=$! -#No sense in setting a deadline on this one, it's order of days for many cases -ip="`bgread ${listener_pid} <&${listener_handle}`" -if test $? -ne 0; then - echo "Failed to read post-benchmark-run IP" 1>&2 - exit 1 -fi - -error="`echo ${ip} | sed 's/.*://'`" +#Wait for a ping from the target +#This assumes that the target's identifier does not change +#This should hold for name in a DNS network, but not necessarily for IP +#Today LAVA lab does not provide DNS, but IP seems stable in practice +#Rather than work around lack of DNS, just make sure we notice if the IP changes +while ! tcpdump -c 1 -i eth0 'icmp and icmp[icmptype]=imcp-echo' | grep -q "${ip} [...] +error="`(. ${topdir}/lib/common.sh; remote_exec "${ip}" 'cat ${target_dir}/RETCODE [...] if test $? -ne 0; then echo "Unable to determine exit code, assuming the worst." 1>&2 error=1 fi -ip="`echo ${ip} | sed 's/:.*//' | sed 's/\s*$//'`" -if test $? -ne 0; then - echo "Unable to determine IP, giving up." 1>&2 - exit 1 -fi - -if ! (. "${topdir}"/lib/common.sh; remote_exec "${ip}" true ${ssh_opts}) > /dev/nu [...] - echo "Unable to connect to target after ${ip:-(unknown)} benchmark run" 1>&2 - exit 1 -fi if test ${error} -ne 0; then echo "Command failed: will try to get logs" 1>&2 -- To stop receiving notification emails like this one, please contact the administrator of this repository.

9 years, 4 months

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

Tcwg-commits July 2015