This is an automated email from the git hooks/post-receive script.

bernie.ogden pushed a change to branch origin/bernie/benchmarking-uinstance-stability in repository toolchain/abe.

at 60cf45a Support injection of metadata into test runs

This branch includes the following new commits:

new b204cdd Rework CPU2006.sh to use 'raw' output new bb7e88a test_CPU2006.sh: Give doit a sensible name new 983f3ad Rename *CPU2006.sh to *CPU200x.sh new 94dc098 test_CPU200x: Paramaterise bits that vary for 2k new 9e7c2c0 CPU200x*: Support CPU2000 results. new 3a3e9b8 Lead regexp with ^ new d1750e6 Remove unused variables new d1c8566 raw is global, stop passing it around everywhere. new fc8691f Handle different spellings of benchmark names new bdda15a Move boilerplate to generate_benchmark and rename new 199c6b6 Test cases where only int or fp was run new 6bbfb61 Add (commented-out) debugging printf new 8be0d1c Add a set -o pipefail new 01c23fe Add exit, error handlers new dcf59b2 Separate benchset definitions from individual runs new 399b3f1 Add tests for individual benchmark runs new 1e8a4ca Test random mixes of subbenchmarks new 9753b68 Fix off-by-1 in iteration counts new b39d49f Test non-ref input sizes new 549471a Update error message new 48f939d Add tests for cases where the script should fail new 48be5f3 Correct naming style of test input new 80f7cd6 Test more realistic values for input sizes new aeed551 Print the leading 0 if integer part is 0 new bcdad70 Make locals in generate_subbenchmark actually local new 047be3e Multiply ratios for CPU2000 new f6ec775 Use spec.cpu200x.invalid rather than ...valid new 2aa9d8a Test case where some subbenchmarks fail new ec39941 WIP: 'Semi-trusted' concept, for testing purposes new c54aeb1 Remove redundant TODO new 45b7b6a Add script to derive all job parameters new 4246b1d Import template.yaml from lava-jobs new 31ba4d2 Benchmark.sh: Allow optional parms to be undefined new 439ea60 Benchmark.sh: Braces-and-spaces new 92452c0 Benchmark.sh: Get bundle server from external LAVA_SERVER def new 35304cd Benchmark.sh: Make LAVA_USER available to params new 81b344d Benchmark.sh,template.yaml: s/ABE_/TESTDEF_/ new 63475a5 Benchmark.sh,template.yaml: Accept public key new 3b6a571 Benchmark.sh,template.yaml: Map from target type to image new 2a92992 Benchmark.sh: Remove redundant comments new 247523b Benchmark.sh: Set the right default branch new 637d2df Benchmark.sh: Set BUNDLE_STREAM_NAME only if unset new b570880 Benchmark.sh: Handle enquoting in parameters new 72bd804 Benchmark.sh: Make bundle stream default paranoid new a10bf01 Benchmark.sh,template.yaml: s/BUNDLE_STREAM_NAME/BUNDLE_STREAM/ new 631f66b Remove old jobdefs new 858b801 template.yaml: Metadata must be a string new c69cfac dispatch-benchmark.py: Script to dispatch benchmarks new e0c65e3 dispatch-benchmark.py: Move dryrun exit to more natural location new c7e8f86 dispatch-benchmark.py: Message on dry-run exit new aac012d dispatch-benchmark.py: Validate input new 686e747 dispatch-benchmark.py: Option to set bundle stream new 3326e0e dispatch-benchmark.py: More detail on keyring fail new 18d9438 Support external sysroot new f0b21cc Apply target context when looking for binaries new 2d5cb47 Always read configs with target context new d859677 Add some (commented-out) debugging output new 8aa4179 CoremarkPro support new a0e931d dispatch-benchmark.py: Support Coremark Pro new da34e4d Gather script for Coremark Pro new cc45976 CoremarkPro.sh: Unique names for time and rate new dc788c1 CoremarkPro.sh: Attach raw data before processing new fb29817 test_CoremarkPro.sh: Don't output errors when testing fail cases new 5aff508 CoremarkPro.sh: Whitespace: Old comment,blank line new c5c4cca CoremarkPro.sh: Name size measurements sanely new 3b9a139 CoremarkPro.sh: [] notation for iterations only new 0983a16 CoremarkPro.sh: Eliminate spaces in TEST_CASE_ID new cace859 CoremarkPro.sh: Cosmetic changes to test IDs new dc11405 CoremarkPro.sh: Count everything that can be multiple new 15471cd CoremarkPro.sh: Make function-local var local new 400ed9f CoremarkPro.sh: Pass name to report_measured new 1a29fef CoremarkPro.sh: Remove commented-out code new ba76222 CoremarkPro.sh: Require exactly one log new 54294e4 CoremarkPro.sh: Report sizes first new 51f8968 CoremarkPro.conf: Fix native paths new d49b1cf Ensure that unset params get value None new f9ba865 Clean up toolchain usage new bbc24f4 benchmark.sh: Rename GCC to something more generic new a6460a3 Do not report names in tcpdump new 524d996 Match tcpdump output even if user gave hostnames new 73e6e18 Match tcpdump output even if user gave username new 202a1fb runbenchmark.sh: Fix indentation new e97e0c1 Check post-run command exit code new 598dbbf Simplify post-run-cmd-not-set NOP new f64d29b target-session: Tidy up kill/wait code new d5fbb5c Add comments about post-run/gather.sh new 2567eda Convert post-run command into multiline string new fa2edb7 Convert post-run cmd to function new e2da049 Wait for post-run command to complete new cade9d8 Return gather script exit code within post-run cmd new 0925d63 gather: set -x in all gather scripts new 73804fc benchmark.sh preserves -x for runbenchmark.sh new 733f894 gather scripts: Provide mime type for attachments new 8ea5146 host-session: Consistent use of {} with args new 4b43ccb Remove redundant ABE_BRANCH parameter new d7c4e10 host-session invokes benchmark scripts with -x new 1355fa0 host-session: shift CLI args when parsing new 237c840 Decouple triple from toolchain new 4dd43f1 Do not infer device validation from triple new 1071f91 Rationalize *-session YAML new 64bbb11 Defer to user-specified TARGET_SESSION new bf530ad Make ssh messages depend on instance new dd73e3a Some more detail around data collection message new 8c7b632 Remove redundant 'tag' option new 5cabcee Update CLI-related messages new fd15217 Remove redundant boot_timeout parameter new 10a6d5e Make TIMEOUT overrideable new 7935ece Increase default timeout from 0.5 to 1.5 hours. new a8ba392 controlledrun.sh output log set by board conf new 254f27c controlledrun.sh logs to /dev/console if running in LAVA new 8c9803f Always set safe_output when in a trusted LAVA new 21dcf20 tee stderr to log, when there is a log. new 6ae1b98 Output 'Running ${cmd}' to the log new b9a84c8 Preserve stdout/stderr separation in cmd output new ccbb3a3 Benchmark.sh: Support deploy_linaro_kernel new c0972b5 Benchmark.sh: Support mustang host/target new 58d9c03 Close all tmpdir files before deletion new f6eaed0 Permit host machine to be of any type new 714232f Add missing tcpdump dependency new 7df5dad Template for use in trusted LAVA uinstance new 0a3303f Tighten CoremarkPro gather script, add many tests new bd469b8 s/bench-session-debian/host-session/ new 1a3da32 Move output_param to top of Benchmark.sh new 2e8f956 Apply NAME=VALUE CLI overrides before Benchmark.sh new 70dcb35 Replace output_param with output_{value,slot} new 60cf45a Support injection of metadata into test runs

The 126 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference.