This is an automated email from the git hooks/post-receive script.

bernie.ogden pushed a change to branch bernie/benchmarking-uinstance-arbitrary in repository toolchain/abe.

discards a7f2dbd Support injection of metadata into test runs discards 09fb5ea Replace output_param with output_{value,slot} discards 9be83a4 Apply NAME=VALUE CLI overrides before Benchmark.sh discards a1eeab3 Move output_param to top of Benchmark.sh discards 270c091 s/bench-session-debian/host-session/ discards b09f36a Tighten CoremarkPro gather script, add many tests discards 122c2c0 Template for use in trusted LAVA uinstance discards bbb395c Add missing tcpdump dependency discards 43c9228 Permit host machine to be of any type discards 09bdfbd Close all tmpdir files before deletion discards 1d8de3f Benchmark.sh: Support mustang host/target discards eb64205 Benchmark.sh: Support deploy_linaro_kernel discards bd88f32 Preserve stdout/stderr separation in cmd output discards 50d142c Output 'Running ${cmd}' to the log discards aa42d7d tee stderr to log, when there is a log. discards 2d538c1 Always set safe_output when in a trusted LAVA discards eac453a controlledrun.sh logs to /dev/console if running in LAVA discards 00b4918 controlledrun.sh output log set by board conf discards 467b46c Increase default timeout from 0.5 to 1.5 hours. discards e02a4ac Make TIMEOUT overrideable discards dc7208a Remove redundant boot_timeout parameter discards 3c16f60 Update CLI-related messages discards 4ac9fa7 Remove redundant 'tag' option discards ca1a754 Some more detail around data collection message discards 005130d Make ssh messages depend on instance discards f06b375 Defer to user-specified TARGET_SESSION discards 2b80013 Rationalize *-session YAML discards 4b05f35 Do not infer device validation from triple discards d5fc4fa Decouple triple from toolchain discards 4776e83 host-session: shift CLI args when parsing discards 46515e0 host-session invokes benchmark scripts with -x discards 21191fe Remove redundant ABE_BRANCH parameter discards 7c16e82 host-session: Consistent use of {} with args discards 2c45884 gather scripts: Provide mime type for attachments discards 718f081 benchmark.sh preserves -x for runbenchmark.sh discards db734ca gather: set -x in all gather scripts discards 3ee3e85 Return gather script exit code within post-run cmd discards 739ef0f Wait for post-run command to complete discards ce9723a Convert post-run cmd to function discards a83bb6f Convert post-run command into multiline string discards bc78241 Add comments about post-run/gather.sh discards 0a2c98d target-session: Tidy up kill/wait code discards 5a2d355 Simplify post-run-cmd-not-set NOP discards fee0475 Check post-run command exit code discards 2737c8f runbenchmark.sh: Fix indentation discards 664a22a Match tcpdump output even if user gave username discards dbb5765 Match tcpdump output even if user gave hostnames discards ab195d6 Do not report names in tcpdump discards 00efa90 benchmark.sh: Rename GCC to something more generic discards 7564dc4 Clean up toolchain usage discards 5247c78 Ensure that unset params get value None discards 98cb66e CoremarkPro.conf: Fix native paths discards 538e8c9 CoremarkPro.sh: Report sizes first discards 840c59c CoremarkPro.sh: Require exactly one log discards a2f6a36 CoremarkPro.sh: Remove commented-out code discards d6f9433 CoremarkPro.sh: Pass name to report_measured discards 5333caf CoremarkPro.sh: Make function-local var local discards cc8f4c6 CoremarkPro.sh: Count everything that can be multiple discards f08116e CoremarkPro.sh: Cosmetic changes to test IDs discards 875550c CoremarkPro.sh: Eliminate spaces in TEST_CASE_ID discards 4294622 CoremarkPro.sh: [] notation for iterations only discards 40401bf CoremarkPro.sh: Name size measurements sanely discards 85eb137 CoremarkPro.sh: Whitespace: Old comment,blank line discards 3791bcd test_CoremarkPro.sh: Don't output errors when testing fail cases discards d3d5564 CoremarkPro.sh: Attach raw data before processing discards 0e89377 CoremarkPro.sh: Unique names for time and rate discards 3bdbebf Gather script for Coremark Pro discards 4d88d39 dispatch-benchmark.py: Support Coremark Pro discards 3fa590a CoremarkPro support discards d30aab9 Add some (commented-out) debugging output discards 0090cf7 Always read configs with target context discards f7cd930 Apply target context when looking for binaries discards 850f9e8 Support external sysroot discards ac935d1 dispatch-benchmark.py: More detail on keyring fail discards 670b53f dispatch-benchmark.py: Option to set bundle stream discards aae6971 dispatch-benchmark.py: Validate input discards 4f3b11e dispatch-benchmark.py: Message on dry-run exit discards e2916a4 dispatch-benchmark.py: Move dryrun exit to more natural location discards 0966ff6 dispatch-benchmark.py: Script to dispatch benchmarks discards 3fe5540 template.yaml: Metadata must be a string discards efcde62 Remove old jobdefs discards 2872939 Benchmark.sh,template.yaml: s/BUNDLE_STREAM_NAME/BUNDLE_STREAM/ discards db3b049 Benchmark.sh: Make bundle stream default paranoid discards 592e193 Benchmark.sh: Handle enquoting in parameters discards ae8681e Benchmark.sh: Set BUNDLE_STREAM_NAME only if unset discards 25154c3 Benchmark.sh: Set the right default branch discards 6c3a29c Benchmark.sh: Remove redundant comments discards 2405952 Benchmark.sh,template.yaml: Map from target type to image discards c4ebf8e Benchmark.sh,template.yaml: Accept public key discards 4c2f3f8 Benchmark.sh,template.yaml: s/ABE_/TESTDEF_/ discards 0731d06 Benchmark.sh: Make LAVA_USER available to params discards c441b22 Benchmark.sh: Get bundle server from external LAVA_SERVER def discards da36899 Benchmark.sh: Braces-and-spaces discards d6ec046 Benchmark.sh: Allow optional parms to be undefined discards de3233a Import template.yaml from lava-jobs discards 7cb078f Add script to derive all job parameters discards 15581eb Remove redundant TODO discards 7b847ed WIP: 'Semi-trusted' concept, for testing purposes discards 86f4ae8 CPU2006.sh: Attach raw output to run discards 145055f fakebench.sh: Attach to run, not a test case discards db6df86 Gather script + tests for CPU2006 adds 38363e1 Gather script + tests for CPU2006 adds ba30d67 fakebench.sh: Attach to run, not a test case adds bfb5cff CPU2006.sh: Attach raw output to run adds b204cdd Rework CPU2006.sh to use 'raw' output adds bb7e88a test_CPU2006.sh: Give doit a sensible name adds 983f3ad Rename *CPU2006.sh to *CPU200x.sh adds 94dc098 test_CPU200x: Paramaterise bits that vary for 2k adds 9e7c2c0 CPU200x*: Support CPU2000 results. adds 3a3e9b8 Lead regexp with ^ adds d1750e6 Remove unused variables adds d1c8566 raw is global, stop passing it around everywhere. adds fc8691f Handle different spellings of benchmark names adds bdda15a Move boilerplate to generate_benchmark and rename adds 199c6b6 Test cases where only int or fp was run adds 6bbfb61 Add (commented-out) debugging printf adds 8be0d1c Add a set -o pipefail adds 01c23fe Add exit, error handlers adds dcf59b2 Separate benchset definitions from individual runs adds 399b3f1 Add tests for individual benchmark runs adds 1e8a4ca Test random mixes of subbenchmarks adds 9753b68 Fix off-by-1 in iteration counts adds b39d49f Test non-ref input sizes adds 549471a Update error message adds 48f939d Add tests for cases where the script should fail adds 48be5f3 Correct naming style of test input adds 80f7cd6 Test more realistic values for input sizes adds aeed551 Print the leading 0 if integer part is 0 adds bcdad70 Make locals in generate_subbenchmark actually local adds 047be3e Multiply ratios for CPU2000 adds f6ec775 Use spec.cpu200x.invalid rather than ...valid adds 2aa9d8a Test case where some subbenchmarks fail adds ec39941 WIP: 'Semi-trusted' concept, for testing purposes adds c54aeb1 Remove redundant TODO adds 45b7b6a Add script to derive all job parameters adds 4246b1d Import template.yaml from lava-jobs adds 31ba4d2 Benchmark.sh: Allow optional parms to be undefined adds 439ea60 Benchmark.sh: Braces-and-spaces adds 92452c0 Benchmark.sh: Get bundle server from external LAVA_SERVER def adds 35304cd Benchmark.sh: Make LAVA_USER available to params adds 81b344d Benchmark.sh,template.yaml: s/ABE_/TESTDEF_/ adds 63475a5 Benchmark.sh,template.yaml: Accept public key adds 3b6a571 Benchmark.sh,template.yaml: Map from target type to image adds 2a92992 Benchmark.sh: Remove redundant comments adds 247523b Benchmark.sh: Set the right default branch adds 637d2df Benchmark.sh: Set BUNDLE_STREAM_NAME only if unset adds b570880 Benchmark.sh: Handle enquoting in parameters adds 72bd804 Benchmark.sh: Make bundle stream default paranoid adds a10bf01 Benchmark.sh,template.yaml: s/BUNDLE_STREAM_NAME/BUNDLE_STREAM/ adds 631f66b Remove old jobdefs adds 858b801 template.yaml: Metadata must be a string adds c69cfac dispatch-benchmark.py: Script to dispatch benchmarks adds e0c65e3 dispatch-benchmark.py: Move dryrun exit to more natural location adds c7e8f86 dispatch-benchmark.py: Message on dry-run exit adds aac012d dispatch-benchmark.py: Validate input adds 686e747 dispatch-benchmark.py: Option to set bundle stream adds 3326e0e dispatch-benchmark.py: More detail on keyring fail adds 18d9438 Support external sysroot adds f0b21cc Apply target context when looking for binaries adds 2d5cb47 Always read configs with target context adds d859677 Add some (commented-out) debugging output adds 8aa4179 CoremarkPro support adds a0e931d dispatch-benchmark.py: Support Coremark Pro adds da34e4d Gather script for Coremark Pro adds cc45976 CoremarkPro.sh: Unique names for time and rate adds dc788c1 CoremarkPro.sh: Attach raw data before processing adds fb29817 test_CoremarkPro.sh: Don't output errors when testing fail cases adds 5aff508 CoremarkPro.sh: Whitespace: Old comment,blank line adds c5c4cca CoremarkPro.sh: Name size measurements sanely adds 3b9a139 CoremarkPro.sh: [] notation for iterations only adds 0983a16 CoremarkPro.sh: Eliminate spaces in TEST_CASE_ID adds cace859 CoremarkPro.sh: Cosmetic changes to test IDs adds dc11405 CoremarkPro.sh: Count everything that can be multiple adds 15471cd CoremarkPro.sh: Make function-local var local adds 400ed9f CoremarkPro.sh: Pass name to report_measured adds 1a29fef CoremarkPro.sh: Remove commented-out code adds ba76222 CoremarkPro.sh: Require exactly one log adds 54294e4 CoremarkPro.sh: Report sizes first adds 51f8968 CoremarkPro.conf: Fix native paths adds d49b1cf Ensure that unset params get value None adds f9ba865 Clean up toolchain usage adds bbc24f4 benchmark.sh: Rename GCC to something more generic adds a6460a3 Do not report names in tcpdump adds 524d996 Match tcpdump output even if user gave hostnames adds 73e6e18 Match tcpdump output even if user gave username adds 202a1fb runbenchmark.sh: Fix indentation adds e97e0c1 Check post-run command exit code adds 598dbbf Simplify post-run-cmd-not-set NOP adds f64d29b target-session: Tidy up kill/wait code adds d5fbb5c Add comments about post-run/gather.sh adds 2567eda Convert post-run command into multiline string adds fa2edb7 Convert post-run cmd to function adds e2da049 Wait for post-run command to complete adds cade9d8 Return gather script exit code within post-run cmd adds 0925d63 gather: set -x in all gather scripts adds 73804fc benchmark.sh preserves -x for runbenchmark.sh adds 733f894 gather scripts: Provide mime type for attachments adds 8ea5146 host-session: Consistent use of {} with args adds 4b43ccb Remove redundant ABE_BRANCH parameter adds d7c4e10 host-session invokes benchmark scripts with -x adds 1355fa0 host-session: shift CLI args when parsing adds 237c840 Decouple triple from toolchain adds 4dd43f1 Do not infer device validation from triple adds 1071f91 Rationalize *-session YAML adds 64bbb11 Defer to user-specified TARGET_SESSION adds bf530ad Make ssh messages depend on instance adds dd73e3a Some more detail around data collection message adds 8c7b632 Remove redundant 'tag' option adds 5cabcee Update CLI-related messages adds fd15217 Remove redundant boot_timeout parameter adds 10a6d5e Make TIMEOUT overrideable adds 7935ece Increase default timeout from 0.5 to 1.5 hours. adds a8ba392 controlledrun.sh output log set by board conf adds 254f27c controlledrun.sh logs to /dev/console if running in LAVA adds 8c9803f Always set safe_output when in a trusted LAVA adds 21dcf20 tee stderr to log, when there is a log. adds 6ae1b98 Output 'Running ${cmd}' to the log adds b9a84c8 Preserve stdout/stderr separation in cmd output adds ccbb3a3 Benchmark.sh: Support deploy_linaro_kernel adds c0972b5 Benchmark.sh: Support mustang host/target adds 58d9c03 Close all tmpdir files before deletion adds f6eaed0 Permit host machine to be of any type adds 714232f Add missing tcpdump dependency adds 7df5dad Template for use in trusted LAVA uinstance adds 0a3303f Tighten CoremarkPro gather script, add many tests adds bd469b8 s/bench-session-debian/host-session/ adds 1a3da32 Move output_param to top of Benchmark.sh adds 2e8f956 Apply NAME=VALUE CLI overrides before Benchmark.sh adds 70dcb35 Replace output_param with output_{value,slot} adds 60cf45a Support injection of metadata into test runs

This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this:

* -- * -- B -- O -- O -- O (a7f2dbd) \ N -- N -- N refs/heads/bernie/benchmarking-uinstance-arbitrary (60cf45a)

You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B.

Any revisions marked "omits" are not gone; other references still refer to them. Any revisions marked "discards" are gone forever.

No new revisions were added by this update.

Summary of changes: config/bench/gather/CPU2006.sh | 202 ----------------- config/bench/gather/CPU200x.sh | 207 +++++++++++++++++ config/bench/gather/test_CPU2006.sh | 108 --------- config/bench/gather/test_CPU200x.sh | 337 ++++++++++++++++++++++++++++ config/bench/lava/target-session-tools.yaml | 1 + 5 files changed, 545 insertions(+), 310 deletions(-) delete mode 100755 config/bench/gather/CPU2006.sh create mode 100755 config/bench/gather/CPU200x.sh delete mode 100755 config/bench/gather/test_CPU2006.sh create mode 100755 config/bench/gather/test_CPU200x.sh