[Tcwg-commits] [toolchain/abe] branch bernie/benchmarking-uinstance-stability updated (bf503f6 -> 2bf6b4a)

This is an automated email from the git hooks/post-receive script.

bernie.ogden pushed a change to branch bernie/benchmarking-uinstance-stability in repository toolchain/abe.

discards bf503f6 Add a TODO about setting COMPILER_FLAGS on CLI discards 7137218 Pass COMPILER_FLAGS through default_makeflags discards 3bb5cb1 Use LAVA_JOB_NAME, not JOB_NAME, in LAVA wrapper discards 5003deb Benchmark.sh: Only set timeout if user didn't discards c6788f5 Remove EEMBC.conf discards b9951c4 Benchmark.sh: Add comment to validation func discards cd2d3ec Benchmark.sh: Set timeout depending on benchmark. discards 9768ccf Set 'build' in Coremark-Pro default_makeflags discards 2b6d035 Get Coremark-Pro from new Coremark-Pro repository discards 3a3fc72 host-session: Add missing 'test' discards 41a221c Handle possibility that gather script is a symlink discards 1b1fd40 CPU200x.sh: Work with arbitrary name discards a670f6c Benchmark.sh: Isomorphic tidying of validate() discards f4f7b4f Benchmark.sh: Catch 2 triple/toolchain mismatches discards 8da0346 CPU200x: Fix unit for geomean of runtime discards b8a51ce Benchmark.sh: Ensure BUNDLE_STREAM has trailing / discards e61bf86 Benchmark.sh: Validate/fix up {LAVA,BUNDLE}_SERVER discards 8ebcf30 Benchmark.sh: Let user override BUNDLE_SERVER discards 55df560 dispatch-benchmark.py: Force explicit --triple discards 9bc55ac Create/support CPU200{0,6}.sh symlinks discards b4e49d2 Confirm that one of TOOLCHAIN and PREBUILT is set discards 9c27017 Only look up token in keychain for non-dry-runs discards 81f2084 Make JOB_NAME user-settable discards 5e4b212 Centralize default-setting and input validation omits 60cf45a Support injection of metadata into test runs omits 70dcb35 Replace output_param with output_{value,slot} omits 2e8f956 Apply NAME=VALUE CLI overrides before Benchmark.sh omits 1a3da32 Move output_param to top of Benchmark.sh omits bd469b8 s/bench-session-debian/host-session/ omits 0a3303f Tighten CoremarkPro gather script, add many tests omits 7df5dad Template for use in trusted LAVA uinstance omits 714232f Add missing tcpdump dependency omits f6eaed0 Permit host machine to be of any type omits 58d9c03 Close all tmpdir files before deletion omits c0972b5 Benchmark.sh: Support mustang host/target omits ccbb3a3 Benchmark.sh: Support deploy_linaro_kernel omits b9a84c8 Preserve stdout/stderr separation in cmd output omits 6ae1b98 Output 'Running ${cmd}' to the log omits 21dcf20 tee stderr to log, when there is a log. omits 8c9803f Always set safe_output when in a trusted LAVA omits 254f27c controlledrun.sh logs to /dev/console if running in LAVA omits a8ba392 controlledrun.sh output log set by board conf omits 7935ece Increase default timeout from 0.5 to 1.5 hours. omits 10a6d5e Make TIMEOUT overrideable omits fd15217 Remove redundant boot_timeout parameter omits 5cabcee Update CLI-related messages omits 8c7b632 Remove redundant 'tag' option omits dd73e3a Some more detail around data collection message omits bf530ad Make ssh messages depend on instance omits 64bbb11 Defer to user-specified TARGET_SESSION omits 1071f91 Rationalize *-session YAML omits 4dd43f1 Do not infer device validation from triple omits 237c840 Decouple triple from toolchain omits 1355fa0 host-session: shift CLI args when parsing omits d7c4e10 host-session invokes benchmark scripts with -x omits 4b43ccb Remove redundant ABE_BRANCH parameter omits 8ea5146 host-session: Consistent use of {} with args omits 733f894 gather scripts: Provide mime type for attachments omits 73804fc benchmark.sh preserves -x for runbenchmark.sh omits 0925d63 gather: set -x in all gather scripts omits cade9d8 Return gather script exit code within post-run cmd omits e2da049 Wait for post-run command to complete omits fa2edb7 Convert post-run cmd to function omits 2567eda Convert post-run command into multiline string omits d5fbb5c Add comments about post-run/gather.sh omits f64d29b target-session: Tidy up kill/wait code omits 598dbbf Simplify post-run-cmd-not-set NOP omits e97e0c1 Check post-run command exit code omits 202a1fb runbenchmark.sh: Fix indentation omits 73e6e18 Match tcpdump output even if user gave username omits 524d996 Match tcpdump output even if user gave hostnames omits a6460a3 Do not report names in tcpdump omits bbc24f4 benchmark.sh: Rename GCC to something more generic omits f9ba865 Clean up toolchain usage omits d49b1cf Ensure that unset params get value None omits 51f8968 CoremarkPro.conf: Fix native paths omits 54294e4 CoremarkPro.sh: Report sizes first omits ba76222 CoremarkPro.sh: Require exactly one log omits 1a29fef CoremarkPro.sh: Remove commented-out code omits 400ed9f CoremarkPro.sh: Pass name to report_measured omits 15471cd CoremarkPro.sh: Make function-local var local omits dc11405 CoremarkPro.sh: Count everything that can be multiple omits cace859 CoremarkPro.sh: Cosmetic changes to test IDs omits 0983a16 CoremarkPro.sh: Eliminate spaces in TEST_CASE_ID omits 3b9a139 CoremarkPro.sh: [] notation for iterations only omits c5c4cca CoremarkPro.sh: Name size measurements sanely omits 5aff508 CoremarkPro.sh: Whitespace: Old comment,blank line omits fb29817 test_CoremarkPro.sh: Don't output errors when testing fail cases omits dc788c1 CoremarkPro.sh: Attach raw data before processing omits cc45976 CoremarkPro.sh: Unique names for time and rate omits da34e4d Gather script for Coremark Pro omits a0e931d dispatch-benchmark.py: Support Coremark Pro omits 8aa4179 CoremarkPro support omits d859677 Add some (commented-out) debugging output omits 2d5cb47 Always read configs with target context omits f0b21cc Apply target context when looking for binaries omits 18d9438 Support external sysroot omits 3326e0e dispatch-benchmark.py: More detail on keyring fail omits 686e747 dispatch-benchmark.py: Option to set bundle stream omits aac012d dispatch-benchmark.py: Validate input omits c7e8f86 dispatch-benchmark.py: Message on dry-run exit omits e0c65e3 dispatch-benchmark.py: Move dryrun exit to more natural location omits c69cfac dispatch-benchmark.py: Script to dispatch benchmarks omits 858b801 template.yaml: Metadata must be a string omits 631f66b Remove old jobdefs omits a10bf01 Benchmark.sh,template.yaml: s/BUNDLE_STREAM_NAME/BUNDLE_STREAM/ omits 72bd804 Benchmark.sh: Make bundle stream default paranoid omits b570880 Benchmark.sh: Handle enquoting in parameters omits 637d2df Benchmark.sh: Set BUNDLE_STREAM_NAME only if unset omits 247523b Benchmark.sh: Set the right default branch omits 2a92992 Benchmark.sh: Remove redundant comments omits 3b6a571 Benchmark.sh,template.yaml: Map from target type to image omits 63475a5 Benchmark.sh,template.yaml: Accept public key omits 81b344d Benchmark.sh,template.yaml: s/ABE_/TESTDEF_/ omits 35304cd Benchmark.sh: Make LAVA_USER available to params omits 92452c0 Benchmark.sh: Get bundle server from external LAVA_SERVER def omits 439ea60 Benchmark.sh: Braces-and-spaces omits 31ba4d2 Benchmark.sh: Allow optional parms to be undefined omits 4246b1d Import template.yaml from lava-jobs omits 45b7b6a Add script to derive all job parameters omits c54aeb1 Remove redundant TODO omits ec39941 WIP: 'Semi-trusted' concept, for testing purposes omits 2aa9d8a Test case where some subbenchmarks fail omits f6ec775 Use spec.cpu200x.invalid rather than ...valid omits 047be3e Multiply ratios for CPU2000 omits bcdad70 Make locals in generate_subbenchmark actually local omits aeed551 Print the leading 0 if integer part is 0 omits 80f7cd6 Test more realistic values for input sizes omits 48be5f3 Correct naming style of test input omits 48f939d Add tests for cases where the script should fail omits 549471a Update error message omits b39d49f Test non-ref input sizes omits 9753b68 Fix off-by-1 in iteration counts omits 1e8a4ca Test random mixes of subbenchmarks omits 399b3f1 Add tests for individual benchmark runs omits dcf59b2 Separate benchset definitions from individual runs omits 01c23fe Add exit, error handlers omits 8be0d1c Add a set -o pipefail omits 6bbfb61 Add (commented-out) debugging printf omits 199c6b6 Test cases where only int or fp was run omits bdda15a Move boilerplate to generate_benchmark and rename omits fc8691f Handle different spellings of benchmark names omits d1c8566 raw is global, stop passing it around everywhere. omits d1750e6 Remove unused variables omits 3a3e9b8 Lead regexp with ^ omits 9e7c2c0 CPU200x*: Support CPU2000 results. omits 94dc098 test_CPU200x: Paramaterise bits that vary for 2k omits 983f3ad Rename *CPU2006.sh to *CPU200x.sh omits bb7e88a test_CPU2006.sh: Give doit a sensible name omits b204cdd Rework CPU2006.sh to use 'raw' output omits bfb5cff CPU2006.sh: Attach raw output to run omits ba30d67 fakebench.sh: Attach to run, not a test case omits 38363e1 Gather script + tests for CPU2006 omits d9965fd gather/fakebench.sh: All output to console omits 787ec14 WIP: Experiment in collecting metadata new a630e1e gather/fakebench.sh: All output to console new 0e149e9 Gather script + tests for CPU2006 new 508b69b fakebench.sh: Attach to run, not a test case new bc98d46 CPU2006.sh: Attach raw output to run new 1399e74 Rework CPU2006.sh to use 'raw' output new 69ef488 test_CPU2006.sh: Give doit a sensible name new cb8610b Rename *CPU2006.sh to *CPU200x.sh new aa5096f test_CPU200x: Paramaterise bits that vary for 2k new 708a90e CPU200x*: Support CPU2000 results. new ae6a127 Lead regexp with ^ new d6c911b Remove unused variables new 6e3dfe6 raw is global, stop passing it around everywhere. new fffbc69 Handle different spellings of benchmark names new 02be24e Move boilerplate to generate_benchmark and rename new d4d3c45 Test cases where only int or fp was run new 64d90ff Add (commented-out) debugging printf new 222796a Add a set -o pipefail new 92f02ad Add exit, error handlers new 5764caf Separate benchset definitions from individual runs new 098bccc Add tests for individual benchmark runs new 0096b3d Test random mixes of subbenchmarks new 1917007 Fix off-by-1 in iteration counts new e966d7c Test non-ref input sizes new aa48cd9 Update error message new 8dd3514 Add tests for cases where the script should fail new d0ae3f2 Correct naming style of test input new 711a076 Test more realistic values for input sizes new fa4a4f8 Print the leading 0 if integer part is 0 new e90eb06 Make locals in generate_subbenchmark actually local new 9f37afc Multiply ratios for CPU2000 new 866ffdd Use spec.cpu200x.invalid rather than ...valid new f09e56a Test case where some subbenchmarks fail new 38826e2 'Semi-trusted' concept, for testing purposes new 0521d9e Remove redundant TODO new 1753346 Add script to derive all job parameters new c6d7bae Import template.yaml from lava-jobs new a803692 Benchmark.sh: Allow optional parms to be undefined new d019643 Benchmark.sh: Braces-and-spaces new 2b45d79 Benchmark.sh: Get bundle server from external LAVA_SERVER def new 8ee6618 Benchmark.sh: Make LAVA_USER available to params new 55ca654 Benchmark.sh,template.yaml: s/ABE_/TESTDEF_/ new a3227e4 Benchmark.sh,template.yaml: Accept public key new 1212d25 Benchmark.sh,template.yaml: Map from target type to image new dd218c6 Benchmark.sh: Remove redundant comments new d5e5ddf Benchmark.sh: Set the right default branch new c30abe9 Benchmark.sh: Set BUNDLE_STREAM_NAME only if unset new a28dc75 Benchmark.sh: Handle enquoting in parameters new eeb06d8 Benchmark.sh: Make bundle stream default paranoid new 439b40a Benchmark.sh,template.yaml: s/BUNDLE_STREAM_NAME/BUNDLE_STREAM/ new 731aee7 Remove old jobdefs new ebf80ac template.yaml: Metadata must be a string new 927250b dispatch-benchmark.py: Script to dispatch benchmarks new 71ac192 dispatch-benchmark.py: Move dryrun exit to more natural location new adabef4 dispatch-benchmark.py: Message on dry-run exit new fbb7343 dispatch-benchmark.py: Validate input new 8c5e81e dispatch-benchmark.py: Option to set bundle stream new 751555c dispatch-benchmark.py: More detail on keyring fail new b33631d Support external sysroot new afcadd5 Apply target context when looking for binaries new b26847e Always read configs with target context new a745b27 Add some (commented-out) debugging output new e4f867d CoremarkPro support new 2755ec9 dispatch-benchmark.py: Support Coremark Pro new d10f47a Gather script for Coremark Pro new 0c69511 CoremarkPro.sh: Unique names for time and rate new c0c13a0 CoremarkPro.sh: Attach raw data before processing new 4d1f345 test_CoremarkPro.sh: Don't output errors when testing fail cases new c986230 CoremarkPro.sh: Whitespace: Old comment,blank line new e68d8b1 CoremarkPro.sh: Name size measurements sanely new 9116bdc CoremarkPro.sh: [] notation for iterations only new 2500638 CoremarkPro.sh: Eliminate spaces in TEST_CASE_ID new 27664f1 CoremarkPro.sh: Cosmetic changes to test IDs new e9edf0e CoremarkPro.sh: Count everything that can be multiple new 58950cb CoremarkPro.sh: Make function-local var local new 370e2a5 CoremarkPro.sh: Pass name to report_measured new 85a75e0 CoremarkPro.sh: Remove commented-out code new 4de6b50 CoremarkPro.sh: Require exactly one log new 74d5121 CoremarkPro.sh: Report sizes first new 5ee32c8 CoremarkPro.conf: Fix native paths new 059a517 Ensure that unset params get value None new f98f406 Clean up toolchain usage new 9b4ad5b benchmark.sh: Rename GCC to something more generic new 1ac4b63 Do not report names in tcpdump new 8c94f48 Match tcpdump output even if user gave hostnames new fd1af66 Match tcpdump output even if user gave username new c9ba85f runbenchmark.sh: Fix indentation new b0ceb31 Check post-run command exit code new 2d06d8f Simplify post-run-cmd-not-set NOP new fe4c5a1 target-session: Tidy up kill/wait code new d8d1d46 Add comments about post-run/gather.sh new 9b5081e Convert post-run command into multiline string new 8bddeb4 Convert post-run cmd to function new 419540e Wait for post-run command to complete new 5ce2268 Return gather script exit code within post-run cmd new 50fbd0d gather: set -x in all gather scripts new f5f69e8 benchmark.sh preserves -x for runbenchmark.sh new 14843bb gather scripts: Provide mime type for attachments new 544ae0a host-session: Consistent use of {} with args new 4df9bed Remove redundant ABE_BRANCH parameter new 9d84d6e host-session invokes benchmark scripts with -x new a693134 host-session: shift CLI args when parsing new 3ff63d5 Decouple triple from toolchain new 3f646d8 Do not infer device validation from triple new 652d9de Rationalize *-session YAML new 4a9b70d Defer to user-specified TARGET_SESSION new f30c654 Make ssh messages depend on instance new 2900fbb Some more detail around data collection message new d6db174 Remove redundant 'tag' option new 1f7649e Update CLI-related messages new 2646b63 Remove redundant boot_timeout parameter new cad3b0f Make TIMEOUT overrideable new 50b0954 Increase default timeout from 0.5 to 1.5 hours. new 80b22bb controlledrun.sh output log set by board conf new cf6de56 controlledrun.sh logs to /dev/console if running in LAVA new b554388 Always set safe_output when in a trusted LAVA new 841332c tee stderr to log, when there is a log. new 4b22180 Output 'Running ${cmd}' to the log new 00c3424 Preserve stdout/stderr separation in cmd output new d5566cc Benchmark.sh: Support deploy_linaro_kernel new 07e3b8d Benchmark.sh: Support mustang host/target new b002ace Close all tmpdir files before deletion new 7547d6a Permit host machine to be of any type new 597a910 Add missing tcpdump dependency new 56f064b Template for use in trusted LAVA uinstance new 3cefd21 Tighten CoremarkPro gather script, add many tests new 77d0fbb s/bench-session-debian/host-session/ new b3cd81c Move output_param to top of Benchmark.sh new 06fcb70 Apply NAME=VALUE CLI overrides before Benchmark.sh new 05949b5 Replace output_param with output_{value,slot} new 93c354b Support injection of metadata into test runs new 1fcc296 Centralize default-setting and input validation new 9ac5985 Make JOB_NAME user-settable new ed71b46 Only look up token in keychain for non-dry-runs new 0e084be Confirm that one of TOOLCHAIN and PREBUILT is set new 2454830 Create/support CPU200{0,6}.sh symlinks new 55db400 dispatch-benchmark.py: Force explicit --triple new b0fd2d0 Benchmark.sh: Let user override BUNDLE_SERVER new 3e63f3d Benchmark.sh: Validate/fix up {LAVA,BUNDLE}_SERVER new 2253030 Benchmark.sh: Ensure BUNDLE_STREAM has trailing / new ce30010 CPU200x: Fix unit for geomean of runtime new f489162 Benchmark.sh: Catch 2 triple/toolchain mismatches new 498e5c8 Benchmark.sh: Isomorphic tidying of validate() new 09c13e9 CPU200x.sh: Work with arbitrary name new f245188 Handle possibility that gather script is a symlink new 1d7ef33 host-session: Add missing 'test' new 4224e83 Get Coremark-Pro from new Coremark-Pro repository new 07d4a9a Set 'build' in Coremark-Pro default_makeflags new b1beb64 Benchmark.sh: Set timeout depending on benchmark. new f781052 Benchmark.sh: Add comment to validation func new 4538068 Remove EEMBC.conf new 09a7b69 Benchmark.sh: Only set timeout if user didn't new 5b0e185 Use LAVA_JOB_NAME, not JOB_NAME, in LAVA wrapper new 75c5411 Pass COMPILER_FLAGS through default_makeflags new 2bf6b4a Add a TODO about setting COMPILER_FLAGS on CLI

This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this:

* -- * -- B -- O -- O -- O (bf503f6) \ N -- N -- N refs/heads/bernie/benchmarking-uinstance-stability (2bf6b4a)

You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B.

Any revisions marked "omits" are not gone; other references still refer to them. Any revisions marked "discards" are gone forever.

The 154 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference.

Summary of changes: config/bench/gather/fakebench.sh | 2 -- 1 file changed, 2 deletions(-)