From: Kees Cook keescook@chromium.org Date: Wed, 4 Mar 2020 10:17:41 -0800
On Wed, Mar 04, 2020 at 05:52:04PM +0900, Kuniyuki Iwashima wrote:
Currently tests are often written in C and shell script. In many cases, the script passes some arguments to the C program. However, the helper functions do not support arguments, so many tests are written without helper functions.
This patch allows us to handle argc and argv in each tests and makes it easier to write tests flexibly with helper functions.
Signed-off-by: Kuniyuki Iwashima kuniyu@amazon.co.jp
Interesting! Do you have an example that uses this? I wonder if it might make sense instead to allow extending the struct __test_metadata with test-specific options so that individual tests don't have to re-parse argv every time (the main test running could instead do it once and set variables in struct __test_metadata (or somewhere else).
I added a sample test program at the end of this mail.
There are some functions that are not TEST() but are passed __test_metadata to in order to use ASSERT_EQ in the function. I did not extend __test_metadata because I thought argc and argv would not be used in such functions.
e.g. kill_thread_or_group() in tools/testing/selftests/seccomp/seccomp_bpf.c
But, I have not thought about re-parsing, thank you! Also I thought up that it is better to pass argc and argv to FIXTURE_SETUP/TEARDOWN.
Now I have two idea.
1. pass argc and argv to FIXTURE_SETUP/TEARDOWN. 2. define COMMON_FIXTURE and COMMON_FIXTURE_SETUP/TEARDOWN, and pass COMMON_FIXTURE to all tests. (I think it is not good to extend __test_metadata because argc and argv is not metadata, so it is good to setup another vars with args)
I think each has pros and cons.
1. Pros - shell script only has to call a C program once with some arguments and each FIXTURE_SETUP differs from one another - shell script can call the same C program with different arguments and each FIXTURE_SETUP differs from one another Cons: - if TEST()s use the same FIXTURE, the same FIXTURE_SETUP is called in each TEST()s.
2. Pros: - we do not have to re-parse argc and argv in each TEST()s. Cons: - 1. may give more flexibility than 2.
Which would you think is better? I would be happy if you tell me another idea!
Thanks.
===sample=== #include "./kselftest_harness.h"
TEST(argc_test) { int i; for (i = 0; i < argc; i++) TH_LOG("argv[%d]: %s", i, argv[i]); }
FIXTURE(argc_f) { int data; };
FIXTURE_SETUP(argc_f) { self->data = 92; ASSERT_EQ(92, self->data); }
FIXTURE_TEARDOWN(argc_f) { }
TEST_F(argc_f, argc_test_f) { int i; for (i = 0; i < argc; i++) TH_LOG("fixture: %d\targv[%d]: %s", self->data, i, argv[i]); }
TEST_HARNESS_MAIN ============