Hello,

KernelCI is hosting a bi-weekly call on Thursday to discuss improvements to existing upstream tests, the development of new tests to increase kernel testing coverage, and the enablement of these tests in KernelCI. In recent months, we at Collabora have focused on various kernel areas, assessing the tests already available upstream and contributing patches to make them easily runnable in CIs.

Below is a list of the tests we've been working on and their latest status updates, as discussed in the last meeting held on 2024-07-11:

*USB/PCI devices kselftest*

- Upstream test to detect unprobed devices on discoverable buses: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... - Updated KernelCI PRs according to feedback, now waiting for the first test results: https://github.com/kernelci/kernelci-core/pull/2577 and https://github.com/kernelci/kernelci-pipeline/pull/642

*Error log test*

- Proposing new kselftest to report device log errors: https://lore.kernel.org/all/20240423-dev-err-log-selftest-v1-0-690c1741d68b@... - Series got Acked-By from Greg, going to be picked up by Shuah soon - Feedback from Tim Bird: this series follows an unusual model where tests can only fail but never pass, as no test case is generated unless there is an error. It takes an unusual approach to detect regressions and fixes. The autogenerated test case names are not very descriptive.

*Suspend/resume in cpufreq kselftest*

- Enabling suspend/resume test within the cpufreq kselftest in KernelCI - - Sent patch upstream for adding RTC wakeup alarm in the cpufreq kselftest: https://lore.kernel.org/all/2e667d-668ff800-1-22d70300@133606496/ - Received a review from Rafael J. Wysocki, who suggested using the rtcwake utility instead of the sysfs entry

*Boot time test*

- Drafted initial implementation with two scripts, a config fragment and a bootconfig file - One script generates a YAML file containing initial timestamps for relevant boot events, parsed from the trace file (run once) - The other script is the actual test, which takes the generated YAML file and a delta in seconds as arguments. The script then parses the current trace file and checks if any timestamp deviates from the reference timestamps in the YAML file by more than the specified delta. - Tracking only a few functions at the moment (populate_rootfs, unpack_to_rootfs, run_init_process). Next steps: refine bootconfig file to include more tracepoints (potentially initcalls too?). Useful tracepoints should be discussed upstream. - Will present this at LPC 2024 (embedded and IoT MC)

*Support for benchmark data in KTAP*

- Tim Bird is working on adding performance data to KTAP output, which can be used in tests to detect slowdowns - The idea is to keep reference values and criteria separate from the test itself - There is a need to store per-platform files with previous times for comparison - Will need to figure out where these files can be stored so they can be shared and used by different people and systems. Potential options: KCIDB or https://github.com/kernelci/platform-test-parameters - Submitted a proposal for LPC 2024 - Other related topics for discussion at LPC 2024 include: how to avoid device tree overhead in the boot process and boot phases (time-critical vs non-critical)

*TAP conformance in kselftests*

- Focusing on standardizing the way kernel's testing modules report results - Discussion ongoing upstream over patches converting tests to TAP: https://lore.kernel.org/all/fb305513-580a-4bac-a078-fe0170a6ffa2@linuxfounda... and https://lore.kernel.org/all/6d82fa16-ed2e-41f1-a466-c752032b6f68@linuxfounda...

Please reply to this thread if you'd like to join the call or discuss any of the topics further. We look forward to collaborating with the community to improve upstream tests and expand coverage to more areas of interest within the kernel.

Best regards,

Laura Nao