On Wed, 21 May 2025 10:05:13 +0300 Tariq Toukan wrote:
On 21/05/2025 1:59, Jakub Kicinski wrote:
On Tue, 20 May 2025 21:38:03 +0300 Tariq Toukan wrote:
Test verifies that netdevsim correctly implements devlink ops callbacks that set tc-bw on leaf or node rate object.
Please add a test that can actually validate a NIC HW. The test probably needs to be in Python to use a remote endpoint, and should live under tools/testing/../drivers/net/hw
We had a long conversation about what we expect from the API vs how your HW works. One of the test cases should confirm the expected behavior, IOW fail on mlx5. Which is fine, unlikely that any NIC will have 100% compliance. But at least we will be documenting the expectations.
No problem with that, we'll add.
We could've saved this extra cycle if my questions [1] exactly about this topic weren't ignored. Area is vague and not well defined. We can continue with the iterative guess and fix cycles, or alternatively get it clearly and formally defined.
I started a couple of times on answering but my hands go a little limb when I have to explain things so obvious like "testing is a crucial part of software development" :S I mean.. nvidia certainly tests their code, so I'm not sure where the disconnect is. I had a short conversation with Gal at some conference where he, AFAIU, was doubting that device testing can be part of an open source project.
It certainly is not advantageous to companies to have to share their test code. So when you ask me for details on the rules what I hear is "how can we make sure we do as little as possible".
Broadly, any new uAPI should come with tests which exercise the functionality. We started a decade or so ago with netdevsim tests which just validate the API layer itself. That did not provide sufficient validation of the real implementations, crucially it did not check whether shallow APIs (devlink) actually behave the same when implemented by multiple vendors. So two years ago we built the Python harness to be able to write tests for NIC functionality. That is the level of testing we expect now. Obviously there will always be exceptions. For instance I was pushing for common tests for the time sync code (DPLL etc.) but I was convinced by the experts that it's hard and that they generally test with $x0,000 measurement equipment. So fair, I guess that's too hard. But for BW shaping tests? IIRC mlxsw has qdisc offload tests for BW shaping upstream.