On Fri, Oct 23, 2026 at 12:59:14PM -0300, Jason Gunthorpe wrote:
On Fri, Oct 09, 2026 at 08:43:01PM +0300, Leon Romanovsky wrote:
On Wed, Oct 07, 2026 at 03:40:45PM -0500, Bjorn Helgaas wrote:
On Wed, Oct 07, 2026 at 09:32:27AM +0300, Leon Romanovsky wrote:
On Tue, Oct 06, 2026 at 03:49:47PM -0500, Bjorn Helgaas wrote:
On Thu, Oct 01, 2026 at 02:55:10PM +0300, Leon Romanovsky wrote:
From: Leon Romanovsky leonro@nvidia.com
pci_bridge_has_acs_redir() treats Request and Completion Redirect as interchangeable. On asymmetric fabrics, a control for only the reverse TLP direction can unnecessarily force P2PDMA through the host bridge.
Does "the reverse TLP direction" refer to Completions?
In general, the P2P code treats TLPs flowing from device A to device B the same as TLPs flowing from device B to device A.
However, in the context of this commit message, yes: completions flow in the opposite direction from the device's perspective.
And I guess asymmetric fabrics must mean fabrics where Request Redirect and Completion Redirect are not set the same way?
Yes, in some of our systems, deviceA to deviceB is going through different route than deviceB to deviceA.
It is a little more subtle, I think practical systems try very hard to avoid that since many host bridges do not like seeing only half of an exchange.
What we end up with is Relaxed Ordering and Strict Ordering take different flows in the fabric.
But I asked about ordering because it sounds like this patch expects to encounter asymmetric fabrics where Request Redirect and Completion Redirect may not be set the same way, and the spec implies that asymmetry may result in ordering violations.
My guess is that the hardware and system architects have ensured this doesn't happen. At least, I haven't received any complaints about this series during internal testing.
Yes, the ACS combination is selected to avoid ordering issues
Does a user have any way to know whether the configuration is safe?
I'm wondering whether users might use "pci=config_acs=" to configure ACS so things appear to work as they prefer but may not provide the ordering guarantees drivers depend on.
If that's a possibility, do we have any notes in dmesg that we are depending on fabric behavior that the OS can't discover and verify?
Bjorn