On 11/04/2025 8:11 pm, Robin Murphy wrote: [...]
OK, I found it, but I'm still not sure what exactly to make of it - it's the pci_request_acs() in of_iommu_configure(), now being called early enough to actually have an effect. Booting with EDK2 already using PCI prior to Linux, here's what I get for `sudo lspci -vv | grep ACSctl` with 6.15-rc1:
ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-
whereas with the 6.14 behaviour they are all '-'. I don't have a working root filesystem with the U-Boot setup, but if I boot it with "pci=config_acs=000000@pci:0:0" then the kernel does assign the bridge windows and discover the ethernet/SATA endpoints again. I can spend some time getting NFS working next week, but if you're able to get lspci output off a machine in the "broken" state easily that would be handy to compare.
So at this point it would seem to be something about how Linux configures ACS when doing it from scratch. What I don't really know is where to go from there. I do know Juno's possibly a bit odd in that the switch supports ACS, but both the root port and endpoints either side of it don't. Could this be tickling some subtle bug in the PCI layer, and what is EDK2 doing that makes it not happen?
Just following up on where I ran out of ideas. I poked at things a little more, and from a process of elimination, the culprit appears to be is that we enable ACS source validation on the downstream port while its secondary bus is still 0, *then* we get to the "bridge configuration invalid" bit and reconfigure the bus numbers, but after that, config space accesses to the secondary bus still apparently fail to work as expected.
What's now beyond me is whether this is just an ACS quirk of this particular switch, and/or whether there's something we could or should be doing in the PCI layer.
All I can suggest a this point is that you could at least sidestep the problem on the LKFT boards by updating them to a less-ancient version of U-Boot which supports PCIe for Juno (looks like that landed in 2020.10), which should then configure the switch at boot such that the bus numbering doesn't need to change when Linux probes it - that appears to be the only "magic" thing that EDK2 is doing.
Thanks, Robin.