On Tue, Feb 14, 2023 at 09:18:24PM +0000, Jonathan Cameron wrote:
On Tue, 14 Feb 2023 14:01:23 -0500 Gregory Price gregory.price@memverge.com wrote:
On Tue, Feb 14, 2023 at 10:39:42AM -0800, Dan Williams wrote:
Gregory Price wrote:
On Sun, Feb 05, 2023 at 05:02:29PM -0800, Dan Williams wrote:
Summary:
CXL RAM support allows for the dynamic provisioning of new CXL RAM regions, and more routinely, assembling a region from an existing configuration established by platform-firmware. The latter is motivated by CXL memory RAS (Reliability, Availability and Serviceability) support, that requires associating device events with System Physical Address ranges and vice versa.
Ok, I simplified down my tests and reverted a bunch of stuff, figured i should report this before I dive further in.
Earlier i was carrying the DOE patches and others, I've dropped most of that to make sure i could replicate on the base kernel and qemu images
QEMU branch: https://gitlab.com/jic23/qemu/-/tree/cxl-2023-01-26 this is a little out of date at this point i think? but it shouldn't matter, the results are the same regardless of what else i pull in.
Kernel branch: https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/log/?h=for-6.3/c...
Note that I acted on this feedback from Greg to break out a fix and merge it for v6.2-final
http://lore.kernel.org/r/Y+CSOeHVLKudN0A6@kroah.com
...i.e. you are missing at least the passthrough decoder fix, but that would show up as a region creation failure not a QEMU crash.
So I would move to testing cxl/next.
I just noticed this, already spinning a new kernel. Will report back
Not ruling out the driver yet, but Fan's tests with hardware has me leaning more towards QEMU.
Same, not much has changed and I haven't tested with hardware yet. Was planning to install it on our local boxes sometime later this week.
Was just so close to setting up a virtual memory pool in the lab, was getting antsy :]
Could you test it with TCG (just drop --enable-kvm)? We have a known limitation with x86 instructions running out of CXL emulated memory (side effect of emulating the interleave). You'll need a fix even on TCG for the corner case of an instruction bridging from normal ram to cxl memory. https://lore.kernel.org/qemu-devel/20230206193809.1153124-1-richard.henderso...
Performance will be bad, but so far this is only way we can do it correctly.
Jonathan
Siiiggghh... i had this patch and dropped --enable-kvm, but forgot to drop "accel=kvm" from the -machine line
This was the issue.
And let me tell you, if you numactl --membind=1 python, it is IMPRESSIVELY slow. I wonder if it's even hitting a few 100k instructions a second.
This appears to be the issue. When I get a bit more time, try to dive into the deep dark depths of qemu memory regions to see how difficult a non-mmio fork might be, unless someone else is already looking at it.
~Gregory