Tree: https://github.com/rth7680/qemu.git tgt-arm-vhe-5
Testcase: qemu-test:~rth/linux/initramfs-min.cpio.gz
The host kernel could be anything, but I've been using the same Image.gz that is inside the cpio archive.
./aarch64-softmmu/qemu-system-aarch64 -m 4G \ -M virt,virtualization=on,gic-version=max -cpu max \ -kernel Image.gz -initrd initramfs-min.cpio.gz
At the shell prompt, ./test will run a guest kernel with kvm.
As momentarily discussed with PMM in the hallway:
As soon as the guest kernel enables interrupts,
arch_timer_starting_cpu enable_percpu_irq irq_percpu_enable gic_unmask_irq -- Incorrect exception delivery.
the GTIMER_PHYS interrupt is delivered to EL2 (seems to be ok), the host kernel does something (haven't dug into what exactly, bug presumably setting bits that are supposed to pass the virq to the guest), and immediately another interrupt is delivered to EL2. Repeat.
Whether this is incorrect routing of the virq interrupt, or incorrect masking/acking of the hard irq interrupt at EL2, I do not yet know.
PMM: I don't know the answer to either (a) or (b) as asked on hangouts. I think (b) is correct, but I can't be sure. I'm trying to understand how (a) is supposed to work now. In particular, I can't find any code that sets HCR_EL2.{VI,VF}, only tests them.
r~
On Thu, 31 Oct 2019 at 13:15, Richard Henderson richard.henderson@linaro.org wrote:
As soon as the guest kernel enables interrupts,
arch_timer_starting_cpu enable_percpu_irq irq_percpu_enable gic_unmask_irq -- Incorrect exception delivery.
the GTIMER_PHYS interrupt is delivered to EL2 (seems to be ok), the host kernel does something (haven't dug into what exactly, bug presumably setting bits that are supposed to pass the virq to the guest), and immediately another interrupt is delivered to EL2. Repeat.
Whether this is incorrect routing of the virq interrupt, or incorrect masking/acking of the hard irq interrupt at EL2, I do not yet know.
PMM: I don't know the answer to either (a) or (b) as asked on hangouts. I think (b) is correct, but I can't be sure. I'm trying to understand how (a) is supposed to work now. In particular, I can't find any code that sets HCR_EL2.{VI,VF}, only tests them.
HCR_EL2.{VI,VF} aren't set by the CPU, they're set by software (ie the hypervisor running in QEMU). They're for the situation where the hypervisor wants to cause a VIRQ or VFIQ to occur directly (ie not because the hypervisor has programmed the GIC and the GIC is injecting a VIRQ/VFIQ). I have a feeling Linux doesn't use them and always uses the GIC.
What I had in mind for 'a' was the implementation of the bit of the spec that says "When executing at EL2 or Non-secure EL0, any physical interrupt that is configured to be taken at EL2 is subject to the Process state interrupt mask. If the mask bit is set, then the corresponding interrupt will not be taken. If the mask bit is not set, then the corresponding interrupt will be taken." (ie handling of PSTATE.[AIF] when HCR_EL2.{E2H, TGE} == {1, 1}.)
thanks -- PMM
On 10/31/19 2:30 PM, Peter Maydell wrote:
HCR_EL2.{VI,VF} aren't set by the CPU, they're set by software (ie the hypervisor running in QEMU). They're for the situation where the hypervisor wants to cause a VIRQ or VFIQ to occur directly (ie not because the hypervisor has programmed the GIC and the GIC is injecting a VIRQ/VFIQ). I have a feeling Linux doesn't use them and always uses the GIC.
This seems to be the case.
What I had in mind for 'a' was the implementation of the bit of the spec that says "When executing at EL2 or Non-secure EL0, any physical interrupt that is configured to be taken at EL2 is subject to the Process state interrupt mask. If the mask bit is set, then the corresponding interrupt will not be taken. If the mask bit is not set, then the corresponding interrupt will be taken." (ie handling of PSTATE.[AIF] when HCR_EL2.{E2H, TGE} == {1, 1}.)
As far as I can see this is all correct.
In particular, the manipulations to HCR_{AMO,FMO,IMO} that are done by arm_hcr_el2_eff based on HCR_{E2H,TGE} means that arm_excp_unmasked and arm_phys_excp_target_el are correct as-is.
The case I'm trying to debug, guest EL1 timer, TGE == 0 && IMO == 1. Which, according to D1-10 routes to EL2, and according to D1-13 is not masked by PSTATE.
I really don't understand how this is supposed to work.
The only thing I can imagine is that the guest EL1 timer is not really supposed to generate a real interrupt, but to silently generate a virq, but I don't see anything in section D11 (Generic Timer in AArch64 State) that validates that hypothesis.
r~
On Fri, 1 Nov 2019 at 13:52, Richard Henderson richard.henderson@linaro.org wrote:
The case I'm trying to debug, guest EL1 timer, TGE == 0 && IMO == 1. Which, according to D1-10 routes to EL2, and according to D1-13 is not masked by PSTATE.
I really don't understand how this is supposed to work.
The only thing I can imagine is that the guest EL1 timer is not really supposed to generate a real interrupt, but to silently generate a virq, but I don't see anything in section D11 (Generic Timer in AArch64 State) that validates that hypothesis.
OK, how about option B -- have we correctly implemented the new-for-VHE timer, are we mapping the various guest accesses to timer registers to the right underlying QEMU timers ?
thanks -- PMM
linaro-toolchain@lists.linaro.org
-
Peter Maydell -
Richard Henderson