On 16/12/15 17:08, Daniel Thompson wrote:

SysRq-L does not generate a backtrace from all CPUs when I test it on my Inforce IFC6410 platform (Snapdragon 600). The stack dump code, triggered by IPI_CPU_BACKTRACE, never runs on the other CPUs. Eventually we hit the 10 second timeout and a subset of the expected stack dumps on are shown on the console.

It is likely this is because SGI IDs 14 and 15 have been reserved for use by secure world on this platform. For IFC6410 platform the code works as expected when IPI_CPU_BACKTRACE is set to any value in the interval 9..13.

Signed-off-by: Daniel Thompson daniel.thompson@linaro.org

arch/arm/kernel/smp.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c index b26361355dae..78205927fcd4 100644 --- a/arch/arm/kernel/smp.c +++ b/arch/arm/kernel/smp.c @@ -73,7 +73,8 @@ enum ipi_msg_type { IPI_CPU_STOP, IPI_IRQ_WORK, IPI_COMPLETION,

• IPI_CPU_BACKTRACE = 15,

• IPI_CPU_BACKTRACE = 13,
• /* 14 and 15 are reserved; they do not work on some Krait CPUs */

}; static DECLARE_COMPLETION(cpu_running);

It looks to me that we're just moving the goal posts and keep using a SGI that is likely to be reserved by the secure side. Quoting the GIC spec:

In any system that implements the ARM Security Extensions, to support a consistent model for message passing between processors, ARM strongly recommends that all processors reserve: * ID0-ID7 for Non-secure interrupts * ID8-ID15 for Secure interrupts.

Of course, this is only a recommendation, but it is one that is likely to be followed...

Now, we are already using our 8 "non-secure" SGIs, but we can immediately reclaim one (IPI_CALL_FUNC_SINGLE is not that useful anymore). I'll cook a patch for that.

Thanks,

M.

Hi Daniel,

On 16/12/15 17:08, Daniel Thompson wrote:

It is possible for the secure world to reserve certain SGI IDs for itself. Currently we have limited visibility of which IDs are safe to use for IPIs.

Modify the GIC initialization code to actively search for reserved SGI IDs and report if any are found. Warn even more loudly if the reserved SGIs overlap with the normal IPI range.

When run on an Inforce IFC6410 (Snapdragon 600) this code produces the following messages:

CPU0: Detected reserved SGI IDs: 14-15
CPU1: Detected reserved SGI IDs: 15
CPU2: Detected reserved SGI IDs: 15
CPU3: Detected reserved SGI IDs: 15
~~~ cut here ~~~

Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
---
 drivers/irqchip/irq-gic.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index abf2ffaed392..541622da7049 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -490,6 +490,7 @@ static void gic_cpu_init(struct gic_chip_data *gic)
   void __iomem *base = gic_data_cpu_base(gic);
   unsigned int cpu_mask, cpu = smp_processor_id();
   int i;
+	DECLARE_BITMAP(sgi_mask, 16);
 
   /*
    * Setting up the CPU map is only relevant for the primary GIC
@@ -511,6 +512,58 @@ static void gic_cpu_init(struct gic_chip_data *gic)
   	for (i = 0; i < NR_GIC_CPU_IF; i++)
   		if (i != cpu)
   			gic_cpu_map[i] &= ~cpu_mask;
+
+		/*
+		 * Fiddle with the SGI set/clear registers to try identify
+		 * any IPIs that are reserved for secure world.
+		 */
+		bitmap_fill(sgi_mask, 16);
+
+		for (i = 0; i < 16; i++) {
+			void __iomem *set_reg =
+			    dist_base + GIC_DIST_SGI_PENDING_SET + (i & ~3);
+			void __iomem *clear_reg =
+			    dist_base + GIC_DIST_SGI_PENDING_CLEAR + (i & ~3);
+			unsigned long mask = cpu_mask << (8*(i%4));
+			unsigned long flags, pending, after_clear, after_set;

Please make these u32, as unsigned long is 64bit on arm64. Another thing to note is that GICD_CPEND{S,C}SGIRn are byte accessible, so you can save yourself some this hassle shifting things around and just write a single byte. You're already writing 16 times anyway...

Another thing to consider is that these locations are only defined on GICv2 and not GICv1, so this patch is likely to cause trouble on older HW.

• 	local_irq_save(flags);
  

Why do you need to do this? The CPU interface is not enabled yet, so I can't see how you could get an interrupt on this CPU.

• 	/* record original value */
  

• 	pending = readl_relaxed(set_reg);
  

• 	/* clear, test, set, and test again */
  

• 	writel_relaxed(mask, clear_reg);
  

• 	after_clear = readl_relaxed(set_reg);
  

• 	writel_relaxed(mask, set_reg);
  

• 	after_set = readl_relaxed(set_reg);
  

It should be enough to write to the SET register, and read back, as the bit is RAZ/WI when the interrupt is Group-0.

• 	/* restore original value */
  

• 	writel_relaxed(mask & ~pending, clear_reg);
  

• 	local_irq_restore(flags);
  

• 	if (mask & ~after_clear && mask & after_set)
  

• 		clear_bit(i, sgi_mask);
  

• }
  

• /*
  

•  * Show the SGI mask if it is "interesting". Here interesting
  

•  * means that the set/clear register is implemented
  

•  * (mask is not full) and it tells us that the secure world
  

•  * has reserved some SGIs (mask is not empty).
  

•  */
  

• if (!bitmap_full(sgi_mask, 16) && !bitmap_empty(sgi_mask, 16))
  

• 	pr_info("CPU%d: Detected reserved SGI IDs: %*pbl\n",
  

• 		cpu, 16, sgi_mask);
  

• /*
  

•  * Yell if the reserved IDs make the system unviable.
  

•  */
  

• if (!bitmap_full(sgi_mask, 16) &&
  

•     find_first_bit(sgi_mask, 16) < NR_IPI)
  

• 	pr_crit("CPU%d: Not enough SGI IDs; expect failure\n",
  

• 		cpu);
  

  }

gic_cpu_config(dist_base, NULL);

Thanks,

M.