Hi Volodymyr,
On Mon, May 8, 2017 at 2:22 PM, Volodymyr Babchuk vlad.babchuk@gmail.com wrote:
Hmm, can you share your UART driver somewhere? I just want to take a look. Also, can JLink perform a table walk for a specified VA? If I remember well, it allows you to access to CP15...
The UART issue was related to defining the UART memory space as MEM_AREA_IO_NSEC. If I change it to MEM_AREA_IO_SEC, then the messages are printed ok!
The UART1 range needs to be 0xF8020000 to 0xF8024000; however the MMU translation table sets up 1MB (0xf8000000 to 0xf80fffff).
Looking at SAMA5 datasheet the peripheral SECURAM falls under this range and the peripheral security type for it is "always secure". Could this be the reason why UART needs to be defined as MEM_AREA_IO_SEC? Does op-tee support 4k/16k mapping instead of 1MB to workaround this issue?
DEBUG: [0x400] TEE-CORE:add_phys_mem:324: CFG_SHMEM_START type NSEC_SHM 0x3e000000 size 0x00400000 DEBUG: [0x400] TEE-CORE:add_phys_mem:324: CFG_TA_RAM_START type TA_RAM 0x3e500000 size 0x01b00000 DEBUG: [0x400] TEE-CORE:add_phys_mem:324: CFG_TEE_RAM_START type TEE_RAM 0x3e400000 size 0x00100000 DEBUG: [0x400] TEE-CORE:add_phys_mem:324: CONSOLE_UART_BASE type IO_SEC 0xf8000000 size 0x00100000 DEBUG: [0x400] TEE-CORE:add_va_space:363: type RES_VASPACE size 0x00a00000 DEBUG: [0x400] TEE-CORE:dump_mmap_table:476: type TEE_RAM va 0x3e400000..0x3e4fffff pa 0x3e400000..0x3e4fffff size 0x00100000 (pgdir) DEBUG: [0x400] TEE-CORE:dump_mmap_table:476: type TA_RAM va 0x3e500000..0x3fffffff pa 0x3e500000..0x3fffffff size 0x01b00000 (pgdir) DEBUG: [0x400] TEE-CORE:dump_mmap_table:476: type NSEC_SHM va 0x40000000..0x403fffff pa 0x3e000000..0x3e3fffff size 0x00400000 (pgdir) DEBUG: [0x400] TEE-CORE:dump_mmap_table:476: type IO_SEC va 0x40400000..0x404fffff pa 0xf8000000..0xf80fffff size 0x00100000 (pgdir) DEBUG: [0x400] TEE-CORE:dump_mmap_table:476: type RES_VASPACE va 0x40500000..0x40efffff pa 0x00000000..0x009fffff size 0x00a00000 (pgdir) DEBUG: [0x400] TEE-CORE: section map [3e400000 3e500000] MEM-RW-X-S DEBUG: [0x400] TEE-CORE: section map [3e500000 40000000] MEM-RW-XN-S DEBUG: [0x400] TEE-CORE: section map [40000000 40400000] MEM-RW-XN-NS DEBUG: [0x400] TEE-CORE: section map [40400000 40500000] DEV-RW-XN-S DEBUG: [0x400] TEE-CORE: section map [40500000 40f00000] not-mapped MESSAGE: [0x400] TEE-CORE:core_init_mmu_regs:829: TTBR0 = 0x3e458059 MESSAGE: [0x400] TEE-CORE:core_init_mmu_regs:830: TTBR1 = 0x3e458059 INFO: TEE-CORE: DEBUG: [0x0] TEE-CORE:init_canaries:167: #Stack canaries for stack_tmp[0] with top at 0x3e45e7f8 DEBUG: [0x0] TEE-CORE:init_canaries:167: watch *0x3e45e7fc DEBUG: [0x0] TEE-CORE:init_canaries:168: #Stack canaries for stack_abt[0] with top at 0x3e45f038 DEBUG: [0x0] TEE-CORE:init_canaries:168: watch *0x3e45f03c DEBUG: [0x0] TEE-CORE:init_canaries:170: #Stack canaries for stack_thread[0] with top at 0x3e461078 DEBUG: [0x0] TEE-CORE:init_canaries:170: watch *0x3e46107c DEBUG: [0x0] TEE-CORE:init_canaries:170: #Stack canaries for stack_thread[1] with top at 0x3e4630b8 DEBUG: [0x0] TEE-CORE:init_canaries:170: watch *0x3e4630bc INFO: TEE-CORE: OP-TEE version: 2.4.0-68-g8f386ec-dev #1 Mon May 8 19:43:54 UTC 2017 arm INFO: TEE-CORE: Initialized DEBUG: [0x0] TEE-CORE:init_primary_helper:612: Primary CPU switching to normal world boot
Issue2: Hang after jumping to Linux kernel? If I remove the above while(1) and let the board boot, it appears to be stuck in the address range (physical) of the Linux kernel. I am not sure if it needs to be using virtual addresses here... (gdb) mon halt target state: halted target halted in ARM state due to debug-request, current mode: Supervisor cpsr: 0x40000093 pc: 0x22003314
Have you checked what it code is at given pc?
I ended up disabling serial console in op-tee to prevent mapping 0xf8000000 to 0xf80fffff (1MB) as "secure" (to allow linux to access them). Even after doing that I still end up seeing the above issue (PC stuck at 0x22003314).
Disassembling shows code stuck checking TXEMPTY bit of UART_SR (presumably Linux earlyprintk) but the UART registers always returns incorrect data "00000002" (think its related to permission): (gdb) mon arm disassemble 0x22003310 0x22003310 0xe5913014 LDR r3, [r1, #0x14] 0x22003314 0xe3130c02 TST r3, #0x200 0x22003318 0x0afffffc BEQ 0x22003310 0x2200331c 0xe1a0f00e MOV r15, r14 0x22003320 0xf8020000 UNDEFINED INSTRUCTION 0x22003324 0xf7020000 UNDEFINED INSTRUCTION 0x22003328 0xe92d4030 STMDB r13!, {r4, r5, r14} 0x2200332c 0xe1a0e1a2 MOV r14, r2, LSR #0x3 0x22003330 0xe1a0c000 MOV r12, r0 0x22003334 0xe1a03001 MOV r3, r1
(5208) r0 (/32): 0x00000055 (dirty) (5209) r1 (/32): 0xF8020000 (5210) r2 (/32): 0xF7020000 (5211) r3 (/32): 0x00000002 (5212) r4 (/32): 0x00000055 (5213) r5 (/32): 0x220042FD (5214) r6 (/32): 0x00000024 (5215) r7 (/32): 0x00000000 (5216) r8 (/32): 0x223B62F0 (5217) r9 (/32): 0x223A62F0 (5218) r10 (/32): 0x20008000 (5219) r11 (/32): 0x223A52CC (5220) r12 (/32): 0x00000020 (5223) pc (/32): 0x22003314 (5233) sp_svc (/32): 0x223A62C0 (5234) lr_svc (/32): 0x22000960 (5239) cpsr (/32): 0x40000093
//Trying to view UART registers: (gdb) mon mdw 0xF8020000 8 0xf8020000: 00000002 00000002 00000002 00000002 00000002 00000002 00000002 00000002
Looking at the TTBR0/1 registers: (gdb) mon arm mrc 15 0 2 0 0 536887296 (0x20004000) (gdb) mon arm mrc 15 0 2 0 1 0
Looking at the translation table entries, looks like the UART memory region is being mapped as "secure": UART MSB bits (0xF80), entry offset in table (0xF80 * 4 = 0x3E00) Translation table entry is at: 0x20004000 + 0x3E00 = 0x20007e00 (gdb) mon mdw 0x20007e00 0x20007e00: f8000c12 NS bit is set to zero?
Questions: I noticed the TTBR0/1 register contents are different between OP-TEE and Linux. Is this expected? Which code is setting the TTBR0 register to 0x20004000 and its entries as 0xXXX00C12? Is this done by Linux, if so does it clear the NS bit by default? Are there any platform specific support that needs to be added for the hand-off between op-tee and linux to happen correctly?
Any guidance on this is appreciated, since I am unclear how the handoff between op-tee and Linux happens.
Thanks, Akshay