Thanks Stefan! We completely missed DFAR and DFSR are **banked** across secure and non-secure worlds and the assembler path and the ::read are seeing two separate copies of these registers.
We verified that we hit .macro _nonsecure_to_secure in monitor handler with the csu abort exception. We wrote values to registers in the macro, when the exception was DAB_TYPE. _state dump printed those specific values. Though this macro is hit multiple times, the prior hits are through fiq exception, and so these specific values must have been set on the csu abort. So now we are confident that the flow after UART abort is as expected, through the monitor handler.
(1) dfar (secure world) = 53fbc080 physical address, which is in the UART range, and we have made UART secure in csu_config.h (2) dfsr (secure world) = 00001008, which gives b101000 for the status bits, matching "bx01000 precise external abort, nontranslation" with "x=1=AXI Slave error caused the abort" (according to http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0344k/Bgbiagh...) (3) monitor handler gets the abort, so all normal world registers would be appropriately saved. Especially, _state->ip will be lr-8, address holding the normal world instruction that caused the abort.
This is good.
_state->ip holds a normal world virtual address VA_nw, which we convert to physical address PA with va_to_pa function in os/src/server/tz_vmm/include/vm_base.h. We take PA as secure world virtual address VA_sw (based on a linear mapping we see at repos/os/src/server/tz_vmm/include/ram.h. *(VA_sw) gives the normal world instruction at _state->ip. The problem is the address in that instruction doesn't match the address in dfar (secure).
Possible issues: (a) The VA_nw->PA->VA_sw is wrong and we are reading the wrong memory. (seems unlikely) (b) We decode the instruction wrong, thumb vs. arm (we will verify this) (c) normal world cached this address and didn't write it to RAM, so secure world is reading a stale value (you suggested this sometime back).
For (c), we need to flush the normal world cache from secure world. Does genode have any functions for this? I can see some cache/invalidate related code for arm or arm_v7 or cortex_a8 in the following files:
Defining cache invalidate instructions: base-hw/src/core/include/spec/cortex_a8/cpu.h, base-hw/src/core/spec/cortex_a8/cpu.cc, base-hw/src/core/include/spec/arm_v7/cpu_support.h, base-hw/src/core/spec/arm_v7/cpu.cc, base-hw/src/core/include/spec/arm/cpu_support.h,
Using those cache invalidate instructions: base-hw/src/core/spec/arm/kernel/pd.cc, base-hw/src/core/spec/arm/kernel/thread.cc, base-hw/src/core/spec/arm/kernel/thread_update_pd.cc
Will see what these do and ask more specific questions if needed.
Thanks a lot for your patience and help.
Riju