ARM64 exception assembly: five instruction walkthroughs

Platform · Exception entry · Exception return · Thread context switching

This chapter follows five instruction sequences in locore.s and its included macros. The aim is to make each register substitution and stack change readable. The existing entry/return guides explain the complete control-flow map; this chapter concentrates on the instructions that construct a context, enter C, and restore execution. Scheduler stack switches, continuations and initial thread entry are covered in thread context switching.

The central distinction is between the address containing interrupted state and the active stack used by the handler. They often differ. A second distinction is between an architectural exception return address in ELR_EL1 and the ordinary link register x30/lr. An exception does not make those two registers interchangeable.

Before following the instructions

For these paths the CPU supplies the exception return PC and previous processor state through ELR_EL1 and SPSR_EL1; syndrome/address registers supply exception-dependent information. XNU reads them explicitly. The hardware has not populated an arm_context_t, pushed every general register, or called C. Software must preserve registers before using them as scratch. The relevant reads are in SPILL_REGISTERS; the vector selection and stack setup are in locore.s.

The boot path installs the vector address in VBAR_EL1; the non-SPTM and SPTM setup differ. The existing entry guide traces both. A lower-EL AArch64 synchronous exception selects Lel0_synchronous_vector_64, whereas an interrupted EL1 SP0 execution selects the corresponding SP0 vector. The .align 7 slots establish the vector-table spacing; they are not C function calls.

Use these names for addresses in the walkthrough:

These are symbolic addresses, not invented layout constants. genassym.c emits ARM_CONTEXT_SIZE from sizeof(arm_context_t) and SS64_*/NS64_* offsets from offsetof. The C context contains integer and NEON saved-state objects, each with its own header/layout. In particular, SS64_X0 is not an assertion that the first byte of arm_context_t is x0.

The generating declarations are at SS64 offsets and NS64 offsets; the fields are in arm_saved_state64 and arm_neon_saved_state64. No generated assym.s from a particular product build is assumed here.

1. EL0 entry: use SP as a destination address, then replace it with a kernel stack

The short lower-EL vector invokes MAP_KERNEL and BRANCH_TO_KVA_VECTOR before reaching the long handler. With __ARM_KERNEL_PROTECT__, MAP_KERNEL reads TTBR0_EL1 into x18, sets its low ASID bit, writes it back, then writes TCR_EL1_BOOT and synchronizes. The branch macro uses x18 to walk thread → CpuDatap → cpu_exc_vectors and loads the selected handler address. Without that configuration the branch is direct. This is why x18 has a special preservation policy here; it is already scratch before the ordinary spill.

At the long vector, SP1 points to the exception stack and SP_EL0 still contains U. The early save is:

Preserve x0/x1, capture the user SP, and select PCB storage

	stp		x0, x1, [sp, #-16]!					// Save x0 and x1 to the exception stack
#if __ARM_KERNEL_PROTECT__
	mov		x18, #0 						// Zero x18 to avoid leaking data to user SS
#endif
	mrs		x0, TPIDR_EL1						// Load the thread register
	LOAD_USER_PCB	dst=x0, src=x0, tmp=x1		// Load the user context pointer
	mrs		x1, SP_EL0							// Load the user stack pointer
	str		x1, [x0, SS64_SP]					// Store the user stack pointer in the user PCB
	msr		SP_EL0, x0							// Copy the user PCB pointer to SP0
	ldp		x0, x1, [sp], #16					// Restore x0 and x1 from the exception stack
	msr		SPSel, #0							// Switch to SP0
	stp		x0, x1, [sp, SS64_X0]				// Save x0, x1 to the user PCB
	stp		x2, x3, [sp, SS64_X2]				// Save x2, x3 to the user PCB
	mrs		x1, TPIDR_EL1						// Load the thread register

LOAD_USER_PCB expands to an address calculation, pointer load and AUTDA_DIVERSIFIED; the latter conditionally authenticates and handles failure. This pointer check is separate from the later signature over saved control-flow state.

The macro's remaining tail handles optional SME state and finishes with mov x0,sp, so x0 becomes B. The synchronous path then chooses a real handler stack:

Select the kernel stack and branch to the common dispatcher

.macro EL0_64_VECTOR_SWITCH_TO_KERN_STACK
	LOAD_KERN_STACK_TOP	dst=x1, src=x1, tmp=x0
	mov		x0, sp
	mov		sp, x1
.endmacro

el0_synchronous_vector_64_long:
	EL0_64_VECTOR	sync
	EL0_64_VECTOR_SWITCH_TO_KERN_STACK
	adrp	x1, EXT(fleh_synchronous)@page			// Load address for fleh
	add		x1, x1, EXT(fleh_synchronous)@pageoff
	mov		x2, #(FLEH_DISPATCH64_OPTION_SYNC_EXCEPTION)
	b		fleh_dispatch64

LOAD_KERN_STACK_TOP is allowed to clobber x0 as its temporary. Therefore mov x0,sp deliberately recovers B before mov sp,x1 changes active SP0 to K. The adrp/add pair forms the handler address, first its page then its page offset; it does not dereference a function-pointer object. x2 receives the spill-option word. The final b transfers control without overwriting LR, whose interrupted value still has to be saved by the dispatcher.

Mermaid source

flowchart LR
    A["Vector entry<br/>active SP1 = E<br/>SP0 = U"] --> B["Temporary staging<br/>SP1 = E - 16<br/>save x0 and x1"]
    B --> C["B.saved_sp = U<br/>SP0 = B<br/>restore x0/x1; SP1 = E"]
    C --> D["SPSel = 0<br/>active sp = B<br/>store x0–x3 in PCB"]
    D --> F["Recover x0 = B<br/>active sp = K<br/>x1 = handler; x2 = options"]

For a lower-EL IRQ/FIQ, EL0_64_VECTOR_SWITCH_TO_INT_STACK instead loads I, recovers x0 from the old SP, and changes the active stack. The saved context remains B in both cases.

2. EL1 SP0 entry: create a context below the interrupted stack pointer

An interrupted kernel call chain must survive. The ordinary same-EL SP0 macro does not substitute a user PCB. It allocates an arm_context_t below S.

Build the kernel context and optionally move handler execution to the interrupt stack

.macro EL1_SP0_VECTOR
	msr		SPSel, #0							// Switch to SP0
	sub		sp, sp, ARM_CONTEXT_SIZE			// Create exception frame
	stp		x0, x1, [sp, SS64_X0]				// Save x0, x1 to exception frame
	stp		x2, x3, [sp, SS64_X2]				// Save x2, x3 to exception frame
	add		x0, sp, ARM_CONTEXT_SIZE			// Calculate the original stack pointer
	str		x0, [sp, SS64_SP]					// Save stack pointer to exception frame
	INIT_SAVED_STATE_FLAVORS sp, w0, w1
	mov		x0, sp								// Copy saved state pointer to x0
.endmacro

.macro EL1_SP0_VECTOR_SWITCH_TO_INT_STACK
	// SWITCH_TO_INT_STACK requires a clobberable tmp register, but at this
	// point in the exception vector we can't spare the extra GPR.  Instead note
	// that EL1_SP0_VECTOR ends with x0 == sp and use this to unclobber x0.
	mrs		x1, TPIDR_EL1
	LOAD_INT_STACK_THREAD	dst=x1, src=x1, tmp=x0
	mov		x0, sp
	mov		sp, x1
.endmacro

For the ordinary synchronous case, handler execution continues below C. For IRQ/FIQ, the frame stays on the interrupted stack while the handler uses the CPU interrupt stack. The IRQ vector selects those macros explicitly.

Mermaid source

flowchart TB
    subgraph Old["Interrupted kernel stack"]
        High["Existing kernel frames<br/>addresses at and above S"]
        Saved["C = S - ARM_CONTEXT_SIZE<br/>saved registers; saved_sp = S<br/>integer + NEON context"]
        Low["Synchronous handler frames<br/>grow below C"]
        High --- Saved --- Low
    end
    subgraph IRQ["CPU interrupt stack"]
        Top["I: separate stack top"]
        Body["IRQ/FIQ C call frames<br/>grow below I"]
        Top --- Body
    end
    Saved -. "x0, then x21, still names C" .-> Body

This is a layout relationship, not a claim that every kernel exception can safely use its interrupted SP. The synchronous vector first stages x0/x1 on the exception stack and checks relevant syndrome classes for a potentially corrupt kernel stack. SPTM IRQ/FIQ paths have a separate redzone check before creating the frame. SP1 exceptions use different fatal/recovery paths. Those exceptional cases are covered in exception entry.

The interrupt prologue publishes the context in CPU_INT_STATE and increments preemption accounting. Its epilogue clears that pointer, decrements the count, and loads the thread kernel-stack top before common return. Thus the IRQ C handler's SP does not determine the final restored SP; SS64_SP does.

3. Spill first, then call C: follow the changing jobs of x0, x1, x21 and LR

Both entry cases reach fleh_dispatch64 with x0=context, x1=handler, x2=options, and a usable kernel or interrupt stack. Original x0–x3 and SP are already saved. The dispatcher optionally signs its handler address with pacia x1,sp, then invokes the spill macro.

The general-register spill saves values before reusing the registers

	.if \mode != KERNEL_MODE
	stp		x2, x3, [x0, SS64_X2]
	.endif
	stp		x4, x5, [x0, SS64_X4]
	stp		x6, x7, [x0, SS64_X6]
	stp		x8, x9, [x0, SS64_X8]
	stp		x10, x11, [x0, SS64_X10]
	stp		x12, x13, [x0, SS64_X12]
	stp		x14, x15, [x0, SS64_X14]
	stp		x16, x17, [x0, SS64_X16]
	stp		x18, x19, [x0, SS64_X18]
	stp		x20, x21, [x0, SS64_X20]
	stp		x22, x23, [x0, SS64_X22]
	stp		x24, x25, [x0, SS64_X24]
	stp		x26, x27, [x0, SS64_X26]
	stp		x28, fp,  [x0, SS64_X28]
	str		lr, [x0, SS64_LR]

The .if is an assembler-time macro choice. On this call mode == KERNEL_MODE, so the x2/x3 stores are omitted: live x2 now contains options, not the interrupted x2. Each remaining stp saves an adjacent pair at a generated offset. The final pair stores x28 and fp/x29; str lr preserves the interrupted x30 separately. The KERNEL_MODE macro argument describes the spill convention, not whether the interrupted execution was EL0 or EL1.

The next part stores q0–q31, then uses already-saved x24/x25 to read FPSR/FPCR and stores their 32-bit values. It reads ELR/SPSR into x22/x23 and FAR/ESR into x20/x21. The PAC signing path passes the selected PC/CPSR/LR/x16/x17 values to ml_sign_thread_state; it saves the live handler address and LR around that helper's custom ABI. Afterwards the dispatcher may clear now-unneeded live user values, but their saved copies remain intact.

The dispatcher finally assigns x21=context and x22=handler, performs configured accounting, and uses braa x22,sp or br x22. This is another branch, not a C-style call. fleh_dispatch64 and its final branch establish these register roles.

The synchronous first-level handler prepares ordinary C arguments

	ARM64_JUMP_TARGET
	mrs		x1, ESR_EL1							// Load exception syndrome
	mrs		x2, FAR_EL1							// Load fault address
	mrs		lr, ELR_EL1
	/* NB: lr might not be a valid address (e.g. instruction abort). */
	PUSH_FRAME

The PUSH_FRAME and pop macros explain why the return sequence uses POP_FRAME_WITHOUT_LR: it restores FP and advances SP while discarding the stored link-register slot. No ordinary ret should jump to that synthetic record.

On SPTM builds the handler has additional lockdown checks before the following continuation:

The explicit PC/FAR stores, C call, and return dispatcher branch

Lfleh_synchronous_continue:
	/* We've had our chance to lockdown, release PC/FAR */
	str		x25, [x0, SS64_PC]
	str		x2,  [x0, SS64_FAR]
#endif /* CONFIG_SPTM */

	bl		EXT(sleh_synchronous)
	POP_FRAME_WITHOUT_LR

#if XNU_MONITOR && !CONFIG_SPTM
	CHECK_EXCEPTION_RETURN_DISPATCH_PPL
#endif

	mov		x28, xzr		// Don't need to check PFZ if there are ASTs
	b		exception_return_dispatch

Here bl sleh_synchronous sets live LR to the next assembly instruction. If C returns, execution reaches the pop; x21 still carries the context as a callee-preserved register. mov x28,xzr disables the interrupt-only PFZ check for this return, and b exception_return_dispatch enters the state-based return path.

sleh_synchronous derives state = &context->ss, decodes ESR and verifies user/kernel origin. Its SVC branch requires a 64-bit user context and calls handle_svc; that function reads the saved trap number and selects platform, Mach or BSD handling. Handler results modify the saved state, not a live x0 that can simply be carried unchanged through all these calls. For example, the absolute-time trap writes the result into saved x0.

Mermaid source

sequenceDiagram
    participant V as Long vector
    participant B as Saved context B or C
    participant F as Dispatcher/FLEH
    participant C as C handler
    V->>B: Save original x0–x3 and SP
    V->>F: x0=context, x1=handler, x2=options
    F->>B: Save remaining GPR/SIMD/control state
    F->>F: x21=context#59; branch to handler
    F->>C: bl sleh_synchronous(context, ESR, FAR, ...)
    C->>B: Inspect and possibly change saved state
    C-->>F: Return, if operation uses this call chain
    F->>F: Discard synthetic LR#59; branch to return dispatch

A configuration-sensitive spill-option caveat

FLEH_DISPATCH64_OPTION_SYNC_EXCEPTION selects DONT_SPILL_ELR_FAR on SPTM builds. However, this invocation passes x2 as the option register, and the PAC-signing branch overwrites w2 with the CPSR before a helper documented to clobber x2. The later option test still names that register. The visible code therefore does not justify a configuration-independent claim that the original option word survives and always defers those first stores. The explicit stores in fleh_synchronous above are clear. This is a configuration-sensitive static-source ambiguity, not a confirmed runtime defect or security conclusion. The compiled configuration and external monitor guarantees have not been established by this walkthrough.

4. Returning is a loop over saved state, not a single ret

exception_return_dispatch loads saved CPSR through x21 and tests PSR64_MODE_EL_MASK. The mode bits choose return_to_kernel or return_to_user; the current handler itself is already running in the kernel in both cases.

For user return, x3 is freshly loaded from TPIDR_EL1 before the AST check:

Read pending work only after masking exceptions

	msr		DAIFSet, #DAIFSC_ALL				// Disable exceptions
	ldr		x4, [x3, ACT_CPUDATAP]				// Get current CPU data pointer
	ldr		w0, [x4, CPU_PENDING_AST]			// Get ASTs
	cbz		w0, no_asts							// If no asts, skip ahead

	cbz		x28, user_take_ast					// If we don't need to check PFZ, just handle asts

Handle work and retry the check if C returns

user_take_ast:
	PUSH_FRAME
	bl		EXT(ast_taken_user)							// Handle all ASTs, may return via continuation
	POP_FRAME_WITHOUT_LR
	b		check_user_asts								// Now try again

Each iteration constructs/discards a temporary unwind frame around ast_taken_user; it does not consume the PCB. The final b repeats the check, allowing newly posted work to be seen. If user preemption blocks with a continuation, thread_preempted later calls thread_exception_return; arm64_thread_exception_return reloads the PCB into x21 and enters this return path. It need not resume the old bl ast_taken_user instruction's call chain.

Kernel return differs: return_to_kernel checks saved IRQ masking, current preemption nesting and AST_URGENT, and restores SP to the kernel context before eligible preemption. The interrupted kernel frames must remain resumable. See context switching for the saved-stack versus continuation mechanics.

5. Authenticate candidates, restore registers, and stop using the context address

The final path first does mov sp,x21, making active SP0 point at the saved context. This is a deliberate temporary addressing state like the early EL0 spill, not the final stack for resumed execution.

With __ARM_KERNEL_PROTECT__, the return-mapping calculation checks the destination mode, computes the restore label's offset from ExceptionVectorsBase, adds ARM_KERNEL_PROTECT_EXCEPTION_START, and branches to that alias. The CPU is still executing at EL1. Moving to the retained mapping now allows later instructions to reduce kernel visibility without removing their own instruction mapping.

Load and authenticate restore candidates before writing hardware return state

Lexception_return_restore_registers:
	mov 	x0, sp								// x0 = &pcb
	// Loads authed $x0->ss_64.pc into x1 and $x0->ss_64.cpsr into w2
	AUTH_THREAD_STATE_IN_X0	x20, x21, x22, x23, x24, x25, el0_state_allowed=1

	msr		ELR_EL1, x1							// Load the return address into ELR
	msr		SPSR_EL1, x2						// Load the return CPSR into SPSR

The authentication macro initially loads saved CPSR into w2, saved PC into x1, and x16/x17 from the context. On PAC builds it preserves additional scratch registers, then uses this sequence:

Preserve the exact candidate values across the custom checker ABI

	mov		\tmp1, x1
	mov		\tmp2, x2
	ldr		x3, [x0, SS64_LR]
	mov		x4, x16
	mov		x5, x17
	bl		EXT(ml_check_signed_state)
	mov		x1, \tmp1
	mov		x2, \tmp2
	mov		x16, x4
	mov		x17, x5
	msr		SPSel, #0

The macro restores LR from x3 and recovers its scratch inputs. The caller then writes the checked PC/CPSR into ELR_EL1/SPSR_EL1. Saved-state authentication covers selected control-flow values; it does not authenticate the saved SP, every GPR or all SIMD bytes. The hash implementation also deliberately masks the EL0 carry bit used by BSD syscall result encoding.

Mermaid source

flowchart LR
    M["Saved context memory"] --> L["Load candidate PC, CPSR,<br/>LR, x16, x17"]
    L --> P["Preserve candidates<br/>in helper-preserved registers"]
    P --> A["Check signature<br/>mismatch -> panic"]
    A --> H["ELR = checked PC<br/>SPSR = checked CPSR"]
    A --> R["LR/x16/x17 from<br/>checked register copies"]
    H --> E["eret"]
    R --> E
    M --> O["Restore other registers,<br/>SIMD and saved SP"]
    O --> E

The intervening restore code handles FPSR/FPCR and either NEON or active streaming SME state, then configured tag/synchronization work. Ordinary GPR restoration deliberately skips x16/x17/LR, already loaded and checked, and handles x18 according to the platform policy.

The final loads consume the context pointer last

	// Skip x16, x17 - already loaded + authed by AUTH_THREAD_STATE_IN_X0
	// Skip x18 - already restored or trashed above (below with __ARM_KERNEL_PROTECT__)
	ldr		x19, [x0, SS64_X19]
	ldp		x20, x21, [x0, SS64_X20]
	ldp		x22, x23, [x0, SS64_X22]
	ldp		x24, x25, [x0, SS64_X24]
	ldp		x26, x27, [x0, SS64_X26]
	ldr		x28, [x0, SS64_X28]
	ldr		fp, [x0, SS64_FP]
	// Skip lr - already loaded + authed by AUTH_THREAD_STATE_IN_X0

	// Restore stack pointer and our last two GPRs
	ldr		x1, [x0, SS64_SP]
	mov		sp, x1

#if __ARM_KERNEL_PROTECT__
	ldr		w18, [x0, SS64_CPSR]				// Stash CPSR
#endif /* __ARM_KERNEL_PROTECT__ */

	ldp		x0, x1, [x0, SS64_X0]				// Restore the GPRs

The x20/x21 pair now restores interrupted values; it must no longer be used for the handler's bookkeeping. ldr x1,[x0,SS64_SP] loads U or S; mov sp,x1 installs it as SP0. Yet x0 still points at the kernel context, so the last ldp x0,x1,[x0,SS64_X0] can recover both registers using the old base value. It has no writeback operand. After that instruction, x0 is no longer a context pointer, and this path performs no more context loads.

On protected-kernel builds the preceding ldr w18,[x0,SS64_CPSR] saves mode information for the mapping branch. This is a separate memory load; actual exception return uses the PC/CPSR already installed in hardware. An EL0 destination writes TCR_EL1_USER, then changes the ASID selection:

Select the user ASID after the return code is in its retained mapping

	/* Switch to the user ASID (low bit clear) for the task. */
	mrs		x18, TTBR0_EL1
	bic		x18, x18, #(1 << TTBR_ASID_SHIFT)
	msr		TTBR0_EL1, x18
	mov		x18, #0

#if defined(ERET_IS_NOT_CONTEXT_SYNCHRONIZING)
	isb		sy
#endif /* defined(ERET_IS_NOT_CONTEXT_SYNCHRONIZING) */

Lskip_ttbr1_switch:
#endif /* __ARM_KERNEL_PROTECT__ */

	ERET_NO_STRAIGHT_LINE_SPECULATION

bic clears the same ASID bit that entry's orr set. x18 is cleared before returning to userspace; required instruction barriers are configuration-dependent. The final macro expands to eret followed by a straight-line speculation-limiting sequence. eret consumes ELR/SPSR to resume the chosen PC and state; it does not pop the synthetic C frame and does not branch through live LR. For a kernel return, the EL0 mapping-reduction branch is skipped.

Configuration boundaries and reading checks

For an interrupt that occurred during SPTM/TXM/SK execution, xnu_return_to_gl2 selects a dedicated resume boundary. The monitor's internal register/context restoration is external to this source. Follow exception levels and monitors, SPTM, TXM, and Exclaves execution rather than extending the ordinary EL0/EL1 trace by assumption.

The decisive reading checks are: identify the active SP bank; track B/C independently of handler SP; verify each register was saved before becoming scratch; distinguish b/br from bl; find where x21 stops naming the context; and ensure the final loads cannot accidentally reload a value after its authentication. These checks apply to the actual instruction sequences above without requiring a guessed stack-size constant.

This chapter read the selected vector, spill, dispatcher, synthetic-frame, AST, authentication and restore bodies, their offset definitions, and the relevant synchronous C boundary. It does not rederive every syndrome handler, debug/MTE/SME branch, fatal SP1 path, CPU workaround, or monitor implementation. Exact excerpts are mechanically checked against the checkout; the path was not booted or single-stepped on hardware.