ARM64 bootstrap memory, ownership and lockdown

Platform overview · Boot to PID 1 · ARM64 pmap · Kernel allocation · MTE storage and page lifetime · SPTM

Before the kernel can allocate an ordinary object, it needs page tables, physical-page descriptors, maps and zones. Those facilities themselves require memory. ARM64 startup breaks that dependency cycle using image-resident storage, physical allocation cursors and an early mapping allocator. Later, it replaces temporary CPU stacks, releases selected image/boot memory and makes protected state immutable. This chapter follows those ownership changes and the MTE controls established around them. Memory Tagging Extension support adds a second bootstrap requirement: before an allocator can return a checked pointer, its data pages need tag-capable mappings and allocation-tag backing, and each CPU needs the appropriate checking and tag-generator state. The MTE page-lifetime companion follows allocation, retagging, faults and reclamation after these prerequisites exist.

The two local ARM64 boot implementations are selected separately: the config_pmap_ppl source set includes start.s, arm_init.c and arm_vm_init.c; the config_sptm set includes start_sptm.s and arm_init_sptm.c. Common VM bootstrap follows both. “Legacy” below means the non-SPTM implementation, with further PPL/integrity guards identified where relevant. It does not mean that every branch in that file is active in one product. Build selection.

1. Boot arguments describe supplied memory; they do not allocate it

The bootloader supplies the ordinary boot_args. The record describes the initial address/size inputs; it is not recovered from a running VM map. boot.h:50–65:

typedef struct boot_args {
	uint16_t                Revision;                       /* Revision of boot_args structure */
	uint16_t                Version;                        /* Version of boot_args structure */
	uint64_t                virtBase;                       /* Virtual base of memory */
	uint64_t                physBase;                       /* Physical base of memory */
	uint64_t                memSize;                        /* Size of memory */
	uint64_t                topOfKernelData;        /* Highest physical address used in kernel data area */
	Boot_Video              Video;                          /* Video Information */
	uint32_t                machineType;            /* Machine Type */
	void                    *deviceTreeP;           /* Base of flattened device tree */
	uint32_t                deviceTreeLength;       /* Length of flattened tree */
	char                    CommandLine[BOOT_LINE_LENGTH];  /* Passed in command line */
	uint64_t                bootFlags;              /* Additional flags specified by the bootloader */
	uint64_t                memSizeActual;          /* Actual size of memory */
} boot_args;

The SPTM arm_init() copies both argument records into late-read-only storage. It explicitly uses the monitor's first available physical address because SPTM and other components have consumed pages after the bootloader's boundary. A copied record preserves the values; it does not transfer ownership of every referenced region into the allocator. Argument adoption, physical aperture/root adoption.

Image segments add another set of intervals. arm_vm_init() discovers Mach-O ranges for text, constant data, bootstrap data, link information and prelinked images. arm_vm_prot_init() expands the low image boundary for kernel-collection headers and selected device-tree/trust-cache data. The TrustCache entry is a physical-address/length record with size and placement checks. SPTM also imports auxiliary-collection boundaries; the legacy path constructs/protects local mappings. These ranges prevent “everything before the free cursor” from being treated as one homogeneous disposable buffer. Legacy segment/protection setup, SPTM segment boundaries.

Bootloader allocation, firmware carveout policy and monitor-internal image validation are not supplied by these consumers. Where XNU checks supplied relationships, the chapter explains those checks; it does not reconstruct how the prior stage selected the addresses.

2. First translations and stacks precede ordinary kernel allocation

Legacy/PPL-selected cold entry

start_first_cpu() masks interrupts, keeps the boot-argument PA in x20, loads initial virtual/physical bases into x22/x23, and uses x25 for the bootstrap-table storage. It computes virtual stack addresses from the initial base relationship, installs the exception stack in SP_EL1 and the interrupt stack in SP_EL0, and prepares lr as the virtual address of arm_init(). These are temporary CPU stacks, not ordinary thread stacks allocated by kmem_alloc(). Register and stack preparation.

It then creates a V=P identity mapping for the running transition and a kernel-virtual mapping of the image/memory span. The macro allocates additional L2 tables as required while advancing VA/PA cursors and the free table pointer. The earlier “four pages” explanatory comment is not a universal allocation count: the later implementation explicitly reserves two L1 roots and a variable number of L2 pages, bounded by the bootstrap storage configuration. The code also adjusts its bases to avoid mapping memory before the kernel-cache header. Mapping macro, base adjustment and variable tables, image-resident table buffer.

common_start programs the translation regime and table bases, cache attributes and vector address, then enables translation/caches through SCTLR and continues at the prepared virtual C entry. Warm/secondary paths have different inputs and can select an already-established cpu_ttep; they must not be described as rebuilding the cold VM state. Translation setup, vectors/SCTLR and C argument.

SPTM cold entry

The SPTM entry receives a reason in x0, ordinary boot arguments in x1 and monitor arguments in x2. Cold startup installs the supplied image's temporary stack addresses, preserves the argument pointers in x26/x27, rebases/signs absolute addresses and initializes static branches. It sets XNU vectors before issuing SPTM_FUNCTIONID_FIXUPS_COMPLETE. The local comments identify the initial executable region and monitor transition; the source does not contain the monitor's page-table construction or protected-register enforcement. Entry contract, cold setup and handoff.

Do not transplant the legacy TTBR-writing sequence into this route. SPTM arm_vm_init() adopts the monitor root via libsptm_state.root_table_paddr; it does not construct that root with the legacy alloc_ptpage(TRUE) sequence. Adopted root.

Mermaid source

flowchart TB
    B["Boot arguments and image storage"] -->|"PPL-selected entry"| L["Identity and kernel-VA bootstrap tables<br/>temporary CPU stacks"]
    B -->|"SPTM-selected entry"| S["Monitor-established mappings<br/>fixups-complete handoff"]
    L -->|"construct root<br/>and aperture"| LV["Legacy arm_vm_init"]
    S -->|"adopt root<br/>and aperture"| SV["SPTM arm_vm_init"]
    LV -->|"publish PA / VA bounds"| V["Common VM bootstrap"]
    SV -->|"publish PA / VA bounds"| V
    V -->|"normal kernel thread and allocators exist"| F["Finalize mappings and stacks<br/>release selected storage and lock down"]

MTE needs a tagged pointer, tagged storage and a checking execution context

MTE compares a pointer's logical tag with the allocation tag of the addressed 16-byte granule on a checked memory access. The two tags serve different roles: a pointer carries the caller's expectation, while the backing tags describe the current allocation. Retagging storage on allocation or reuse can therefore make a stale pointer fail even when its virtual address still maps valid RAM. It can also detect an access crossing into a differently tagged neighbor. The tag is only four bits, and the implementation describes neighboring-tag exclusion as a best effort; this is not complete bounds checking or proof that every stale pointer fails. Granule and storage ratio, tagging both pointer and memory.

Boot must establish several independent pieces. Their presence must not be inferred from a single HAS_MTE test:

Platform feature declarations, XNU policy predicates, root configuration. Follow the MTE companion for the task/map and allocation decisions.

Build selection, boot policy and feature reporting are different checks

In this snapshot, H17's header defines both HAS_MTE and the reseeding requirement. Board configurations including that header supply the compile-time capability; for example T8142 and T6050 do so explicitly. These are source configuration names, not a claim about every shipped device or OS configuration. H17, T8142, T6050.

The boot-policy matrix below applies only inside a HAS_MTE build. The bit values and defaults come from mte_xnu.h:40–94; the consuming body is mte_config_setup().

The forcing and validation branches are worth reading together. osfmk/arm64/sptm/arm_init_sptm.c:292–315:

	/* -disable_mte is the legacy big hammer that switches off all of MTE */
	if (static_if_boot_arg_uint64(args, "-disable_mte", 0)) {
		mte_config = MTE_CONFIG_DISABLE_ALL;
		mte_debug_config = MTE_DEBUG_DISABLE_ALL;
	}
#if KASAN
	/* Until KASAN supports MTE, force disable. */
	mte_config = MTE_CONFIG_DISABLE_ALL;
	mte_debug_config = MTE_DEBUG_DISABLE_ALL;
#endif /* KASAN */

	MTE_EVAL_BOOTARG(MTE_ENABLE_KERNEL, mte_config, mte_config_kern_enabled);
	MTE_EVAL_BOOTARG(MTE_ENABLE_KERNEL_PURE_DATA, mte_config, mte_config_kern_data_enabled);
	MTE_EVAL_BOOTARG(MTE_ENABLE_USER, mte_config, mte_config_user_enabled);
	MTE_EVAL_BOOTARG(MTE_ENABLE_USER_PURE_DATA, mte_config, mte_config_user_data_enabled);

	/* Panic on invalid configurations. */
	if (mte_kern_data_enabled() && !mte_kern_enabled()) {
		panic("Kernel DATA tagging requires kernel tagging");
	}

	if (mte_user_data_enabled() && !mte_user_enabled()) {
		panic("User DATA tagging requires user tagging");
	}

There is a source-level qualification to the user-only example in the header. It advertises mte_config=0x4, but immediately below the predicates the implementation defines mte_enabled() as only mte_kern_enabled(), with a comment saying user-only operation is not expected to be deployed. Common tag-storage initialization uses that alias. Consequently the mask is expressible in the debug parser, but the local source does not justify advertising it as an independently verified boot mode. Example and limitation, alias, storage initialization guard.

Feature reporting is separate again. commpage_init_arm_optional_features_pfr1() decodes the architectural MTE version and related fields into gARM_FEAT_MTE, MTE2/3/4 and ASYNC indicators; PFR2 reports store-only support, and a later helper derives canonical/no-address-tag feature indicators from MTE4. These values describe reported CPU features. They do not replace the build guard, boot policy, allocator choice or task admission path. PFR1/PFR2 reporting, derived features.

The policy is fixed during the writable-text fixup window

SPTM cold entry calls arm_static_if_init() after rebasing the image and before setting final vectors and making SPTM_FUNCTIONID_FIXUPS_COMPLETE. The C wrapper passes boot_args.CommandLine directly to static_if_init(). That initializer links each key to its patch sites, invokes registered callbacks including mte_config_setup(), and flushes the instruction cache. On ARM64, a changed static condition becomes a branch or NOP instruction at its patch point. This is how a boot choice becomes a cheap hot-path condition rather than a continually writable global switch. Entry ordering, command-line wrapper, initializer, instruction patcher.

__static_if_key_delta() rejects changes once startup_phase >= STARTUP_SUB_TUNABLES; its comment also identifies the earlier SPTM text retyping boundary. FIXUPS_COMPLETE and the much later machine_lockdown() are therefore distinct security milestones. The former closes the local text-fixup window; the latter finishes read-only data/mapping preparation. Neither is a license to modify MTE boot policy at runtime. Timing guard and patch traversal.

Mermaid source

sequenceDiagram
    participant B as Prior boot stage / SPTM
    participant A as start_sptm.s
    participant C as Static-if setup
    participant V as arm_init and common VM
    participant L as machine_lockdown
    B->>A: Cold reason, boot args, SPTM args and initial mappings
    A->>C: Rebase image then parse CommandLine
    C->>C: Validate MTE policy and patch branches
    C-->>A: Flush instruction cache
    A->>B: Final vectors and FIXUPS_COMPLETE
    Note over B,A: Monitor text retyping is an external implementation
    A->>V: Enter arm_init
    V->>V: Initialize tag generator and adopt tag-storage geometry
    V->>V: Build VM metadata and tagged early-zone backing
    V->>L: Run bootstrap thread and release eligible boot memory
    L->>B: Finish mappings and request read-only lockdown

The sequence shows dependency order, not every intervening startup call. In particular, the monitor supplies initial translation state before XNU's generator and VM bookkeeping are ready.

Register controls: what is visible and what belongs to SPTM

The local header defines an MTE-specific control bundle selecting tag access and synchronous checking for both EL1 and EL0. A definition alone does not establish where that bundle is written: osfmk/arm64/proc_reg.h:601–602:

#define SCTLR_EXTRA               (SCTLR_ATA_ENABLED | SCTLR_ATA0_ENABLED | SCTLR_TCF_SYNC | SCTLR_TCF0_SYNC)
#define SCTLR_MTE_CONFIG          SCTLR_EXTRA

TCF/access definitions, MTX/base composition, base constants, MAIR value, index alias, GCR/RGSR fields.

The legacy start.s contains a conditional write of the tagged MAIR byte into slot 4, then writes MAIR and invalidates translations. It also writes TCR_EL1_BOOT and SCTLR_EL1_DEFAULT. However, the local SCTLR_EL1_REQUIRED/OPTIONAL/DEFAULT composition does not include SCTLR_EXTRA, and no local use of SCTLR_MTE_CONFIG supplies that missing link. Therefore the legacy SCTLR-default write is not evidence that this file enables MTE checking. The MAIR/TCR writes describe that selected entry implementation, not an instruction sequence performed by start_sptm.s. Actual SCTLR default composition. In the SPTM route, XNU adopts the root/aperture and says SPTM owns system-register locking; the actual initialization/enforcement of those protected controls is outside this checkout. Legacy MAIR write, legacy SCTLR write, SPTM responsibility.

Likewise, proc_reg.h describes MTE_TAG_OFFSET_EL2, a physically contiguous tag array aligned to 1 MiB and a lock bit, but this tree supplies no programming body for that register. The local evidence is a register contract plus XNU's consumption of first_tag_storage_paddr/n_tag_storage_frames; it is not evidence that XNU writes the EL2 register itself. Tag-offset definition, SPTM geometry adoption.

Initial tag randomness is per CPU and precedes tagged allocator use

arm_init() calls arm_mte_tag_generator_init(true) before entering the later VM/bootstrap phases. For a physical platform, have_apple_mte_tag_generator is a compile-time true constant. On APPLEVIRTUALPLATFORM, the boot CPU inspects AIDR_EL1.MTEVER and sets a late-read-only flag only for the recognized version. This probe selects generator behavior; it is not a general runtime discovery mechanism that inserts the MTE implementation into a binary compiled without it. Boot call, generator selection.

arm_mte_random_rgsr_el1_seed() obtains early_random(), masks it to the supported seed field, forces three low seed bits in Apple-generator mode, and retries a zero seed in the other mode. The documented reason is concrete: a zero LFSR seed would produce an endless zero stream. Initialization sets GCR_EL1.RRND when using the Apple generator and then writes the seed. The source explicitly relies on RGSR's ordering contract rather than adding a barrier between those writes. osfmk/arm64/machine_routines.c:3271–3278:

	if (have_apple_mte_tag_generator) {
		uint64_t gcr_el1 = __builtin_arm_rsr64("GCR_EL1");
		gcr_el1 |= GCR_EL1_RRND;
		__builtin_arm_wsr64("GCR_EL1", gcr_el1);
	}

	uint64_t seed = is_boot_cpu ? arm_mte_random_rgsr_el1_seed() : getCpuDatap()->mte_rgsr_el1_seed;
	__builtin_arm_wsr64("RGSR_EL1", seed);

This mitigates predictable tag selection; it does not turn the four-bit tag into a cryptographic identity. The later reseeding path exists precisely because the platform's tag generator is not treated as a sufficient long-lived source of unpredictability.

3. Early table allocation is a physical cursor or image reservation

The legacy alloc_ptpage(map_static) returns a virtual address. Integrity configurations can satisfy static tables from ropagetable_begin..ropagetable_end, an image-resident area that will become read-only. Otherwise it converts avail_start to a physical-aperture pointer and increments the physical cursor by ARM_PGBYTES. It neither asks a zone for an object nor removes a vm_page from the free queue. Without the relevant integrity configuration, the static request is forced to the dynamic cursor route. Legacy allocator.

The SPTM helper with the same general role returns a physical address, accepts a table level, and calls sptm_retype(..., XNU_PAGE_TABLE, ...). The level is part of the frame-conversion request. The similarly named functions are therefore not interchangeable APIs. SPTM allocator.

The SPTM dynamic branch and frame-type publication are visible in arm_init_sptm.c:1728–1740:

	} else {
		/* This is a RW allocation. Simply grab a page from [avail_start] */
		paddr = avail_start;
		avail_start += ARM_PGBYTES;
	}

	/* Retype the page to XNU_PAGE_TABLE, with the desired level */
	sptm_retype_params_t retype_params;
	retype_params.level = level;
	sptm_retype(paddr, XNU_DEFAULT, XNU_PAGE_TABLE, retype_params);

	return paddr;
}

ARM_PGBYTES is the hardware table-page granule in these helpers. Later VM ownership uses PAGE_SIZE, the software page size. Both arm_vm_init() implementations round the final free physical start up to a software-page boundary and the managed end down so VM never claims a partial page. An assumed “4 KiB everywhere” calculation would be wrong for the 16 KiB configurations and for test configurations with differing software/hardware sizes. SPTM managed-end calculation, SPTM final cursor, legacy final cursor.

The permanent root and physical aperture change address assumptions

Legacy initialization builds invalid_tte and the later cpu_tte, establishes segment protections and dynamic VA table coverage, constructs the physical aperture, switches TTBR1 to cpu_ttep, installs an invalid TTBR0 and flushes translations. Only then does it set kva_active and recompute table pointers through phystokv(). Root construction, switch and pointer repair.

The PPL physical-aperture builder records ptov_table_entry intervals rather than assuming one affine VA/PA relationship forever. It maps reusable/ownership-changing regions at page granularity, slides bootstrap segments and ordinary memory into their aperture locations, adjusts an unlocked device-tree pointer, and publishes the conversion table. A later gVirtBase/gPhysBase subtraction on an arbitrary aperture pointer can therefore describe the wrong physical address. Interval construction, PPL aperture construction.

SPTM supplies its initial aperture range and root; XNU continues local bookkeeping and allocates additional table pages through monitor interfaces. Some image regions undergo a later sptm_slide_region() handoff during finalization. The implementation of that slide is external. Both source files retain a TODO concerning reclamation of bootstrap table memory; leaving the bootstrap root does not establish that every page once used for tables was freed. SPTM adoption/TODO, legacy TODO.

4. Pmap stealing breaks the map/zone/page-descriptor dependency cycle

After architecture initialization, kernel_bootstrap() calls vm_mem_bootstrap(). The sequence matters: resident-page bootstrap runs before zone bootstrap; special VM objects precede map initialization; kmem_init() then adopts the remaining VA range and advances KMEM/ZALLOC startup. Common bootstrap, VM order.

vm_page_bootstrap() initializes locks and page queues, advances the PMAP_STEAL startup phase and allocates resident hash/lock bookkeeping through pmap_steal_memory(). The vm_map_steal_memory() startup callback reserves enough storage for early kernel/submaps and their entries/nodes before those zones can expand dynamically. With MTE it also creates cell metadata before the canonical page array. This is a deliberate bootstrapping reserve, not an ordinary late allocation of every map entry. Resident bootstrap, early phase, hash allocations, early maps.

pmap_steal_memory_internal() maintains a VA cursor independent of avail_start/first_avail:

1. Obtain the available kernel VA interval from pmap on first use; align the request and advance virtual_space_start.
2. For pages of the new VA span that still need backing, obtain physical frames with pmap_next_page_hi() and install writable kernel translations with pmap_enter().
3. Account them as stolen/wired and return the VA. The API has fatal early-allocation failure paths rather than a general rollback transaction.

Word-aligned subpage allocations can share already-backed pages. The map_addr = round_page(addr) start is significant: this is not one physical page unconditionally allocated per call. pmap_steal_freeable_memory() changes the lifetime hint; on the ARM implementations inspected, pmap_next_page_hi() delegates to pmap_next_page() and ignores that hint. It does not by itself free the returned storage. Steal allocator, wrappers, SPTM physical cursor.

These stages overlap in initialization, but they must not be collapsed into a claim that kalloc was already available to build all its own prerequisites.

Early zones explicitly acquire taggable pages instead of borrowing image DATA

The first VM-map zones exist before ordinary zone expansion. zone_early_mem_init() ordinarily points at zone_early_pages_to_cram in image DATA. With HAS_MTE, zone tagging selected and kernel MTE enabled, it instead calls pmap_steal_zone_memory(size, PAGE_SIZE): the static DATA buffer is canonically tagged, whereas zone elements need storage whose tags may be changed independently. Early-zone choice.

The specialized steal wrapper passes VM_MEM_MAP_MTE. For each newly obtained physical page, pmap_steal_memory_internal() first records stolen-tagged ownership in MTE metadata, converts the physical page to tagged use, initializes every granule to the canonical tag through the aperture pointer, and only then installs the requested VA mapping. The canonical initialization permits bootstrap code to access the storage through its initial canonical pointer; later element tagging supplies allocation-specific tags. osfmk/vm/vm_resident.c:2515–2522:

#if HAS_MTE
		if (pmap_flags & VM_MEM_MAP_MTE) {
			mteinfo_covered_page_set_stolen_tagged(phys_page);
			pmap_make_tagged_page(phys_page);
			/* Force the canonical tag. */
			mte_store_tag((void *)phystokv(ptoa(phys_page)), PAGE_SIZE);
		}
#endif /* HAS_MTE */

The remainder installs the translation and increments wired/stolen counts; failures panic because this early allocator has no general transactional unwind. The default steal wrappers do not pass VM_MEM_MAP_MTE, so “stolen during boot” does not mean “tagged.” Mapping/accounting, three wrappers.

Later zone_metadata_init() relocates those early VM structures into normal zone address space. It populates tagged destination backing when configured, copies bytes with checking temporarily disabled, copies the allocation tags separately, and updates early-map pointers. A byte copy alone would preserve object contents while losing the tag contract expected by existing pointers. Relocation order.

Mermaid source

flowchart TD
    E["Early VM-map zone reserve"] -->|"MTE and zone tagging enabled"| S["Steal ordinary physical page<br/>request VM_MEM_MAP_MTE"]
    S -->|"record ownership"| C["MTE cell marks stolen tagged page"]
    C -->|"frame conversion"| T["Tagged physical page"]
    T -->|"store canonical tag for every granule"| I["Initialize tags through aperture"]
    I -->|"pmap_enter"| V["Early zone VA mapping"]
    V -->|"zone allocator assigns element tags"| A["Live bootstrap VM objects"]
    A -->|"normal zone maps become available"| R["Populate destination<br/>copy bytes and allocation tags"]
    R -->|"rewrite references"| N["Objects in final zone address space"]

This is a bootstrap example of MTE joining the allocator lifecycle. It does not imply that all image globals, translation tables or stolen descriptors get per-allocation tags; each caller and mapping request still determines coverage.

5. Canonical page descriptors make the remaining frames allocatable

pmap_startup() estimates descriptor count from the remaining physical span and PAGE_SIZE + sizeof(struct vm_page), then steals storage for the vm_pages array itself. On ARM with a linear array, the span matters even when retired-page holes exist. The implementation notes that this estimate does not fully include page-table memory needed to map the array. It initializes descriptors while pmap_next_page() supplies frames, records vm_pages_first_pnum, and publishes free pages through startup release rules. Array sizing, descriptor initialization/release.

For SPTM, first use of the physical cursor initializes RAM ranges and optional MTE bookkeeping boundaries; subsequent calls advance first_avail by software pages and decrement available capacity until avail_end. It does not walk a running process map. Physical allocator, MTE range adjustment.

The initial wire count includes memory withheld from general VM even if no ordinary object performed vm_page_wire() on it. It is computed from total memory and free pools with feature-specific deductions, including MTE tag storage. Later releases must repair this accounting. The code also realigns and returns the unconsumed kernel VA range to vm_mem_bootstrap() for kmem_init(). Initial wire accounting, remaining VA publication.

MTE storage is not indiscriminately put on the ordinary free queue. Startup separates active tag backing, inactive claimable storage and recursive/unmanaged storage; some special-object insertion waits until KMEM initialization. See MTE tag storage.

6. Replacing CPU boot stacks must precede releasing their segment

By machine_lockdown(), execution is on a normal kernel thread bound to the boot CPU, so the temporary CPU exception/interrupt stacks can be replaced while they are not active. cpu_stack_alloc() allocates new permanent, zeroed, guarded stacks through kmem_alloc() and publishes their tops/pointers into BootCpuData. Each allocation has a guard page on either side; it is not merely a new pointer into BOOTDATA. Allocation and published fields, bootstrap-thread execution.

arm64_replace_bootstack() requires that kernel-thread context. It loads the new exception-stack top, masks asynchronous exceptions, selects SP_EL1, writes the new stack pointer, returns to SP_EL0 and restores DAIF. It does not copy an active boot call stack. Updating SP_EL1 must also precede lockdown of sensitive instructions on the relevant integrity configurations. Contract and assembly.

The critical assembly sequence is machine_routines_asm.s:1747–1766:

LEXT(arm64_replace_bootstack)
	ARM64_STACK_PROLOG
	PUSH_FRAME
	// Set the exception stack pointer
	ldr		x0, [x0, CPU_EXCEPSTACK_TOP]
	mrs		x4, DAIF					// Load current DAIF; use x4 as pinst may trash x1-x3
	msr		DAIFSet, #(DAIFSC_STANDARD_DISABLE)		// Disable all asynchronous exceptions
	// Set SP_EL1 to exception stack
#if defined(KERNEL_INTEGRITY_KTRR) || defined(KERNEL_INTEGRITY_CTRR) || defined(KERNEL_INTEGRITY_PV_CTRR)
	mov		x1, lr
	bl		EXT(pinst_spsel_1)
	mov		lr, x1
#else
	msr		SPSel, #1
#endif
	mov		sp, x0
	msr		SPSel, #0
	msr		DAIF, x4					// Restore interrupt state
	POP_FRAME
	ARM64_STACK_EPILOG EXT(arm64_replace_bootstack)

Legacy finalization replaces the stacks before freeing the BOOTDATA backing through its aperture address and clearing obsolete image aliases. SPTM finalization replaces them before asking the monitor to slide BOOTDATA, which would remove the old stack mapping. Freeing the physical pages is a later step in both explanations. Legacy order, SPTM order.

Stack guards are separate from MTE allocation tagging

Neither cpu_stack_alloc()'s permanent exception/interrupt stack flags nor stack_alloc_internal()'s ordinary thread-stack flags include KMA_TAG in the inspected bodies. KMA_KSTACK selects the pmap VM_MEM_STACK path; the MTE mapping flag is selected by a separate KMA_TAG branch. Thus the guarded boot-stack replacement above must not be presented as automatic per-frame or per-object MTE stack protection. The code does protect the stack boundaries with guards, but compiler stack tagging and universal stack coverage are not established by these calls. CPU stack flags, thread-stack flags, independent mapping flags.

Secondary CPUs and sleep restore generator state without rebuilding VM

Each cpu_data_t carries mte_rgsr_el1_seed and, under NEEDS_MTE_IRG_RESEED, cpu_irg_reseed_counter. The latter is initialized to zero with CPU data; these are separate from each other and from any page's allocation tag. During processor registration, the boot CPU generates a seed on behalf of each secondary: the source says secondary startup is too early to safely use the corecrypto-backed PRNG. Fields, saved seed, counter initialization, registration seed.

SPTM warm entry finds cpu_data_t by physical CPU ID, installs its exception/interrupt stack tops and dispatches only the expected reset handlers. arm_init_cpu() publishes the active thread and calls arm_mte_tag_generator_init(false), which consumes the saved per-CPU seed instead of requesting new random bytes. Cold VM geometry, static policy patching and boot-zone allocation are not replayed on every secondary startup. Warm stack/reset selection, active thread and generator.

The idle/WFI path saves the current RGSR_EL1 into that same per-CPU field before entering low power. The arm_init_cpu() warm/resume path then restores generator state. This preserves the current generator sequence across sleep-to-RAM and hibernation; it is different from both cold seeding and boundary reseeding. Idle capture, generator restore selection.

The separate arm_init_idle_cpu() handler for non-retention WFI publishes the active thread and calls cpu_idle_exit(TRUE) without calling the tag-generator initializer in its local body. Do not infer that every idle exit repeats the full arm_init_cpu() sequence or that this source exposes all monitor-side register restoration. Idle reset body.

Hibernation reveals why mapping state matters even for a nominally free page. The early restore code may borrow a free physical page for the hibernation header. Such a page can belong to an MTE free queue and retain tagged-memory attributes, while the aperture pointer used to copy it is canonical. arm_init_cpu() brackets that header copy with vm_memtag_disable_checking() and vm_memtag_enable_checking(). The source comment calls this MAIR=0x4; read it as attribute index 4, whose MAIR encoding is 0xF0. This scoped exception avoids assuming that all free pages have ordinary untagged mappings. It does not disable MTE for the rest of resume. Reason and copy bracket, attribute index.

7. Static release disconnects aliases and may delay free-list publication

ml_static_mfree(vaddr, size) takes a virtual range and visits complete software pages. It converts each address through kvtophys() and then phystokv() to find the current physical-aperture alias. For a present translation it disconnects the page's other pmap mappings, restores an allocator-compatible physical-aperture/frame state, and creates canonical VM bookkeeping. It adjusts initial/current wired and kernel-cache counts afterward. Release body.

ml_static_protect() is backend-specific. The SPTM branch retypes nondefault frames to XNU_DEFAULT. The legacy branch updates local permissions, rejects W+X or post-lockdown executable injection, and cannot arbitrarily change part of a block or contiguous-hint group whose permissions differ. A caller must not summarize the SPTM branch as directly editing the same PTEs as legacy. Protection/retype split.

There is a further lifetime boundary: a descriptor can exist before the frame is on a free queue. Until aperture sliding has completed, ml_static_mfree() creates the descriptor and puts it on ml_static_mfree_pre_slide_list. Reusing that page too soon could leave pmap users with pointers to an aperture alias that subsequently moves. After arm_vm_prot_finalize() returns, machine_lockdown() calls ml_release_deferred_pages() to release that entire list; future static frees can publish immediately. Deferred-list rationale and release, caller order.

The publication choice appears in machine_routines.c:2199–2206:

			if (__probable(!ml_static_mfree_queue_up)) {
				vm_page_create_canonical(ppn);
			} else {
				vm_page_t m = vm_page_create(ppn, true, Z_WAITOK);

				vm_page_list_push(&ml_static_mfree_pre_slide_list, m);
			}

Mermaid source

sequenceDiagram
    participant J as Bootstrap-data owner
    participant M as ml_static_mfree
    participant P as Pmap or monitor
    participant V as VM page bookkeeping
    participant L as machine_lockdown
    J->>M: Release an eligible static range
    M->>P: Resolve current PA and aperture alias
    M->>P: Disconnect aliases and restore usable frame state
    M->>V: Create descriptor on deferred list
    L->>P: Complete arm_vm_prot_finalize
    L->>V: Release deferred descriptors to free queues
    Note over V: Allocators may now receive these physical pages

The diagram describes a pre-finalization release. After deferred mode is disabled, the same API publishes through vm_page_create_canonical() directly. It is not a promise that every supplied byte is released: below-aperture requests can return early, only complete pages are visited, absent translations are skipped, and protection failure is fatal rather than silently returning unsafe memory to VM. Checks and failure policy.

8. Finalization and lockdown are related but separate operations

The bootstrap thread first loads the static trust-cache interfaces, runs STARTUP_SUB_LOCKDOWN callbacks, and jettisons kext bootstrap data with OSKextRemoveKextBootstrap(). That must precede machine_lockdown() because it can release or relocate static mappings. General IOKit matching is unlocked afterward through PE_lockdown_iokit(), and BSD startup follows. The startup phase named LOCKDOWN is therefore not itself the final hardware/monitor lock call. Common ordering, module teardown companion.

Legacy finalization, SPTM finalization, machine integrity dispatch.

Linkedit is not unconditionally discarded/slid: keepsyms, configured DTrace symbol retention and the KASAN dynamic denylist alter the decision. Likewise, integrity macros determine which static tables/instructions can become immutable. The final non-SPTM permission update deliberately leaves DATA_CONST until other changes are complete because that region can contain the very tables needed to make those changes. SPTM symbol retention, legacy last protection.

machine_lockdown() performs finalization, releases deferred pages, dispatches the configured integrity operations, invokes the applicable platform HMAC invalidation callback and finally sets lockdown_done. The local SPTM call documents a potentially long retyping operation and disables preemption around it. Its comment about monitor-controlled register locks is an interface explanation; absent monitor code cannot establish the exact protected-register sequence. Machine completion.

Lockdown does not freeze operational MTE state

The immutable and mutable state have different owners. Boot policy has already been patched into protected text, and tag-storage bounds such as mte_tag_storage_start/end/count are late-read-only globals. machine_lockdown() then finalizes mappings and calls sptm_lockdown_xnu() for read-only enforcement. In contrast, individual allocation tags, active/borrowed tag-storage pages, per-CPU generator state and a thread's temporary TCO state must continue changing during normal operation. Protecting policy/configuration is compatible with retagging allocations and changing backing-page ownership through the checked interfaces. Protected geometry declarations, lockdown request, tag write interface.

Entry, return and scheduling preserve the checking boundary

A tagged allocator cannot provide a useful mitigation if exception entry leaves tag checks disabled throughout the kernel. The local vector macro states that exception entry sets PSTATE.TCO; BRANCH_TO_KVA_VECTOR explicitly clears it before servicing the exception. osfmk/arm64/locore.s:234–241:

#if HAS_MTE
	/*
	 * PSTATE.TCO (Tag Check Override) is automatically set by the architecture
	 * whenever an exception is taken. TCO disables tag checking, so we want to
	 * restore it ahead of servicing the exception.
	 */
	msr		TCO, #0
#endif /* HAS_MTE */

When fleh_dispatch64 identifies an exception from EL0, it also reseeds the generator on platforms requiring it and writes a kernel exclusion mask into GCR_EL1: exclude tag F, the kernel canonical tag. On return, the restore path reseeds again under that guard, then checks the target mode in SPSR and selects the EL0 mask only for a user return: exclude tag 0, the user canonical tag. The kernel entry comment deliberately avoids adding a context-synchronization event solely for the mask write, because kernel generation additionally excludes F in its explicit mask. EL0-origin gate, return target test, explicit exclusion.

PACGA_IRG_RESEED obtains the current CPU data via the thread pointer, increments the per-CPU counter without an atomic operation because interrupts must be disabled, and uses two pacga results with different discriminators and the counter to construct fresh generator state. Apple mode also forces the designated low seed bits before writing RGSR. This limits observation of one tag-generator sequence across privilege boundaries. It is not a complete inventory of every context-switch call site, nor a claim that the hardware IRG sequence is cryptographically unpredictable on its own. Reseeding macro and preconditions.

Temporary checking bypasses are deliberately narrow. mte_disable_tag_checking() sets TCO and mte_enable_tag_checking() clears it; they do not save an arbitrary nesting state. Development/debug TCO validation can panic if a disable is entered while checking is already overridden. The byte/tag relocation and hibernation-header copies above are concrete trusted uses. Helpers, debug validation.

That API rule does not mean a scheduler switch loses TCO state. cswitch.s saves TCO in SS64_KERNEL_TCO and restores it with the kernel register context, so a suspended trusted path resumes with its state. This is a different mechanism from nested disable/enable calls. The context-switch synchronization logic also checks whether the new thread is in an unprivileged-access region and flushes a pending synchronization when it cannot defer to exception return. TCO save, restore, pending synchronization. Continue with user-memory copies and the MTE companion for fault classification and task consequences.

9. Reading a later physical page back through boot

Consider a page formerly backing a jettisoned bootstrap-data segment. This is an illustrative ownership trace, with addresses omitted because they depend on the selected image/platform:

1. The supplied image/boot intervals keep it outside the ordinary physical cursor. Its initial inability to move is reflected in initial wired accounting, even without an ordinary VM-object wire call.
2. A bootstrap consumer finishes using the data and calls the static-release path. The page can retain a usable physical-aperture translation after original aliases are disconnected.
3. The release path prepares permissions/frame type and creates a descriptor. If aperture finalization is incomplete, the descriptor waits on the deferred list.
4. Finalization finishes, deferred release puts it into an appropriate VM free queue, and an ordinary allocation can obtain it.
5. The new allocation can insert the page into a different VM object/map while the physical-aperture alias continues to exist for kernel access.

This is why “still mapped,” “has a canonical descriptor,” “counts as wired” and “available to allocation” answer different questions. The same distinction explains active MTE backing: its physical pages are represented by VM but deliberately owned/wired for tags. Static release, MTE companion.

10. Configuration, evidence and limits

This chapter follows selected local cold-boot paths, MTE boot policy and CPU controls, tagged early-zone storage, secondary/resume state, common VM bootstrap, static-page release and final integrity handoffs in xnu-12377.121.6. It does not claim a new hardware boot test, exhaustive SoC register coverage, complete page-table reclamation, firmware allocation knowledge or monitor-internal verification. The inspected source establishes the XNU side of these interfaces. SPTM/firmware register programming, hardware tag-array implementation and external policy enforcement remain unavailable; header constants and comments are not substitutes for those bodies. Normal exception/context switching and hibernation restoration retain their full execution/lifetime explanations in the existing companions.