A practical route through XNU

Guide index · Architecture · Source map

Read an operation, then widen the boundary

For each operation, identify its entry point, the object it acts on, the lock/reference protecting that object, its normal path, its blocking or failure path, and its cleanup. Those six questions are a more manageable unit than “read the VM subsystem.” Keep a small call graph and add only the branches needed to answer your question.

The following passes build on one another. They are an order of study, not a claim that every listed file should be read in full.

Pass 1: learn the identities

Read architecture.md, then compare the definitions of proc, task, thread/thread_ro, and uthread. Write down which layer owns PID identity, the address map, the IPC namespace, scheduling state, and BSD thread state.

Then read proc_task() in kern_proc.c and get_threadtask() in bsd_kern.c. The accessor implementations are useful precisely because current layout differs from older diagrams. Continue into the runtime lifecycle guide and BSD lifecycle guide.

Checkpoint: explain how one process can have several schedulable threads, why a task is not a CPU execution context, and why a reference and a PID are not interchangeable ways to keep a process alive.

For a deeper structure study, continue through key structures, the process and its protected data, the task and its shared RO data, then writing protected object fields. Follow one selected field from its initialization template through an accessor and an update into final cleanup. Compare the combined writable allocation with the separate RO record, and keep backend write permission, object lifetime and field synchronization distinct.

Next follow thread, BSD state and thread_ro. Compare the combined thread/uthread allocation with its separate protected companion, then identify which references survive the final off-CPU transition. Read credential storage and MAC labels alongside it: trace one identity replacement through hash uniquing, process publication, thread-cache refresh and final retirement. Checkpoint: distinguish process identity, an assumed thread identity, a borrowed credential pointer, an owned credential reference and a long-term stored holding.

Pass 2: follow one entry from user space

Pick a small syscall from syscalls.master, locate its implementation, and inspect an architecture dispatcher: ARM or x86. Trace argument extraction and the return convention before following the operation itself. Use the process interface guide to connect dispatch, uthread, restart behavior, and signals.

Next compare a Mach trap in syscall_sw.c with an RPC declaration such as mach_vm.defs. Read the IPC guide to understand why a public function name can lead to a trap wrapper, generated MIG client, or kernel server implementation.

The complete interface map gives a searchable route through every BSD slot, every Mach trap slot, machine-specific selectors, and all MIG subsystems. Use each entry's purpose, argument declaration and implementation path to choose the next function; preserve the distinction between a declaration, a registered receiver and an external service.

For the instructions before and after C dispatch, follow the ARM64 assembly walkthroughs. Track the active SP bank, saved-context address, handler arguments, AST loop and exact register values used by exception return.

Then follow AST publication and consumption and the complete flavor catalog. Trace a producer setting thread state, CPU propagation or a remote check, consumption with the required interrupt state, and any payload cleared before the callback. Compare urgent kernel preemption with work eligible only on return to userspace.

Checkpoint: distinguish a syscall number, a Mach trap number, a Mach message ID, a port name, and a kernel object pointer. None is a substitute for the others.

For a harder IPC exercise, use complex message conversion. Give one header name two dispositions, then add memory and port-array descriptors. Predict which sender resources move and which receiver resources exist after each failure stage. Follow importance propagation separately: a task can stop contributing to the dependency graph while an applied boost remains pending delayed withdrawal.

Pass 3: understand blocking before scheduler policy

Read assert_wait*(), thread_block_reason(), and the wakeup-to-runnable path in sched_prim.c, with waitq.c beside it. Follow the waits and locks guide. Notice where the condition's lock is released, where the thread is registered as waiting, and how a wakeup racing with the block is handled.

Only then read scheduling.md. Separate common runnable/block/context-switch machinery from the selected scheduler's choice of thread and CPU. Study one Clutch bucket or one Edge placement decision at a time rather than trying to memorize the entire policy.

For primitive state transitions, follow reader/writer locks and ticket and pointer locks. Check the selected RW type, whether a failed upgrade retains any ownership, and whether a ticket has already been reserved. Keep a pointer returned under a lock distinct from a pointer value merely observed by a waiter.

Read ARM64 thread switching, entry and exit to connect that decision to cswitch.s. Trace a saved kernel call returning on another stack, a continuation starting on a reset stack, first user entry and the final off-CPU termination/reaping sequence.

Continue with priority, QoS and quanta, realtime deadlines, and CPU eligibility and scheduler variants. Work through the numerical priority/deadline examples, then identify which lock protects a queued thread while its priority or eligible CPUs change.

Checkpoint: explain why waking a thread does not guarantee immediate execution and why a callback cannot always sleep just because its caller is written in C.

Pass 4: translate a virtual address

Start with maps and faults. Locate a map entry, calculate the object offset, and follow a resident-page fault before examining a slow pager fault. Keep map, object, page, and pmap roles distinct. Read the current range/entry locking rules; do not impose a historical single-map-lock model on this source.

Use the mapping walkthroughs to follow anonymous memory, a mapped file, and COW. Read virtual address maps, objects and pages, and fault resolution for the structures and retry rules behind those examples.

Keep the five VM structure companions open beside those operations. Trace map/header/entry allocation, decode the backing and offset, distinguish object references from paging/activity holds, and identify the descriptor corresponding to a physical frame. Then compare pmap-object allocation with root/table/PTD/PV allocation in each ARM64 backend. Finish with a named memory entry: unwrap its single-object copy representation and account for the additional reference acquired by a destination mapping.

Next follow the resulting physical frame into ARM64 pmap, mapping lifetimes, and TLB/cache completion, or the separate x86_64 implementation. Then connect wiring and memory entries and pagers/UPL to page ownership during sharing and I/O. Finish with reclamation and kernel allocation.

Checkpoint: explain how a valid mapping can lack a resident physical page, why a resident page can still fault, how two mappings can share backing, and when an old translation stops being usable. Distinguish virtual size, resident size, compressed contents, and allocation-zone usage.

Then follow the harder retry and reclamation paths: delayed-copy preservation and collapse/bypass, copy-version revalidation and resilient faults, and UPL gathering/partial completion. Explain why an unchanged copy pointer can require new preservation work, why a page-in result does not authorize a PTE at an address that was remapped, and why a failed vector completion can have already retired an earlier child.

Then work through one mapped file and ARM64 fault. Keep a ledger of the vnode, pager, private VM objects, file/object offsets and physical frame across read, private write and unmap. Use the concrete address calculation to test the distinctions above.

For allocator internals, continue from the allocator overview to zones, kalloc routing and type isolation. Follow one metadata record from its macro into startup assignment, then through allocation and compatible free. Finally trace a refill or large request into guarded kernel VM allocation. Check which state is per call site, per type/signature group, per zone, per physical page, or per CPU.

For the protected-domain side of VM, follow SPTM/TXM/Exclaves boundaries. Trace one SPTM mapping/retype operation and one TXM signature association, then distinguish an Exclaves scheduling request from an RPC upcall. Checkpoint: identify what XNU retains across each call, how interrupt return selects the interrupted domain, and when a physical page becomes safe for XNU to reuse.

Use pmap translation walkthroughs to decompose concrete 4 KiB and 16 KiB addresses. Distinguish the physical location of a PTE from the physical page it names, then follow a mapping change through TLB completion and safe reuse.

Continue with MTE tagging and page lifetime to follow the mitigation from pointer tags and process opt-in through first faults, mapping provenance, allocator reuse and fault reporting. Distinguish data-page tags, allocator cells and physical tag backing; follow the zone free example to see when a stale pointer becomes invalid. Bootstrap memory and lockdown supplies the prerequisites: boot-policy selection, CPU controls, tag-generator state, early tagged zones and final protection. Then use ARM64 user-memory copying and AST handling to follow a kernel access to a user buffer through recovery and delayed notification. Track when an address becomes usable separately from when its storage becomes reusable or its tag checks are disabled.

Pass 5: follow a file descriptor into VFS or networking

Read fileproc and fileglob and the generic read/write code in sys_generic.c. Then choose a vnode-backed file or a socket. For a file, continue into VFS: path lookup, authorization, vnode lifetime, operation dispatch, and cache/pager integration. For a socket, continue into networking: socket buffers, protocol callbacks, packet representation, and interface input/output.

Keep the file structure route open beside those operations. Trace the embedded filedesc header and movable arrays, reserve and publish one slot, then duplicate it and account for separate fileproc I/O holds and shared fileglob ownership. Continue through vnode generation/iocount/usecount/hold distinctions into UBC, the vnode pager and the named VM association. Compare descriptor close with vnode reclaim and final pager release.

For a socket, keep the network structure route beside the operation. Distinguish the socket from its protocol attachment, the embedded sockbuf from its mbuf records, and one packet descriptor from shared external bytes. Work through the PCB reference and mbuf capacity examples, then compare a failed copy, a queue flush, protocol detach and final storage release.

Follow namespace mutations through a rename retry, then compare device nodes and FIFOs, stacked filesystem aliases, and an NFS request. Track exactly which layer owns the pathname, vnode identity, persistent reference, per-operation iocount and completion result.

Trace the decision itself through permissions and ACL evaluation: requested actions, ordered ACEs, unresolved rights, POSIX fallback, deletion checks and inheritance are distinct steps. For Skywalk, follow namespace reservation and lookup → classification and delivery → netif queues → FQ-CoDel scheduling. Explain who owns a packet after each conversion, enqueue, pending-neighbor result and drop.

For caching, compare vnode identity and authorization caches, UBC resident/missing-page reads, and buffer-cache ownership. Follow one read hit, one read miss, one truncate and one final close. Identify which of those operations changes file bytes, which releases an active owner, and which actually evicts cached state.

Checkpoint: explain why duplicating a descriptor can share an open-file offset, why a vnode is not just a pathname, and why socket readiness belongs to both networking state and the event/wakeup machinery.

For networking policy, follow NECP rule evaluation into client and flow-registration lifetime. Then trace socket content filtering, keeping pass permission, peek permission and actual reinjection separate. These byte-offset queues differ from Skywalk netif packet-filter channels.

For TCP, first follow connection and timer lifetime, then loss recovery and congestion control. Keep acknowledged stream bytes, SACK delivery evidence and transmission timestamps separate. For a channel transport with different ownership, use Skywalk user and kernel pipes after the packet-pool and channel guides: follow one packet attachment through transfer and distinguish an endpoint close from final ring storage release.

The worked Skywalk operation joins those mechanisms to guarded descriptors, IOKit task mappings and readiness. Explain each row of its two-batch backpressure table: which state changed in userspace, which was validated by the kernel, and which completion view is still stale.

Pass 6: learn IPC ownership and service boundaries

Use the IPC guide to follow a message from user-space port names through copyin, queueing or direct delivery, receive, and copyout. Track rights and out-of-line memory as well as inline bytes. Inspect the destination's right type and any kernel-object conversion.

Keep IPC spaces and entries beside ports and kobjects while reading the implementation. Trace the name's index and generation, the entry's right type and user-reference count, the port's lifetime and state, and the target object reference acquired by conversion. Then repeat the lookup across a table growth or receive-right destruction: locate the exact point where the implementation retries or rejects stale state.

Continue with IPC kmsg storage. Compute one inline/external layout boundary, follow the accessor selected by each representation, and maintain a resource ledger through copyin, queue transfer, copyout and destruction. Compare ordinary messages with the reusable mk_timer allocation: message-resource cleanup and final storage release follow different ownership transitions.

Then use drivers to follow a request from a user-client connection into an IOKit object. Compare a driver work loop with the BSD kevent workloop described in the process guide; the similar names do not imply a shared object type.

The driver index now provides a full reading route from bootstrap and object runtime through registry/matching, notifications, execution, user interfaces, memory and power. Compare a shared queue notification with a DriverKit action, and follow one buffer through CPU mapping before DMA preparation. Continue into segment generation and explain which returned address is physical or IOVA, where a padded mapping introduces a gap, and how a second capacity-limited call resumes.

Use the complete user-buffer DMA example to connect client-task selection, VM COW/wiring, descriptor ranges, command preparation, segment output and release. Then compare matching rescans and deferred starts with notifier selection, replay and removal. Explain why a matched event need not imply a successfully started child, and why callback self-removal does not end that callback's use of its context.

Checkpoint: explain why a port name is local to an IPC space, why sending a right changes ownership/accounting, and why possession of a connection does not mean every external driver method is authorized.

For receive multiplexing, follow port-set membership and persistent readiness and explain how a stale prepost differs from a queued message. Then follow live namespace growth: private copies can be inconsistent until changed entries are rescanned and the locked publication check succeeds.

Pass 7: connect policy and hardware

For signed-file storage, use the cs_blob companion alongside the signature operation chapters below. Follow one RO wrapper through its temporary template, byte-buffer relocation, PAC field preparation, publication, selected protected updates and cleanup. Distinguish a borrowed CodeDirectory/team string from a retained entitlement object and from an inline CDHash.

Read security around an operation you already understand: file open, process execution, task-port access, or a driver request. Find which credential is used, where an operation invokes authorization or a MAC hook, and what implementation lies outside the public source.

For identity changes, compare personas and membership resolution with audit-session capabilities. Explain which fields affect authorization, which identify an audit session, and which references survive process detachment or port-right release.

For code signing, read signature format → vnode attachment → exec/mapping policy → page validation. Trace one slice-relative CodeDirectory slot to a file offset and then to a resident VM page. Explain why an accepted blob, CS_VALID, a page's validated mask and csm_associated do not mean the same thing, and what changes after a cached write, COW, page eviction or vnode reclaim.

Then compare trust-cache control with process signing controls and CSR. Follow one load through admission and payload retention, one csops request through PID/version and command-specific gates, and one boot-derived CSR allowance to its consumer. A trusted code identity, process policy and resident-page integrity answer different questions.

For power transitions, follow device power state selection → system sleep and wake → hibernation image construction → image restoration. Compare x86 sleep/resume with ARM64 CPU lifecycle. Identify the last context that can use ordinary filesystem I/O, the switch to polled I/O, and the separate point at which normal scheduling and services resume.

Then read platform around a fault or syscall you already traced. Study boot order, interrupt context, exception entry/return, and the selected pmap backend. This gives machine code a concrete purpose instead of making assembly the first obstacle.

Checkpoint: distinguish a policy decision from a mechanism that enforces it, and identify which build guards change the path on the target you are studying.

A repeatable source-reading worksheet

The complete table of contents provides a section-level route through the whole handbook. For lifecycle and failure paths, compare completion and cancellation across the relevant subsystems. The SMR chapter supplies the shared reclamation protocol behind many lookup and teardown paths.

For a concrete concurrent lookup, continue from SMR into hash resizing and migration, then symbol interning. Track the borrowed hash match, successful object acquisition, unlink, old-array retirement and eventual backing release separately.

For a function worth understanding, record:

For example, the returned-lock contract of thread_create_internal() is essential to understanding its callers. The VM fault slow path deliberately drops locks and must revalidate the mapping. A vnode lookup must distinguish object lifetime from recycled identity. These are the details that turn a call graph into an explanation.

Efficient searches

Run these read-only searches from the XNU source root, xnu/ inside Platydoc (cd xnu first when starting at the project root). Documentation build/check commands instead run from the parent Platydoc directory:

# Find declarations, implementation entries, and callers together.
rg -n 'thread_block_reason\(' osfmk bsd iokit

# Follow a dispatched operation rather than looking for a direct call.
rg -n 'fo_read|soo_read|vn_read' bsd

# Discover an implementation's build guards and selection lists.
rg -n 'CONFIG_SKYWALK|config_skywalk' config bsd/conf bsd/skywalk

# Look for ownership contracts and nearby invariants in one file.
rg -n 'assert|LCK_MTX_ASSERT|reference|release|deallocate' osfmk/ipc/ipc_port.c

# Find interface definitions and possible generated-code origins.
rg --files osfmk libsyscall iokit | rg '\.(defs|iig)$'

Search output is a starting point. Read enough surrounding code to see the preprocessor branch, error labels, and the success/cleanup contracts. For a function pointer, locate both the table definition and its registration; for a macro, read its expansion; for a generated name, inspect the declarative interface and build rule.

Use diagnostics and tests to check your model

Skywalk diagnostics, configuration and tests connects inspected objects and counters to their exporters and local regression cases. Use its sizing and partial-output examples before treating a successful read as a complete, simultaneous snapshot.

Kdebug storage and readout follows instrumentation records through reservation, timestamp merging, loss markers and consuming output. Use its two-CPU example to distinguish event time from read order, and its output-failure path to see why an error does not restore already-consumed records.

Kernel logging follows one record through encoding, live streaming and Firehose retry, then compares those results with msgbuf text. Use its loss/publication examples before treating an absent log as evidence that an operation never ran.

The observability guide maps questions to traces, samples, counters, snapshots, and debugger macros. A macro that displays the fields you are studying can be a useful second explanation of their relationship. A subsystem test can provide a small scenario and expected result; see build and tests for the different harnesses and their requirements.

Keep three claims separate in your notes: “this body performs this step,” “this configuration selects this body,” and “this happened in a running kernel.” Source reading can establish the first, selected build artifacts help establish the second, and runtime evidence is needed for the third.

Argument, identity and lifetime paths

This route builds a source-level understanding of concrete operations. Read it in the following order, using each companion's object definitions, exact branches, diagrams and failure/cleanup paths. Test references describe available source evidence, not runtime results.

This route documents the selected XNU snapshot; it does not prescribe vulnerability research.

Executable memory, synchronization and packet policy

These companions continue the operation-based route above, concentrating on common code and selected ARM64 configurations. Each states the feature/platform guards that matter.

No kernel build or target runtime test is implied by the worked examples.

Exec stacks, executable I/O and scheduler placement

These three companions extend mechanisms already introduced above. Follow each representation change through the operation, including failures and ownership after the initiating call returns.

The worked layouts and schedules are illustrative; no product configuration or runtime result is inferred from them.

IPsec interfaces, MPTCP and process inspection

This route connects the packet, transport and process mechanisms with three operations whose state outlives an individual call or packet. Read the prerequisites first, then use the worked examples to follow ownership and identity through changes.

These chapters explain source behavior and illustrative schedules; they do not report a kernel or network experiment.