Mach runtime: execution, scheduling, and synchronization

Guide index · Architecture

The runtime turns a task's resources and a thread's saved execution state into work performed by CPUs. It also provides the waits, locks, timers, and accounting used throughout XNU. The central files are in osfmk/kern/, but their clients include BSD, Mach IPC, VM, and I/O Kit. Start with the object relationships below, then follow one blocking operation from its caller through the scheduler. task.h, thread.h, sched_prim.c.

Mermaid source

flowchart TD
    Task[task: resources and thread membership] --> Map[VM address map]
    Task --> Space[IPC namespace and task ports]
    Task --> Thread[thread: execution and scheduling state]
    Thread --> BSD[BSD uthread and process linkage]
    Thread --> Wait[waitq: waiting for an event]
    Wait --> Runnable[thread_go / thread_setrun]
    Runnable --> Pset[processor_set: scheduler queues]
    Pset --> CPU[processor: active_thread]
    CPU --> Switch[machine_switch_context]
    Thread --> Accounting[Recount / ledgers / bank]
    Group[thread group and QoS bucket] --> Pset
    TS[turnstile: priority inheritance] --> Thread

The arrows identify relationships and control transfers; they do not all denote owning references. For ownership and synchronization, inspect the field declarations and the creation/teardown paths linked in objects and lifecycle. In particular, a task is not the scheduler's execution unit, and TH_RUN does not prove that a thread is currently on a CPU. thread.h, state definitions.

Reading route

The key-structures section adds a storage-focused route. Start with the task and its shared read-only data for allocation, references, accessors and final release, then follow a protected field update through the zone API and ARM64 backend.

Thread, uthread and thread_ro continues through combined writable storage, protected associations, creation/publication, accessors, state gates and staged destruction. Its credential companion explains the two identity pointers cached by the thread and the ownership transferred when either changes.

1. Objects and lifecycle: distinguish task, thread, processor, and processor set; trace creation and staged destruction.
2. Scheduling: understand the common scheduler core before studying the Clutch hierarchy and Edge placement policy.
3. Waits, locks, and priority inheritance: follow an event wait and a contended mutex; learn which locks protect the transition.
4. Time, deferred work, and accounting: distinguish timer callouts from worker-thread callbacks, then follow clocks, Recount/coalitions/work intervals, ledger settlement and bank charging through focused companions.
5. AST mechanism and all AST flavors: publication in CPU/thread masks, remote checks, interrupt and user-return consumption, preemption, APCs, signals, guards, telemetry and deferred payloads.

Detailed execution and lifetime paths

The introductory pages above provide the object map. These chapters follow the implementation through decisions, lock handovers, races, and cleanup:

1. Scheduler execution: current-thread eligibility, placement/selection retries, kernel-stack allocation, continuations, ARM64 context switching, departure, and termination reaping.
2. Clutch and Edge: the three-level runnable hierarchy, deadline/warp/starvation windows, group interactivity, cluster migration weights, availability checks, and stealing.
3. Wait and turnstile protocols: wait registration and TH_WAKING, mutex retries, donated turnstile storage, inheritor references, and generation-checked priority-chain traversal.
4. Suspension, stopping and abort: separate persistent suspension counts, safe stopping, wait interruption, abort requests and resumption.
5. Timer and thread-call lifetime: timer-lock collision handling, callback execution context, ONCE rearm, worker references, completion waits, cancellation branches, and owner teardown.

For lock implementations, continue with reader/writer locks and ticket and pointer locks. The first distinguishes the default legacy RW representation from the newer implementation explicitly used by VM; the second follows ticket reservation, invalidation and pointer publication. Keep each primitive's acquisition, blocking and release rules separate from the mutex/turnstile protocol.

For concrete wait clients, read Mach semaphores and eventlinks: credits versus event counters, combined signal/wait ordering, timeout and association, handoff, and destruction while operations remain referenced.

Clocks and accounting mechanisms

1. ARM64 clocks and time domains: counter access, timebase scaling, sleep offsets, calendar/NTP state and commpage read/publication protocols.
2. Recount and resource attribution: per-owner topology, snapshot deltas, sequence reads, terminated/live rollups, coalition reporting and work-interval telemetry/lifetime.
3. Ledgers, limits and settlement: encoded entry features, per-CPU deferred pairs, maintenance-IPI settlement, refill, limits and AST handoff.
4. Bank accounts and charging: voucher attribute roles, shared accounts, adoption and quantum attribution, live bills and final task totals.

Detailed execution policy

The work-interval policy and thread-group companion connects workload configuration to class/priority decisions, explicit joining and group selection. A two-thread semaphore wakeup then traces conditional auto-join, blocking unwind, remote demotion and deferred storage release.

Continue with Thread-group accounting and cluster migration to follow a resolved group into Clutch runnable counts and CPU attribution, then an Edge preference update into future placement, running-thread interrupts and queued-thread requeueing.

1. Priority, QoS, and quanta: effective-policy composition, base/scheduled priority, common versus Clutch decay, quantum accounting, nonurgent preemption, and stacked failsafe demotions.
2. Realtime deadlines and runqueues: policy validation/adjustment, work-interval restrictions, deadline lifetimes, conditional priority/EDF selection, backup/spill notifications, and queue ownership.
3. CPU eligibility and scheduler variants: compile-time policy choice, dualq/AMP alternatives, processor binding, SMT siblings, recommendation changes, deferred migration, and maintenance-starvation recovery.

Useful questions to keep beside a source file are: Which thread is running this function? Can it block? Which lock makes the condition and wait registration atomic? Who retains the objects after a lock is released? Does this build even select the implementation being read? The answers are often in assertions and nearby comments rather than the public function name. Concrete examples are thread_create_internal()'s returned-lock contract, thread_invoke()'s preemption assertion, and compile-time SCHED(f) selection. thread.c, sched_prim.c, sched_prim.h.

Major boundaries

Continue into the neighboring guides for processes and BSD, VM, Mach IPC, drivers, and platform execution.

Sources inspected and limits

“Deep path review” means the named function bodies and adjacent contracts were read. It does not mean every branch in these large files was audited, compiled, or executed.

The existing Clutch/Edge design, task-reference debugging, Recount, and coalition accounting documents complement the source. Their introductory/design sections were consulted; code in this checkout controls implementation-specific statements here.