Suspension, stopping and aborting threads

Runtime guide · Objects and lifecycle · AST mechanism · Task and thread capabilities

Suspension prevents a task or thread from returning to user execution while a hold remains. The request, the hold counter, the off-CPU wait, and the eventual APC parking are separate operations. An abort instead asks a blocked or executing thread to abandon eligible waits; it does not add a suspension hold. This chapter follows those ownership and scheduling boundaries in the selected XNU snapshot.

1. Entry points do not all require the same capability

The MIG declarations use task_read_t for both legacy task suspend/resume and task_suspend2, and thread_read_t for legacy thread suspend/resume and thread_suspend2. A control right can satisfy these read conversions, while an inspect right cannot. thread_abort and thread_abort_safely use thread_act_t, the control-thread conversion. Resuming through a token instead uses that token's special port type. Authority is the declaration plus converter and implementation checks, not the apparent destructiveness of a name. task declarations, task token declarations, thread declarations, thread token declarations.

The task and thread suspend2 protocols differ in this release. The task token is mach_port_move_send_once_t; the thread declaration uses mach_port_move_send_t. A task identity token is a third, unrelated object, used to resolve identity and request a permitted capability later.

2. The counters form two levels of aggregation

task fields, PID flag, thread internal count, thread user counts, scheduler stop.

place_task_hold first rejects an inactive non-corpse task; corpse tasks return success without incrementing these counters. For a live task it increments the legacy subset when appropriate, then increments user_stop_count. Only the zero-to-one transition calls task_hold_locked. Thus three task user stops normally contribute one internal task hold. user-stop aggregation.

task_hold_locked separately increments task.suspend_count. Only its zero-to-one transition notifies the BSD workqueue and visits every member thread, taking each thread mutex and calling thread_hold. Conversely, only the one-to-zero transition in task_release_locked resumes the workqueue and drops those thread holds. Independent kernel holds survive ordinary user resumes. task hold, task release.

Mermaid source

flowchart TD
    U["Task normal, legacy and PID user stops"] -->|"first user stop adds one"| T["task.suspend_count: internal task holds"]
    K["Independent kernel task holds"] -->|"each adds one"| T
    T -->|"task count changes zero to one"| H["One task-wide hold per member thread"]
    D["Direct thread user stops"] -->|"first thread user stop"| H2["One thread-local hold"]
    H --> S["thread.suspend_count"]
    H2 --> S
    S -->|"thread count changes zero to one"| A["Request AST_APC"]
    A --> P["Target parks before user return if still suspended"]

The diagram shows contributions, not an arithmetic sum of all user counters into every thread. Illustrative normal sequence: two task_suspend2 calls make task.user_stop_count = 2, but only one internal task hold reaches each thread. A direct thread_suspend on one member gives that thread a second internal hold. Returning both task tokens removes the task-wide hold; that member remains suspended until its direct thread hold is released.

Thread creation closes the membership race. The ordinary suspended-thread constructor sets one legacy thread user stop and calls thread_hold, then adds another hold if the parent task is suspended. A running-thread constructor has no initial user stop but still adds the parent's task hold. Creation and publication occur with the task/global membership locks held. suspended constructor, running constructor, constructor locking.

3. A hold requests APC; waiting observes the CPU boundary

Under the thread mutex, thread_hold increments the internal counter. The first hold sets AST_APC using the thread scheduling lock at scheduler interrupt level. The setter propagates locally or requests an AST check when the target is running remotely. This is deferred execution on the target, not execution of its suspension callback by the requesting thread. hold, APC publication.

place_task_hold calls task_wait_locked(task, FALSE) on the first user stop. That walks member threads while holding the task lock and calls thread_wait for everyone except the current thread. Direct thread_suspend likewise waits for a non-current target after dropping its mutex. A self-suspending caller must complete the kernel-side request and reach its own APC; it cannot synchronously wait for itself to stop. task iteration, thread caller.

thread_wait(..., FALSE) waits until the target is no longer on a CPU; TRUE also waits for TH_RUN to clear. Its own wait is uninterruptible and uses wake_active under the wake and scheduling locks. The FALSE result does not promise that the target cannot later run kernel code: an existing kernel wait may wake, and APC still prevents user return while the hold survives. This distinction explains the source's “stop executing user code” contract without inventing a permanently frozen kernel stack.

Source: osfmk/kern/sched_prim.c, lines 1649–1668.

	/*
	 * Wait until not running on a CPU.  If stronger requirement
	 * desired, wait until not runnable.  Assumption: if thread is
	 * on CPU, then TH_RUN is set, so we're not waiting in any case
	 * where the original, pure "TH_RUN" check would have let us
	 * finish.
	 */
	while ((oncpu = thread_isoncpu(thread)) ||
	    (until_not_runnable && (thread->state & TH_RUN))) {
		if (oncpu) {
			assert(thread->state & TH_RUN);
			processor = thread->chosen_processor;
			cause_ast_check(processor);
		}

		thread->wake_active = TRUE;
		thread_unlock(thread);

		wresult = assert_wait(&thread->wake_active, THREAD_UNINT);
		wake_unlock(thread);

The separate thread_stop/thread_unstop pair arbitrates stop requests with TH_SUSP. It can be interrupted while waiting, and calls thread_unstop on its interrupted post-acquisition path. Its return is unlocked; its own comment requires an external stop mechanism to keep the thread from becoming runnable again. Neither TH_SUSP nor a completed thread_wait replaces a counted hold. stop and unwind, unstop.

Parking and resuming

The existing APC consumer section follows the complete continuation path. Its decisions are significant here:

1. Clear abort scheduling flags under the scheduling lock.
2. If thread.active is false, terminate the thread before considering suspension.
3. If suspend_count > 0, set suspend_parked, assert an abort-safe wait on the counter, drop the mutex and block with thread_suspended as continuation.
4. On wake, the continuation rechecks the counter and rearms APC if another hold remains, then returns through the exception-return path.

consumer and continuation.

thread_release must run with the mutex held. Reaching zero either starts a never-started thread or clears suspend_parked and wakes that specific thread. Since the wait is abort-safe, an abort can have already removed it from the wait queue; the marker is protocol state, not a queue-membership test.

Source: osfmk/kern/thread_act.c, lines 437–443.

	if (--thread->suspend_count == 0) {
		if (!thread->started) {
			thread_start(thread);
		} else if (thread->suspend_parked) {
			thread->suspend_parked = FALSE;
			thread_wakeup_thread(&thread->suspend_count, thread);
		}

4. Legacy task suspension tracks hidden ordinary send rights

task_suspend first places a TASK_HOLD_LEGACY user stop. It then takes the task IPC lock, lazily creates or makes a send right to task.itk_resume of type IKOT_TASK_RESUME, and copies that ordinary right into the calling task's IPC space. The returned name is intentionally not exposed by this API. Repeated copyout can merge ordinary send rights into a name with additional user references; it does not create unique one-shot token names. legacy suspend, ordinary-right reverse lookup.

The older introductory comment says all legacy rights account for “a single suspension.” The bodies actually increment legacy_stop_count and user_stop_count for each call, aggregate those user stops into one internal task hold, and remove all remaining legacy counts when the last ordinary sender disappears. Do not use that comment to collapse the two counters. increment, aggregate cleanup.

task_resume first calls release_task_hold(TASK_HOLD_LEGACY) under the task lock. It then takes the task IPC lock and the caller's IPC-space write lock, reverse-looks up the hidden resume right, and drops one user reference on successful resume. On failed resume it destroys the found right; if no local right exists but the release succeeded, it prints an out-of-band-resume warning. The counter release therefore is not conditioned on proving that this caller owns a matching hidden right. legacy resume.

A further compatibility distinction is visible in the release helper: legacy mode decrements legacy_stop_count only if positive, but can still decrement user_stop_count when the PID floor permits it. This is weaker matching than the thread legacy-resume helper, which explicitly requires a positive legacy count. Do not promise that mixing the task APIs gives ownership isolation between legacy and token holds.

Source: osfmk/kern/task.c, lines 4097–4107.

		} else {
			if (mode == TASK_HOLD_LEGACY && task->legacy_stop_count > 0) {
				task->legacy_stop_count--;
			}
			if (--task->user_stop_count == 0) {
				release = TRUE;
			}
		}
	} else {
		return KERN_FAILURE;
	}

The resume port's no-senders callback converts the association to a temporary task reference, takes the task lock, checks the notification's make-send generation, and releases TASK_HOLD_LEGACY_ALL. It drops the temporary task reference afterward. It is aggregate ordinary-send cleanup, separate from abandoning a task suspend2 token. registration, callback.

Copyout failure is not a per-call counter rollback

A legacy hold already exists when hidden-right copyout begins. On copyout failure, task_suspend logs and returns the error; it does not directly undo its increment. ipc_object_copyout destroys the supplied right on failure. That may trigger aggregate no-senders cleanup if it removes the last ordinary sender, but this is not an unconditional rollback of exactly the failed call's count. If other ordinary send rights remain, the last-sender condition has not yet occurred. failure reporting, copyout cleanup, ordinary-right destruction.

5. Task suspend2 carries one hold in a send-once right

task_suspend2_grp performs a normal user stop and returns a newly referenced task pointer as its kernel representation of task_suspension_token_t. Failure returns a null token. The MIG output translator converts that referenced pointer into the wire-visible right and consumes the reference. The token is not a separately allocated identity object. suspend and reference, reference deallocator.

Source: osfmk/mach/mach_types.defs, lines 655–660.

type task_suspension_token_t = mach_port_move_send_once_t
#if	KERNEL_SERVER
		intran: task_suspension_token_t convert_port_to_task_suspension_token_mig(mach_port_t)
		outtran: mach_port_t convert_task_suspension_token_to_port_mig(task_suspension_token_t)
#endif	/* KERNEL_SERVER */
		;

convert_task_suspension_token_to_port_grp locks the task, tests active, then takes its IPC lock and creates itk_resume if needed. It makes one send-once right and releases both locks; whether it obtained a right or found an inactive task, it consumes the supplied task reference. The output token therefore keeps the port/right alive; it is not a permanent additional task reference after conversion. A task that dies between suspension and output conversion can yield IP_NULL. output converter.

Mermaid source

flowchart TD
    S["task_suspend2: normal user stop"] --> R["Referenced task pointer returned to MIG"]
    R --> O["Output conversion consumes reference"]
    O -->|"active task"| T["Unique send-once right to IKOT_TASK_RESUME"]
    O -->|"inactive task"| N["No usable output right"]
    T -->|"MOVE_SEND_ONCE request"| M["task_resume2 input conversion takes temporary task reference"]
    M --> E["Release one normal stop and consume task reference"]
    T -->|"right abandoned or holder space destroyed"| A["Send-once disposal callback"]
    A --> V["Resolve live association and take temporary reference"]
    V --> E

On explicit task_resume2, the request transfers the send-once destination right. The token input converter validates IKOT_TASK_RESUME under the port lock and takes a temporary task reference. task_resume2_grp releases one normal hold and consumes that reference, including when release fails. The .defs token type supplies input/output translators but no generic destructor; the implementation's explicit reference consumption matters. input converter, resume consumption.

Abandoning the right follows another path. ipc_notify_send_once_and_unlock recognizes a kernel-space destination; ipc_kobject_notify_send_once_and_unlock decrements the send-once right count, unlocks while retaining a port reference, directly calls task_suspension_send_once, then releases the port. This does not queue an ordinary user notification message to an application server. The callback converts the remaining association, releases one normal hold under the task lock and drops its temporary task reference. notification dispatch, kernel callback dispatch.

Source: osfmk/kern/task.c, lines 4396–4409.

task_suspension_send_once(ipc_port_t port)
{
	task_t task = convert_port_to_task_suspension_token(port);

	if (task == TASK_NULL || task == kernel_task) {
		return; /* nothing to do */
	}

	/* release the hold held by this specific send-once right */
	task_lock(task);
	release_task_hold(task, TASK_HOLD_NORMAL);
	task_unlock(task);

	task_suspension_token_deallocate(task);         /* drop token reference */

For two correctly paired task suspend2 calls, dropping one send-once token releases one user stop; the other remains. Unlike ordinary send rights, send-once copyout does not reverse-merge into an existing name. Using a token once consumes that right, so reusing the old name is an IPC destination error rather than a second authorization to release the same hold. send-once namespace allocation, double-resume test.

6. PID suspension is a distinguished hold with policy and notifications

pid_suspend and pid_resume are BSD entry points. They reject PID zero, find the process, accept the POSIX access predicate or the process-resume/suspend entitlement, and apply the selected MAC check. On macOS, a non-root foreign request can also require the task-access-server upcall. Their process-reference lifetime is the one in these handlers; do not copy the separate task_for_pid release-before-upcall sequence into their explanation. PID suspend admission, PID resume admission.

Under the task lock, task_pidsuspend_locked rejects an already PID-suspended task, sets pidsuspended, and places the distinguished hold; failure clears the flag. This is a single boolean-owned contribution, not arbitrary nested PID suspension. Ordinary releases require user_stop_count > (pidsuspended ? 1 : 0), preserving that last PID hold. PID resume first requires and clears the flag, then releases the corresponding count. PID placement, release floor.

After successful suspension and when message_app_suspended is set, the kernel notifies IOKit and invokes deferred VM reclamation. Successful resume notifies IOKit under the same message_app_suspended condition. CONFIG_FREEZE adds serialization against changing_freeze_state: wait uninterruptibly with the task lock released, claim the transition, release the PID hold, clear frozen on success, then wake transition waiters. suspend notifications, resume/freeze coordination.

The BSD wrapper adds memorystatus hooks and, on the selected non-macOS socket configuration, socket resume. Those hooks are not one atomic transaction with the hold change: memorystatus_on_resume precedes task_pidresume. Return translation distinguishes invalid arguments; the wrapper also has a memory-error-to-SIGKILL/EIO branch, without proving that the current task_pidresume body can produce that result. resume wrapper.

Signal job-control state is another distinction. Signal paths call internal task suspend/resume helpers while updating SSTOP, SRUN, and signal bookkeeping. This does not make every SIGCONT an unconditional release of all Mach or PID suspension state. The signal action, tracing and process-state branches determine which internal release occurs. debugger stop, selected continue branches, signal delivery.

7. Direct thread holds and the different thread token protocol

thread_suspend_internal requires the thread mutex, rejects inactive targets with KERN_TERMINATED, increments user_stop_count, and takes one internal hold only on the first user stop. Legacy mode also increments legacy_user_stop_count. thread_resume_internal rejects zero user stops; legacy mode additionally rejects zero legacy stops. Only the final user-stop removal releases the internal hold. thread count operations.

The public legacy wrappers reject null and kernel-task threads. Suspend waits for a non-current thread to leave the CPU; resume only updates the counters. Despite the old comment above thread_suspend describing hidden send rights and unique send-once tokens, this body creates no resume port or right. Its legacy hold is not automatically coupled to the suspender's IPC-space lifetime by this routine. comment and body.

thread_suspend2 does implement a token, but with ordinary send rights:

1. Preallocate an IKOT_THREAD_RESUME port before taking the thread mutex.
2. Place a normal thread user stop. On failure, unlock and deallocate the unused candidate port.
3. If IPC remains active, publish the candidate only if thread_resume_port is still null, then make a send right under the mutex. This serializes make-send generation changes with cleanup.
4. Unlock, destroy any unused preallocation, wait for a non-current target to leave the CPU, and return the right.

thread token creation. The inactive-IPC branch can leave a null returned right after a successful internal hold; it should not be described as an unconditional token-publication guarantee across termination.

thread_resume2 resolves the port association under the port lock, takes a thread reference, releases one normal thread stop under its mutex and drops the reference. The MIG parameter is move-send, consuming a user reference to the ordinary send right on the request path. resume, wire contract.

Abandoning one ordinary token while other senders remain does not itself release one hold. When all ordinary senders disappear, thread_suspension_no_senders references the thread, takes its mutex, validates the make-send generation, and selects THREAD_SUSPEND_ALL. That sets user_stop_count back to legacy_user_stop_count, removing all remaining nonlegacy stops; it sets TH_SFLAG_AUTO_RESUMED if no user stops remain. Concurrent token creation increments the generation under that same mutex, so stale cleanup cannot remove its newly published hold. callback, all-token release.

This differs from task tokens at both boundaries: multiple thread tokens can be user references to an ordinary send-right name, and automatic cleanup is aggregate. The thread token tests also check that a legacy thread_resume cannot release a stop created only by thread_suspend2.

8. Aborting a wait does not release a suspension

thread_abort takes the thread mutex, rejects inactivity, invokes act_abort, and tries clear_wait(..., THREAD_INTERRUPTED). act_abort sets TH_SFLAG_ABORT, requests APC and cancels priority depression; if an abort was already pending it clears TH_SFLAG_ABORTSAFELY, upgrading it to an unrestricted interruptible abort. The caller does not propagate clear_wait's result. Thus successful thread_abort means the request was accepted, not that the current wait was necessarily broken. abort request.

In particular, clear_wait_internal refuses a THREAD_INTERRUPTED result for TH_UNINT. Aborting a thread does not forcibly unwind an uninterruptible kernel operation, release its locks, or undo side effects already performed by its syscall. uninterruptible rejection.

thread_abort_safely first tries to interrupt an existing safe wait. If the target is not at a safe point or the clear attempt fails, it records TH_SFLAG_ABORT | TH_SFLAG_ABORTSAFELY and requests APC unless a stronger abort is already pending. It returns success for an active target after queuing that request. The .defs comment says an unsafe immediate abort fails; this current body instead defers it.

Source: osfmk/kern/thread_act.c, lines 776–789.

	if (thread->active) {
		spl_t           s = splsched();

		thread_lock(thread);
		if (!thread->at_safe_point ||
		    clear_wait_internal(thread, THREAD_INTERRUPTED) != KERN_SUCCESS) {
			if (!(thread->sched_flags & TH_SFLAG_ABORT)) {
				thread->sched_flags |= TH_SFLAG_ABORTED_MASK;
				thread_set_apc_ast_locked(thread);
				thread_depress_abort_locked(thread);
			}
		}
		thread_unlock(thread);
		splx(s);

thread_mark_wait_locked applies the thread's interruptibility mask before setting at_safe_point. An uninterruptible wait, or an ordinary interruptible wait with only a safe-abort request pending, can still be entered. An eligible abort-safe wait instead returns THREAD_INTERRUPTED without joining the wait state and consumes the safe-abort flags. “Safe” is this explicit scheduler/wait protocol, not a proof that arbitrary application operations are transactional. effective interruptibility and flags.

A suspension APC uses an abort-safe wait. Aborting it may wake the thread, but the thread_suspended continuation sees the remaining suspend_count and rearms APC. Abort is therefore not resume. Conversely, ordinary resume removes a hold but does not cancel unrelated waits on sockets, locks or events.

9. Termination disconnects capabilities and wins over parking

thread_terminate_internal marks the thread inactive under the mutex, requests abort, interrupts an eligible existing wait or starts a never-started thread, then waits off-CPU for a foreign target. The APC consumer checks inactivity before suspension, so remaining holds do not trap a terminating thread permanently in its suspension park. Kernel threads require cooperative self-termination; that path explicitly invokes APC because there is no user-return loop. termination request, staged teardown.

Task IPC disable disconnects IKOT_TASK_RESUME from the task. Token callbacks that can no longer resolve the association do nothing to the task. Later termination deallocates the resume port; outstanding rights must not be treated as a way to resurrect the old task. Thread IPC disable similarly disconnects its resume port. Distinguish the deactivated association, outstanding right storage, task/thread references temporarily acquired by conversion, and the counted hold itself. task disable, task resume-port disposal, thread disable.

Corpse handling has its own qualification: task hold/release helpers return success without changing the counters for a corpse, while output token conversion requires an active task. Success from a helper alone cannot establish that a living runnable task was stopped or that a usable user token was produced. corpse short circuits, release, output activity check.

10. Configuration, tests and reading boundary

Tests were read, not executed. This chapter covers local state, reference conversion, selected entry bodies, callbacks and scheduler/APC boundaries. Generated MIG stubs and userspace task-access/MAC policy providers are not reconstructed. Complete signal, freezer and Exclave internals belong to their own chapters; follow wait protocols, AST dispatch, deferred reclaim and capability acquisition at those boundaries.