VM objects and backing storage

A vm_object gives contents an offset namespace, a lifetime and a backing policy. It does not contain the mapped bytes. This companion follows the object allocation, fields, references, pager association and eventual storage release in xnu-12377.121.6. Start with the structure guide and map and entry companion; continue to page descriptors for resident backing and memory entries and VM copies for containers that carry object references.

The existing objects and pages chapter explains copy and collapse algorithms. Here the emphasis is recognizing which pointer, offset, lock and lifetime obligation a source expression uses.

Inspect the live vm_object layout for the selected source profile.

The object is metadata, not its contents

The central definition is struct vm_object. Its memq links resident descriptors; its pager identifies a provider; its shadow can lead to another object's contents. Neither resident pages nor a pager must exist when the object is created. A map entry can also defer creating an anonymous object until it is needed; follow that map-side publication in maps and entries.

Mermaid source

flowchart TD
    E["Map entry or other owner"] -->|"owns an object reference"| O["vm_object O
lock, counts, offset namespace"]
    O -->|"memq membership"| P["Resident vm_page descriptors"]
    P -->|"physical page identity"| B["Physical bytes"]
    O -->|"pager reference"| G["memory_object provider
operations and backing metadata"]
    G -.->|"mo_control"| A["Borrowed association<br/>back to object O"]
    O -->|"shadow reference"| S["Lower vm_object
separate offset namespace"]

Solid arrows are ownership or membership relationships with the labels shown; the dotted pager backlink is an association, not an extra ordinary VM-object reference. The shadow arrow describes ordinary copy objects; pageout objects have a different hold convention described below. A resident descriptor's object pointer likewise does not independently increment ref_count each time a page is inserted. See insertion accounting and ordinary reference helpers.

Fields whose interpretation changes with the object

Sources: union and page fields, wired/accounting and hints, cache union exclusion, page insertion, lookup, copy setter.

The object has an ordinary read/write lock. vm_object_lock takes it exclusively; vm_object_lock_shared takes it shared. vm_object_unlock also asserts that delayed page-hash insertion has completed. An accessor that returns a pointer or computes an offset does not acquire this lock for its caller.

Allocation and initialization: writable, aligned, initially empty

Ordinary objects come from the writable vm objects zone, not a proc_ro-style RO zone and not a kalloc_type call site. The zone's element size is the configured sizeof(struct vm_object) rounded to VM_PAGE_PACKED_PTR_ALIGNMENT (64 bytes). ZC_ALIGNMENT_REQUIRED matters because packed resident-page links can point back to memq, which must remain the first field. Configuration-dependent fields make a single universal byte-size claim inappropriate. See layout/alignment requirement and zone construction.

osfmk/vm/vm_object.c lines 565–569:

	vm_object_size = (sizeof(struct vm_object) + (VM_PAGE_PACKED_PTR_ALIGNMENT - 1)) &
	    ~(VM_PAGE_PACKED_PTR_ALIGNMENT - 1);

	vm_object_zone = zone_create("vm objects", vm_object_size,
	    ZC_NOENCRYPT | ZC_ALIGNMENT_REQUIRED | ZC_VM);

vm_object_allocate uses Z_WAITOK | Z_NOFAIL, then calls _vm_object_allocate. It can wait for storage; this body does not implement an ordinary nullable allocation-failure return. Initialization copies a constant template, initializes the queue and lock, records provenance, rounds the size, and creates the initial ordinary reference:

osfmk/vm/vm_object.c lines 517–527:

	*object = vm_object_template;
	object->vmo_provenance = provenance;

	vm_page_queue_init(&object->memq);
#if UPL_DEBUG || CONFIG_IOSCHED
	queue_init(&object->uplq);
#endif
	vm_object_lock_init(object);
	vm_object_set_size(object, size, size);

	os_ref_init_raw(&object->ref_count, &vm_object_refgrp);

The template starts with no resident pages, no pager, no shadow/copy, no paging/activity holds, internal = TRUE, alive = TRUE, and symmetric copy strategy. vm_object_set_size rounds to object-page granularity; under KASAN it also preserves an inner-size delta. None of these steps allocates or zeroes the object's content pages.

The same initializer also handles statically allocated, explicitly aligned kernel, compressor, retired-page and other special objects. Their storage does not originate in the dynamic zone. Bootstrap changes their copy strategy to MEMORY_OBJECT_COPY_NONE; HAS_MTE adds tagged kernel/tag-storage variants and corresponding cache attributes. Do not confuse the global compressor_object, whose resident pages hold compressor storage, with an ordinary anonymous object's separate compressor pager. See static stores and bootstrap variants.

Several variants share the same structure

Sources: external setup, compressor setup, private population, UPL map object, special-object deallocation.

References, paging holds and activity holds are different ledgers

An ordinary reference keeps an owner attached to the object. vm_object_reference_locked requires exclusive locking; vm_object_reference_shared performs an atomic retain under the shared lock; vm_object_reference acquires that shared lock around the retain. These functions presuppose that the caller already has a valid object pointer. They are not a stale-address validation mechanism. See the three helpers.

Sources: counter widths, page-list request helpers, activity and paging helpers, UPL hold use.

The shared completion event is signaled only after both paging and activity counts reach zero. vm_object_paging_only_wait deliberately has a narrower contract. The end of an activity is therefore observable without changing the ordinary reference count:

osfmk/vm/vm_object_internal.h lines 694–705:

vm_object_activity_end(vm_object_t object)
{
	vm_object_lock_assert_exclusive(object);
	if (os_dec_overflow(&object->activity_in_progress)) {
		panic("vm_object_activity_end(%p): underflow\n", object);
	}
	if (object->paging_in_progress == 0 &&
	    object->activity_in_progress == 0) {
		vm_object_wakeup((object),
		    VM_OBJECT_EVENT_PAGING_IN_PROGRESS);
	}
}

A last ordinary deallocation can consequently mark the object terminating and hand it to the reaper while work still holds paging/activity state. Waiting synchronously in that situation can deadlock with the operation responsible for ending the hold. The dedicated reaper waits for the holds and completes final release; this is a concrete distinction between logical termination and storage reclamation, not an assumption that ref_count == 0 immediately frees memory. See termination decision and reaper wait.

Pager association is an in-kernel pointer relationship

Inside the kernel, memory_object_control_t is struct vm_object *. A memory_object_t points to a provider header with mo_pager_ops, mo_control, and provider lifetime state. The control allocation and conversion bodies are unusually important to read because nearby historical comments still describe ports:

osfmk/vm/memory_object.c lines 1844–1864:

__private_extern__ memory_object_control_t
memory_object_control_allocate(
	vm_object_t             object)
{
	return object;
}

__private_extern__ void
memory_object_control_collapse(
	memory_object_control_t *control,
	vm_object_t             object)
{
	*control = object;
}

__private_extern__ vm_object_t
memory_object_control_to_vm_object(
	memory_object_control_t control)
{
	return control;
}

memory_object_control_reference and memory_object_control_deallocate are no-ops; memory_object_control_disable clears the supplied pointer. memory_object_to_vm_object checks the provider/control for null and returns the associated object without retaining it. These helpers are not task-local Mach-name lookup. User-visible memory-entry ports are covered separately in memory entries and VM copies.

vm_object_memory_object_associate either uses a prepared internal object or creates an external one. It retains the provider, publishes pager and pager_control under the object lock, drops the lock to call memory_object_init, then reacquires it to finish initialization. The init return is explicitly discarded in this body; do not infer a generic callback-error rollback that is absent here. Internal objects become ready here, whereas external readiness is established through provider setup. memory_object_create_named waits for that readiness before publishing its output control.

Mermaid source

sequenceDiagram
    participant C as Creator
    participant O as VM object
    participant P as Pager provider
    C->>O: Allocate or supply prepared internal object
    C->>O: Lock and retain provider
    C->>O: Set pager and control association
    C->>O: Unlock
    C->>P: memory_object_init(control, PAGE_SIZE)
    P-->>C: Callback returns
    C->>O: Relock and finish initialized state
    alt Internal object
        C->>O: Mark ready and wake waiters
    else Named external object
        C->>O: Wait for provider-established readiness
    end
    C-->>C: Publish control to caller

This combines association with the named creator's subsequent readiness wait; it does not imply that every internal creator returns a named object. Dispatch such as memory_object_data_request calls the provider's operations table directly.

A concrete external owner is UBC. ubc_info_init calls memory_object_create_named, drops its temporary vnode-pager reference, stores ui_control and records UI_HASOBJREF. This named VM-object ownership is distinct from the pointer's representation. The vnode/file-size state remains in UBC and the filesystem path; an external object's union field is not a substitute for that size. Filesystem/provider availability remains the boundary described in pagers and UPL.

Follow offsets through resident, shadow and compressed backing

For an object offset, first distinguish where content is represented from whether the content is immediately usable. A successful resident lookup may find an absent, busy or error-marked descriptor. A missing descriptor can lead to a lower shadow, pager request, decompression, zero fill or failure according to the fault operation. This diagram summarizes representations, not the complete priority/order of the fault state machine:

Mermaid source

flowchart TD
    O["Object O and object-relative offset"] -->|"resident lookup"| R["vm_page or no resident descriptor"]
    R -->|"usable resident page"| P["Physical page identity<br/>and bytes"]
    O -->|"add shadow offset"| S["Lower object and lower offset"]
    O -->|"add paging offset"| G["Pager-relative offset"]
    G -->|"compressor provider"| C["Slot metadata<br/>to compressed storage"]
    G -->|"external provider"| F["Provider-defined backing"]
    R -.->|"unusable state"| W["Caller waits,<br/>retries or fails"]

vm_page_lookup only searches resident identity. vm_object_compressor_pager_state_get is a separate query: while the object is live, internal and has a pager, it passes offset + paging_offset to compressor state lookup; otherwise it returns unknown. “No resident descriptor,” “no compressed slot” and “unknown compressed state” are distinct observations. See fault resolution.

A shadow takes over the caller's reference

In the path that really creates a shadow, vm_object_shadow allocates a new top object and moves the caller's original ownership into result->shadow. It does not retain the source and then leak the caller's old reference:

osfmk/vm/vm_object.c lines 4373–4380:

	/*
	 *	The new object shadows the source object, adding
	 *	a reference to it.  Our caller changes his reference
	 *	to point to the new object, removing a reference to
	 *	the source object.  Net result: no change of reference
	 *	count.
	 */
	result->shadow = source;

The new object gets the original source offset in vo_shadow_offset, and the caller's mapping is adjusted to the new object's offset namespace. With a 16 KiB VM page and 16 KiB map page, an original source range starting at 0x8000 can become a new object with mapping offset zero and vo_shadow_offset = 0x8000. New-object offset 0x4120 refers to lower-object offset 0xc120 when it is satisfied from the shadow. Neither value is a physical address or CPU virtual address. The implementation also handles smaller map-page alignment by preserving an intra-VM-page offset adjustment; see rounding and adjustment and output publication.

The reverse vo_copy relationship is different. It tells a source where changed contents may need to be preserved; the ordinary delayed-copy object's shadow retains the source, while the reverse pointer does not create a symmetric reference cycle. VM_OBJECT_COPY_SET updates the pointer/version but performs no retain. Copy code must acquire stable references and revalidate relationships around lock drops. Follow delayed copies and collapse/bypass for the paired locks and moved pages, rather than interpreting the two pointers as interchangeable. Setter body, delayed-copy construction.

Compressed bytes belong to a separate representation

vm_object_compressor_pager_create begins a paging hold, serializes competing creators with pager_created/pager_ready, and can drop the object lock to allocate the provider. It rejects an object whose page count does not fit the provider's 32-bit slot count; HAS_MTE has an additional tagged-pager availability branch. Once created, association retains the pager, the creator releases its temporary pager reference, and the function reacquires the object lock to end the paging hold.

The compressor_pager has its own header, lock, reference count, total/occupied slot counts and a union for embedded, direct or indirect slots. A slot is not a vm_page *, and compressed storage need not occupy the original physical frame. Resident descriptor release can therefore coexist with persistent logical contents. Read compressor segments and swapfiles for slot-to-segment storage, compaction and swap, and MTE page lifecycle for compressed tags.

Cache membership, last unmap and termination

External-object cache membership is a resident-page reclamation aid. memory_object_mark_unused reaches vm_object_cache_add; mark_used removes membership. Insertion skips empty or ledger-owned objects, sets the age and scan budget under the cache lock, and adds no ordinary reference in that body. Eviction holds the cache lock while finding a candidate and uses an object try-lock before dropping cache protection. It skips busy/wired/cleaning/laundry pages rather than treating the cache queue as ownership of immediately freeable bytes. See cache insertion and scan acquisition/skip paths.

The named object's transition from two references to one can trigger a provider memory_object_last_unmap call. vm_object_deallocate waits for mapping serialization, rechecks the pager/count/named predicate, begins mapping activity, records the callback thread, drops the object lock for the callback, and relocks/rechecks the provider afterward. The count before the callback cannot simply be reused after it. Special-object and ordinary multi-reference paths can avoid full termination; the latter use atomic decrement under shared locking when their predicates permit it. Fast paths.

Mermaid source

flowchart TD
    D["vm_object_deallocate"] -->|"last ordinary ownership remains"| R["Wait for unfinished pager setup
retain reference while waiting"]
    R -->|"attempt termination"| T["External dirty-page preparation
may drop locks and acquire new refs"]
    T -->|"state or references changed"| B["Release caller reference and return"]
    T -->|"still eligible"| M["Set terminating and clear alive
remove cache membership"]
    M -->|"paging or activity holds exist"| Q["Queue asynchronous object reaper"]
    Q -->|"reaper waits for holds"| F["vm_object_reap"]
    M -->|"no holds"| F
    F -->|"detach pager, reap pages, release provider"| Z["Destroy lock and free dynamic object storage"]

The full sequence matters:

1. A last-reference deallocator waits for a created-but-not-ready pager while still retaining that last reference. It saves the ordinary shadow pointer before calling termination. vm_object_deallocate tail.
2. For a persistent external object, termination first reaps/returns pages while it is still alive; pageout can drop locks and allow new references. It checks the termination/refcount predicates again after that work. vm_object_terminate preflight.
3. It sets terminating, clears alive, removes external cache membership, and clears a matching reverse copy link under the shadow's lock. Remaining holds lead to asynchronous reaping. A KERN_FAILURE return here is an internal “caller must stop this deallocation path” result, not necessarily failure to schedule cleanup. Transition and queueing.
4. Reaping requires zero paging/activity holds. It disowns ledger ownership before releasing compressed backing, saves and clears the pager pointer, disables control, and drops the final ordinary reference. Resident pages and purgeable/wired accounting are then drained; pageout objects have their special termination path. Reap body.
5. Provider termination/deallocation runs with the object unlocked; after relocking, reaping clears terminating state and wakes relevant waiters, unlocks, destroys the lock and frees the dynamic zone element. The caller or reaper subsequently releases the saved ordinary shadow reference. Final object release, provider release, asynchronous shadow release.

Destroying the pager association need not free the object

vm_object_destroy handles loss of the provider even while mappings/references remain. It records the no-pager reason, clears persistence/named state, detaches the pager and disables control, then waits for paging, page-list requests and relevant mapping callbacks. It avoids waiting on the current thread's own last-unmap callback. Outside the lock it releases the old provider and, when one existed, the caller's assumed reference.

The apparent alive = FALSE statement in this function is inside #if 00; it is not executed. Consequently this path must not be described as synonymous with vm_object_terminate. A surviving external object can have pager_created set but pager == NULL; cache eviction explicitly recognizes this lost-pager state and disconnects mapped pages before discarding their bytes. Disabled assignment and detach, lost-pager eviction.

Source-reading checkpoints and limits

For a concrete operation, record the object's variant, which union members are valid, who owns the ordinary reference, and whether a lock drop is protected by a paging/activity hold or by stable ownership elsewhere. Then follow each offset conversion separately: map VA to object offset, shadow offset addition, provider offset addition, and resident descriptor to physical page number. Follow the page companion before interpreting resident removal as reusable physical memory, and the pmap companion before interpreting object lifetime as translation lifetime.

This chapter reviews the local object/control/page interfaces and selected external/compressor/UPL paths. It does not reconstruct missing filesystem, device, secure-monitor or hardware-provider implementations. HAS_MTE, KASAN, object tracking and other configuration fields change the representation and selected branches. The exact source citations describe this snapshot; neither the structure layout nor the historical port terminology is a promise about a different XNU release.