MPTCP sessions and TCP subflows

MPTCP presents one application byte stream while using several internal TCP connections. Each TCP connection orders and acknowledges its own bytes; the MPTCP session maps those bytes into a separate connection-wide sequence space, reassembles them, and retains data until the connection-level acknowledgment advances. This chapter follows the local implementation in xnu-12377.121.6, including subflow publication, selected scheduling and reinjection, fallback, and deferred destruction.

Start with the networking guide, policy extensions, TCP connections and timers, and TCP recovery. NECP clients and flow registration explains the policy objects referenced here. The examples describe source behavior, not an observed network exchange or a promise about device configuration.

1. One application socket, several transport identities

The PF_MULTIPATH domain registers a SOCK_STREAM, IPPROTO_TCP protocol whose callbacks are MPTCP operations. It is distinct from an ordinary IPv4/IPv6 TCP socket. Its user-request table supplies connectx, send, receive notification, shutdown and detach operations. The selected implementation is an active-open client: the MPTCP state enum labels LISTEN as unimplemented, and this user-request table supplies no MPTCP listen/accept implementation. Protocol initialization fills omitted accept, listen and ordinary connect slots with EOPNOTSUPP stubs; the implemented active-open operation here is connectx. Domain registration, user requests, state definitions, callback defaults, unsupported operations, listen stub, initialization call.

The multipath PCB, session and MPTCP control block occupy one mpp_mtp allocation. Session creation initializes the backlinks, lists and closed state. Initial association/last-connection IDs are ANY; the first connected event later chooses the association ID from that subflow's connection ID. Aggregate layout, session initialization, subflow fields, association assignment.

Mermaid source

flowchart TD
    A["Application fd"] -->|"socket operations"| S["MPTCP socket: stream send and receive buffers"]
    S -->|"so_pcb"| P["mppcb: shared mutex and work flags"]
    P -->|"mpp_pcbe"| E["mptses: subflows and reinjection queue"]
    E -->|"mpte_mptcb"| C["mptcb: DSNs and connection reassembly"]
    E -->|"list owns references"| X["mptsub A"]
    E -->|"list owns references"| Y["mptsub B"]
    X -->|"mpts_socket"| TA["Internal socket A / inpcb / tcpcb"]
    Y -->|"mpts_socket"| TB["Internal socket B / inpcb / tcpcb"]
    TA -->|"t_mptcb"| C
    TB -->|"t_mptcb"| C
    TA -->|"t_mpsub"| X
    TB -->|"t_mpsub"| Y

Each TCP control block points back to its own mptsub and to the shared mptcb. Internal subflow sockets start without an fd reference. An address tuple belongs to that subflow's Internet PCB, not to the application socket's multipath PCB. Attachment, internal socket creation.

2. Connect admission precedes several kinds of publication

mptcp_usr_connectx() rejects a missing/dead PCB, fallback mode, unsupported address families and incorrect IPv4/IPv6 sockaddr sizes. It performs the service-type entitlement check once and records initial source/destination addresses before calling mptcp_subflow_add(). That check has extended-privilege/delegation and developer-mode branches; aggregation is specifically restricted by its body. It does not prove that every interface or later policy decision permits traffic. Connect callback, service admission.

With the parent socket locked, mptcp_subflow_add() rejects states at or after CLOSE_WAIT, checks the subflow count, copies endpoint information, validates the scope against the interface-index bound, and creates the TCP socket. The count predicate is literally mpte_numflows > MPTCP_MAX_NUM_SUBFLOWS, with the constant defined as 256; it is not a >= 256 admission test. Checking an index bound also does not establish current connectivity. Admission and copied addresses, count constant.

Creation has an ownership boundary. Before mptcp_subflow_socreate(), the new mptsub is private and can be freed directly. That callee takes ownership: it either frees an unattached record or attaches it and uses abort/event cleanup on later errors. Once attached, deletion must use mptcp_subflow_del(). Attachment links the session list, increments its count, takes two mptsub references—list and socket—and increments the parent socket usecount. These changes precede TCP connection establishment. Ownership handoff, attachment references, creation error exits, post-attachment abort.

The add path then assigns a local connection ID, initializes mpts_rel_seq = 1, and installs write/control upcalls. The initial subflow initializes the local MPTCP key/token/initial DSN and marks the parent connecting. Further subflows can exist in the list with CONNECT_PENDING until the connection becomes join-ready. The actual conditional call and ID publication are visible together. Source: mptcp_subr.c, lines 2680–2695.

	mpts->mpts_flags |= MPTSF_CONNECTING;

	/* connect right away if first attempt, or if join can be done now */
	if (!(mpts->mpts_flags & MPTSF_CONNECT_PENDING)) {
		error = mptcp_subflow_soconnectx(mpte, mpts);
	}

	if (error) {
		goto out_err_close;
	}

	if (pcid) {
		*pcid = mpts->mpts_connid;
	}

	return 0;

Returning a connection ID here therefore does not establish a completed handshake. mptcp_subflow_soconnectx() clears pending state, supplies initial/join authentication state, calls the ordinary socket connect machinery, and saves the TCP initial send sequence. The initial address ID is zero; secondary subflows have separate join flags and authentication records. IDs and pending decision, TCP connect, initial versus join setup.

The connected event derives MP_CAPABLE from TCP's negotiated flag. On the first connection it sets the MPTCP state established and calls soisconnected(mp_so), including the ordinary-TCP fallback case. A later status event converts MPTSF_MP_READY into connection-wide MPTCPF_JOIN_READY; the workloop then attempts queued joins. A secondary connection that failed MPTCP negotiation is reset instead of becoming another ordinary TCP carrier. Connected event, ready/fallback publication, pending joins.

The connect-with-data callback also drops the parent lock around pru_sosend, reports the consumed uio amount on success/EWOULDBLOCK, and translates the latter to EINPROGRESS. Its error variable is overwritten by that send when an auio is present. This path must be read as staged connect/send progress, not an atomic connect-plus-data transaction. Combined operation.

3. Lock switching and deferred callbacks protect the session

After attachment, a subflow's tcp_lock() routes to the parent MPTCP socket lock. It checks SOF_MP_SUBFLOW again after acquiring the selected lock and retries if attachment/detachment changed which mutex applies. The parent callbacks use mpp_lock; requested reference accounting changes the parent usecount and mpp_inside. Thus separate TCP PCBs do not imply independent simultaneous mutation of this session. TCP lock selection and retry, parent lock/refcount callbacks.

Two creation steps deliberately release the parent lock:

Socket creation and lock switch, NECP registration and revalidation. This explicit recheck belongs to that NECP branch; it should not be inferred after every unlocked call.

Input/output can themselves cause socket upcalls. MPP_INSIDE_INPUT, MPP_INSIDE_OUTPUT, MPP_INPUT_HANDLE and MPP_WUPCALL defer recursive work, recording pending wakeups/workloop execution instead. The eventual helper clears the current guard and services deferred work when no other guard remains. The event workloop has its own IN_WORKLOOP/WORKLOOP_RELAUNCH guard and temporarily retains both subflow record and socket while dispatching events. Defer predicate, deferred dispatch, event lifetime and relaunch, loop exit.

4. Assign DSNs before choosing a TCP carrier

mptcp_usr_send() accepts connected or permitted preconnect data, rejects unsupported destination/OOB/control forms, calls mptcp_insert_dsn(), and appends to the parent send buffer. Every participating mbuf carries packet-header metadata. SB_NOCOMPRESS prevents socket-buffer compression from erasing those boundaries. After append the callback sets its local mbuf pointer to null: later output/wait errors do not return ownership of already queued data to the caller. Send ownership, buffer setup.

The enqueue-time operation is exact. Source: mptcp_subr.c, lines 5437–5445.

	while (m) {
		VERIFY(m->m_flags & M_PKTHDR);
		m->m_pkthdr.pkt_flags |= (PKTF_MPTCP | PKTF_MPSO);
		m->m_pkthdr.mp_dsn = mp_tp->mpt_sndmax;
		VERIFY(m_pktlen(m) >= 0 && m_pktlen(m) < UINT16_MAX);
		m->m_pkthdr.mp_rlen = (uint16_t)m_pktlen(m);
		mp_tp->mpt_sndmax += m_pktlen(m);
		m = m->m_next;
	}

Despite the field comment calling mpt_sndmax the maximum byte sent, this body advances it when application data is numbered and queued. For this path it is the right edge of assigned data, while mpt_sndnxt tracks progress through selected output and mpt_snduna is the oldest data not cumulatively acknowledged at the MPTCP level. Do not substitute the TCP meaning of snd_max for this enqueue behavior. Initially these three DSNs equal mpt_local_idsn + 1. Sequence fields, initial numbers.

An outgoing mapping connects three quantities:

In the ordinary, non-TFO example, a mapping (D, R, L) relates subflow TCP sequence ISS + R + i to DSN D + i for 0 <= i < L. The arithmetic deliberately avoids wraparound. TFO uses additional saved-ISS/rewind handling and is not modeled by this example. Subflow sequence fields, mapping lookup, receive use of relative sequence plus IRS.

mptcp_subflow_output() chooses from the reinjection queue first, otherwise the parent buffer at the connection output offset. It caps the transfer by connection-window accounting where applicable, subflow capacity, and UINT16_MAX. It copies mbuf representations using m_copym_mode(..., M_COPYM_MUST_COPY_HDR), preserves the DSN and supplies the selected subflow's current relative sequence. Copying an mbuf representation need not copy all underlying external storage. Queue/offset selection, limits and copy.

Source: mptcp_subr.c, lines 3289–3293.

		m->m_pkthdr.pkt_flags |= PKTF_MPTCP;
		m->m_pkthdr.pkt_flags &= ~PKTF_MPSO;
		m->m_pkthdr.mp_dsn = mpt_dsn;
		m->m_pkthdr.mp_rseq = mpts->mpts_rel_seq;
		m->m_pkthdr.len = mlen;

All mbufs made for that output mapping receive its total length; the subflow relative sequence advances by the total copied bytes. The internal TCP send consumes the copied chain. Successful ordinary output advances mpt_sndnxt; reinjection and probing deliberately do not. A reported send error rolls back the relative-sequence increment, while queue operations already performed must still be accounted for. The outer mptcp_output() handles subflow errors locally and returns zero rather than treating them as application-send failures. Neither return establishes packet delivery. Mapping completion and send, output error policy.

TCP output separately calls mptcp_setup_opts(). Its DSS construction retrieves mapping metadata from the subflow send buffer; ordinary TCP still supplies the packet sequence and handles packetization/retransmission. A DSS mapping can outlive a particular packet, so one mapping must not be drawn as necessarily one packet. TCP integration, DSS output lookup, mapping lookup body.

5. Select an eligible subflow, then apply the service policy

mptcp_can_send_more() admits queued reinjection first. Otherwise it checks the assigned-data edge, connection send window, pending Data FIN and closing state. mptcp_get_subflow() then chooses a carrier. Its capacity helper computes min(snd_wnd, snd_cwnd) - subflow_send_buffer_bytes, further limited by sbspace; this is not just the raw congestion-window value. Connection output predicate, capacity calculation.

The normal selection scan requires an output interface, no pending interface feedback, MPTCP-capable state, a connected socket and an established TCP state no later than CLOSE_WAIT. Degraded and preconnect cases are handled earlier as explicit exceptions. It chooses separate noncellular and cellular candidates using RTT and retransmission state, then applies the service-type branch. Eligibility, candidate comparison.

Service branches, slow-subflow predicate. A single available category has an earlier return. The body does not promise round-robin striping, universal lowest-RTT selection, or that an alternate with room will be chosen in every service mode. mpte_active_sub records the latest selected active carrier; it does not mean other subflows have no outstanding bytes. Output loop and active-carrier update.

Path creation is a separate policy operation. mptcp_check_subflows_and_add() examines interface capability/address information, current subflows, service type, Wi-Fi quality and cellular permission; later branches can request a Symptoms decision before creation. NECP callbacks and the privileged Symptoms control channel provide inputs. The local kernel receiver is present, while the external adviser's policy and actual interfaces/peer behavior are outside this trace. Creation policy, advisory receiver and registration.

6. Reinjection preserves the connection range

A transmit-failure hint enters the subflow event path through mptcp_act_on_txfail(). mptcp_subflow_failover_ev() first copies eligible outstanding mappings to mpte_reinjectq, then looks for an alternate usable subflow. If none qualifies, the hint does not manufacture one. Hint production, failover event.

mptcp_reinject_mbufs() skips mappings already marked reinjected or entirely covered by the connection's Data ACK. If TCP acknowledgment has consumed part of a mapping, it can recover the data from the parent send buffer by DSN. It copies the mapping, inserts by DSN with coverage checks, and marks the original mapping's mbufs so the same scan does not continually enqueue them. Allocation failure stops this pass. This is distinct from TCP retransmitting at its original TCP sequence. Copy and parent-buffer lookup, ordered insertion and coverage checks.

The next selected carrier receives the same DSN with its own fresh mpts_rel_seq. A candidate whose send buffer already covers the reinjection's starting DSN is avoided by the selected output branch. Reinjection queue records are trimmed/removed as that output prepares data, and fully Data-ACK-covered queue entries are removed by cleanup. The original TCP copy can still exist and undergo TCP recovery; reinjection is not an ownership transfer that erases the original transmission. Duplicate-carrier check, reinjection selection, queue consumption and sequence update, acknowledged cleanup.

There is also a connection retransmission timer for missing Data-ACK progress. Its selected timeout branch compares mpt_snduna with the tracked DSN, rewinds mpt_sndnxt and calls output while retrying, or records a soft timeout error after its retry limit. This is separate from each TCP subflow's retransmission timers. The timer source supplies local behavior; no network timing or complete retry-policy validation was performed here. Connection timer, tracked starting sequence.

7. Receive mappings feed a second reassembly layer

TCP option processing decodes DSS variants into the subflow's mpts_rcv_map. A zero-length mapping takes the infinite-mapping/fallback notification path; a finite one records DSN, relative sequence, length and checksum. Before TCP trimming/reassembly can obscure the option, mptcp_insert_rmap() copies the metadata into the packet header. The TCP input call site is explicit. DSS receive state, metadata embedding, TCP call site.

The custom subflow receive callback extracts data from the TCP receive buffer without a user uio. It obeys socket-buffer serialization, waits by returning EWOULDBLOCK when a mapping or its complete bytes are not yet available, and can scan later mbufs for mapping metadata. It distinguishes early missing-DSS fallback on the initial subflow from a reset request on a secondary subflow. Receive admission and empty buffer, mapping search and absence, complete-mapping requirement.

mptcp_adj_rmap() splits an mbuf at a mapping boundary when needed and rewrites each extracted piece to DSN + offset, relative_sequence + offset, and that piece's byte length. The extraction removes subflow receive-buffer accounting; checksum validation is conditional on negotiated checksum state and precedes handing the chain upward. A split/checksum failure requests failure handling rather than delivering an unexplained byte range. Mapping adjustment, extraction and checksum, checksum gates and failure.

The input driver temporarily retains the subflow record/socket and passes the extracted chain to mptcp_input(). That function checks the connection receive window, trims or discards already-received DSNs, and either appends the next contiguous bytes to the parent receive buffer or queues them by DSN in mpt_segq. mptcp_reass() has a queue limit, allocation failure and overlap handling. mptcp_reass_present() releases consecutive entries only when the first DSN equals mpt_rcvnxt, advancing the connection boundary and waking readers. Bytes ordered by one TCP subflow can therefore remain unavailable to the application until another subflow fills a DSN gap. Input ownership, connection window/duplicate handling, queue insertion, contiguous release.

Mermaid source

flowchart TD
    T["TCP input: options and per-subflow sequence checks"] -->|"save DSS metadata"| Q["TCP reassembly / subflow receive buffer"]
    Q -->|"complete mapping available"| X["Extract, adjust mapping pieces, validate checksum"]
    X -->|"DSN and bytes"| M["mptcp_input: connection window and duplicate checks"]
    M -->|"gap or queued data"| R["mpt_segq ordered by DSN"]
    R -->|"contiguous from mpt_rcvnxt"| A["Parent receive buffer and reader wakeup"]
    M -->|"next contiguous range"| A
    A -->|"advanced mpt_rcvnxt"| D["DSS Data ACK construction"]

The diagram selects negotiated MPTCP; fallback, incomplete mappings and failure exits are described in the surrounding text. Reader wakeup means data is available in the socket, not that the application has consumed it.

8. Data ACK releases the parent send buffer

DSS ACK processing reconstructs a full Data ACK where necessary and accepts its progress only when it lies between mpt_snduna and mpt_sndmax. mptcp_data_ack_rcvd() computes the delta, drops acknowledged bytes from the parent send buffer, advances mpt_snduna, cleans reinjection entries and wakes writers. Its preceding comment says that actual sbdrop happens in the MPTCP layer; the body shown here performs that drop itself. ACK range validation, 32-bit extension, full ACK body.

Source: mptcp_opt.c, lines 1108–1113.

			sbdrop(&mp_so->so_snd, (int)mp_so->so_snd.sb_cc);
		} else {
			sbdrop(&mp_so->so_snd, (int)acked);
		}

		mp_tp->mpt_snduna += acked;

The ACK-only DSS constructors use the receiver's mpt_rcvnxt; they do not report the application's read offset. A TCP ACK acknowledges the receiving TCP subflow's sequence space. During negotiated MPTCP it does not by itself establish connection-level contiguous receipt, advance the Data ACK, or prove application delivery. The fallback path deliberately translates selected TCP buffer-release progress back into connection bookkeeping, so that exception must remain separate. DSS ACK construction, fallback translation.

9. Worked example: two subflows and one missing connection range

Select established, negotiated MPTCP with two usable subflows, no TFO, no Data FIN, no sequence wrap, adequate receive buffers and successful allocations/checksums. Use an authorized aggregate service for the initial scheduling example. Values are decimal, half-open ranges name bytes, and mapping boundaries are chosen for clarity rather than assumed packet sizes. Applying the local receiver rules to a hypothetical other endpoint illustrates ordering; it does not establish a real peer's implementation.

Initially the sender has mpt_snduna = mpt_sndnxt = mpt_sndmax = 100000. Subflow A is noncellular with TCP ISS = 1000; B is cellular with ISS = 9000; both relative send cursors start at 1. They therefore occupy the selector's two candidate categories. Enqueue 1,600 bytes. The parent retains [100000, 101600), and mpt_sndmax becomes 101600 before output. Assume A has the preferred RTT and capacity for exactly 1,000 bytes, while B has capacity for 1,000; when A's capacity is consumed, B can carry the remaining 600. The aggregate branch can select them in that order.

After successful submission, mpt_sndnxt = 101600, A's relative cursor is 1001 and B's is 601. Assume A's range is delayed/lost and B's arrives. B's receiver can acknowledge TCP sequence 9601, but connection receive-next stays 100000: the 600 bytes wait in MPTCP reassembly. A Data ACK of 100000 frees zero parent bytes; the parent still retains all 1,600 bytes. Assume previous valid DSS acknowledgment state already confirmed MPTCP, so this is not the early missing-DSS fallback case.

Now a failover event scans A's unacknowledged mapping. Assume A still has zero capacity because its 1,000 bytes remain TCP-unacknowledged, while B's previous TCP ACK restores its 1,000-byte capacity and all other eligibility conditions remain satisfied. The aggregate selector can choose B; assume the relevant copy succeeds. The reinjected mapping is (100000, 601, 1000), occupying B's TCP range [9601, 10601). Its DSN is unchanged even though its TCP range differs from A's. Neither mpt_sndmax nor ordinary mpt_sndnxt advances for this reinjection.

Mermaid source

sequenceDiagram
    participant S as Sender MPTCP
    participant A as TCP subflow A
    participant B as TCP subflow B
    participant R as Receiver MPTCP
    S->>A: DSN 100000 length 1000, relative 1
    S->>B: DSN 101000 length 600, relative 1
    B->>R: Later DSN range arrives first
    R->>R: Queue 600 bytes, receive-next remains 100000
    B-->>S: TCP ACK 9601, Data ACK still 100000
    S->>B: Reinject DSN 100000 length 1000, relative 601
    B->>R: Missing DSN range arrives
    R->>R: Append 1000 bytes, then queued 600 bytes
    R-->>S: Data ACK 101600
    S->>S: Release 1600 parent-buffer bytes

B's TCP sequence stream remains contiguous: its original 600 bytes precede its reinjected 1,000. At the connection layer those two ranges are delivered in the opposite order, according to DSN. Once the missing range is present, mpt_rcvnxt advances through 101000 to 101600; Data ACK 101600 lets the sender release 1,600 parent-buffer bytes. A later copy of A's original range is already below receive-next and is discarded at the connection layer. TCP recovery/ACK cleanup of A remains a separate obligation. None of these acknowledgments establishes that the receiving application has read the bytes.

10. Fallback changes the numbering contract used for delivery

Initial TCP establishment can succeed without MPTCP negotiation. The first connected event then requests fallback; failure of a later join requests reset instead. Missing DSS metadata before full establishment also distinguishes the initial subflow from secondary ones. These are specific local branches, not a claim that every malformed or absent option causes the same recovery. Negotiation result, missing mapping.

The status handler marks the session FALLBACK_TO_TCP, clears JOIN_READY and frees the reinjection queue. The workloop requests reset of other subflows that have not already been marked degraded, while the selected fallback flow remains the stream carrier. The receive path discards data from a degraded subflow that is not active; the nearby comment says “all but one,” but the predicate implements acceptance only from the active degraded flow. Parent input then accepts the surviving stream without ordinary DSS reassembly. Status transition, workloop cleanup, active-flow receive predicate, fallback input.

Send-side fallback has explicit synchronization flags and adjusts which parent bytes must be resent. mptcp_fallback_sbdrop() derives connection acknowledgment progress from the surviving TCP buffer's mappings, conservatively handling a drop in the middle of an mbuf. This bridges the bookkeeping after Data ACKs cease; it must not be applied to the negotiated two-subflow example above. Output synchronization, Data-ACK interaction, TCP-derived release.

11. Subflow loss, stream close and storage release are different events

A disconnected subflow clears its connection/capability flags and returns a delete result to the workloop. The handler can terminate the session before initial establishment or when the active fallback carrier dies; an ordinary established multipath subflow disappearing is not automatically the same as closing the parent stream. The workloop calls mptcp_subflow_soclose(), whose CLOSED flag prevents repeated close. The TCP PCB/socket can remain until its own references and timer/PCB conditions permit disposal. Disconnected event, workloop close, subflow close.

TCP garbage collection eventually calls mptcp_subflow_del() after the subflow is no longer searchable and its socket usecount is zero. Deletion removes the session-list entry, disables callbacks, removes its authentication record, drops the parent usecount, clears session/socket pointers and drops the two attachment references. It clears SOF_MP_SUBFLOW and TCP backlinks before the collector releases the parent lock and disposes the Internet PCB. That ordering ends parent lock routing before the underlying TCP storage disappears. GC handoff, unlink and references, final subflow free.

Closing the application stream instead enters mptcp_disconnect()/mptcp_usrclosed(). Abortive or pre-established cases close immediately at the protocol level; the graceful route drives a connection-level Data FIN state machine. Data FIN consumes one DSN and is distinct from each TCP FIN. The local-close path can wait for outstanding output, and the output loop has a further all-data-acknowledged condition before finishing shutdown. Received Data FIN advances receive-next and makes the parent unable to receive more. Disconnect decisions, shutdown helpers, output close gate, Data FIN state machine.

mptcp_close() marks the protocol terminated, frees connection reassembly, marks the parent disconnected and requests subflow disconnection. It does not free the combined PCB allocation. mp_pcbdetach() marks the multipath PCB dead and schedules GC. The MPTCP collector waits for parent usecount to reach zero, then destroys session-owned option/interface/reinjection state, removes the PCB from the global list, disposes NECP state and socket buffers, and frees storage. Session destruction asserts the subflow list is empty. Logical close, dead marking, GC and disposal, session destruction, PCB disposal.

12. Configuration and evidence limits

The selected source includes MPTCP as a build option and lists the MPTCP bodies as optional mptcp files. The multipath group appears in ARM64 iPhoneOS configuration descriptions as well as other targets; it is not a macOS-only mechanism. These files and configuration descriptions do not establish which options a shipping kernel enabled. Build option, source selection, iPhoneOS group.

Runtime enablement, cached/forced protocol versions, service-type admission, interface state and peer negotiation further select behavior. Local initialization handles versions 0 and 1 with different key-digest choices; this chapter does not replace their option/HMAC implementation with a generic protocol specification. The wire-option implementation is present, but the exact peer's implementation, network path, Symptoms policy producer and product-specific authorization are external evidence. Runtime controls, version controls, key/token/IDSN initialization, option implementation.

This review covers the active session/subflow lifecycle and selected common data paths. It does not exhaust TFO rewind, every DSS width/checksum combination, JOIN authentication, ADD_ADDR/NAT64/address selection, service-policy heuristics, urgent-progress timers, option replay, diagnostics, or all allocation/failure interleavings. Those bodies are additional local code to review, not automatically missing providers. No kernel, network or userspace test was executed; the numerical exchange is a worked source model.