xtc_batch(3)

---

xtc_batch(3)

XTC_BATCH(3) Library Functions XTC_BATCH(3)

xtc_batch_create, xtc_batch_create_async, xtc_batch_submit, xtc_batch_submit_mt, xtc_batch_cancel, xtc_batch_cancel_mt, xtc_batch_resume, xtc_batch_resume_mt, xtc_batch_close, xtc_batch_destroy, xtc_batch_get_stats, xtc_batch_get_async_stats, xtc_batch_loop_for_device, xtc_batch_router_create, xtc_batch_router_destroy, xtc_batch_route - bounded continuous batching

#include <xtc_batch.h>
int xtc_batch_create(xtc_loop_t *loop, const xtc_batch_opts_t *opts,


                    xtc_batch_t **out);
int xtc_batch_create_async(xtc_loop_t *loop,


                    const xtc_batch_async_opts_t *opts, xtc_batch_t **out);
int xtc_batch_submit(xtc_batch_t *group, void *request, uint64_t charge,


                    uint64_t *out_handle);
int xtc_batch_submit_mt(xtc_batch_t *group, void *request, uint64_t charge,


                    uint64_t *out_handle);
int xtc_batch_cancel(xtc_batch_t *group, uint64_t handle);
int xtc_batch_cancel_mt(xtc_batch_t *group, uint64_t handle);
int xtc_batch_resume(xtc_batch_t *group, uint64_t handle);
int xtc_batch_resume_mt(xtc_batch_t *group, uint64_t handle);
int xtc_batch_close(xtc_batch_t *group, enum xtc_batch_close_mode mode);
int xtc_batch_destroy(xtc_batch_t *group);
int xtc_batch_get_stats(xtc_batch_t *group, xtc_batch_stats_t *out);
int xtc_batch_get_async_stats(xtc_batch_t *group,


                    xtc_batch_async_stats_t *out);
int xtc_batch_loop_for_device(xtc_exec_t *exec,


                    const xtc_accel_dev_t *dev, int *out_loop_idx);
int xtc_batch_router_create(xtc_batch_t **groups, int n,


                    xtc_batch_router_t **out);
void xtc_batch_router_destroy(xtc_batch_router_t *router);
xtc_batch_t *xtc_batch_route(xtc_batch_router_t *router, uint64_t key);

A group owns a pinned driver process on the supplied loop. Each synchronous step callback receives one immutable vector of ready members, not one call per request. Finished members leave; waiting members refill at subsequent iteration boundaries. No scheduling policy is required. The API is available when built with --enable-sched-policy; disabled builds return XTC_E_NOSYS.

Initialize the size and version fields of options and statistics descriptors to sizeof the descriptor and XTC_BATCH_VERSION. The descriptor is copied; callback context and submitted request pointers remain borrowed. Options specify capacity, max_active, resource_capacity, sched_class, step, complete, and user. Both callbacks are mandatory. Capacity must be at least max_active, and both must be positive. Resource capacity must be positive. A NULL class selects default scheduling; a supplied class must belong to this loop and cannot be registered to another batch group. Ordinary non-batch processes may share that class. The class registration remains reserved until successful group destruction.

Create is allowed only before the loop's first run or simulation step. Setup must be externally serialized against execution. All storage is allocated before publishing the driver; create failure publishes no group. Allocation size arithmetic is checked, including on 32-bit targets.

Submit accepts a fixed maximum resource charge in 1..resource_capacity. Slots include both waiting and active requests. Admission to the active set is strict first-come, first-served: a younger request cannot bypass an older one whose charge does not fit. READY and BLOCKED requests both occupy active slots and retain their resource reservation until their outcome becomes terminal. Submission itself reserves a slot, not active resource units. Full or closed groups return XTC_E_AGAIN. Failed submission causes no callback and leaves the output handle unchanged. Request may be NULL if the application callback supports it.

Handles are nonzero, monotonically issued uint64_t identities, never reused within a group. Exhaustion returns XTC_E_RANGE rather than wrapping. Handles must be paired with the group that issued them; they are not transferable.

The step callback has signature:

int step(const xtc_batch_item_t *items, xtc_batch_result_t *results,


         size_t count, void *user);

Each item contains handle and request. Results are initialized to UNSET before each invocation. Set each state to XTC_BATCH_CONTINUE, XTC_BATCH_DONE, XTC_BATCH_BLOCKED, or XTC_BATCH_ERROR. ERROR requires an existing nonzero XTC_E_* error in its error field; other states ignore error. Omitted or invalid results fail that member with XTC_E_INTERNAL. A nonzero step return fails the entire current vector with that existing XTC_E_* error, or XTC_E_INTERNAL for an invalid error value. Neither vectors nor callback side effects are retried automatically.

The terminal callback has signature:

void complete(uint64_t handle, void *request, int outcome, void *user);
Outcome is XTC_OK for DONE, XTC_E_ABORTED for cancellation, or the recorded failure. The request borrow lasts through callback return. Completion invocation is recorded before entering user code; unexpected driver exit never starts the same notification twice. Recorded terminal outcomes, including DONE, survive unexpected driver exit. Other outstanding work fails with XTC_E_INTERNAL (or XTC_E_ABORTED if already cancelled). Such exit closes the group.

Callbacks run outside runtime bookkeeping and may yield, but must return in bounded time for progress. While any callback is in flight, ALL mutators on that group return XTC_E_DEADLK, including peer processes scheduled while the callback is suspended. Destroy is a mutator. Read-only statistics are allowed and report committed state, not partially written result words. Cancellation is masked across callbacks through result recording and completion accounting. A nonreturning or crashing callback cannot be guaranteed to finish its own application cleanup; exactly-once invocation does not imply successful application processing. Arbitrary nonlocal jumps out of the driver are unsupported.

Cancel records a request for terminal cancellation at a boundary. A cancellation recorded before the next vector excludes the request. Already recorded DONE wins over later cancellation. Stale handles return XTC_E_INVAL after terminal notification. Duplicate cancellation before notification is harmless. An owner-loop xtc_batch_cancel also accepts a live handle from xtc_batch_submit_mt that has not been admitted yet (still on the ring, or held for a slot): the request is cancelled and never runs. If an earlier hand-in on the ring is still being written by its producer, the call returns XTC_E_AGAIN; retry.

Resume changes a live BLOCKED request to READY. Duplicate resume on READY is harmless. WAITING and terminal requests are not resumable and return XTC_E_INVAL. Resume does not release or reacquire a blocked request's fixed resource charge.

xtc_batch_create_async builds a group whose step is a split submit / collect pair instead of one synchronous step, so an iteration may launch device or pool work and be awaited WITHOUT stalling the driver or the other members. The group is otherwise identical: submit, cancel, resume, close, destroy, and get_stats behave as for a synchronous group, and continuous batching's variable-length completion is preserved across the asynchronous boundary. Options carry capacity, max_active, resource_capacity, exec_timeout_ns, sched_class, submit, collect, complete, an optional cancel_hint, and user. submit, collect, and complete are mandatory. exec_timeout_ns must not be negative; 0 means no engine timeout. Initialize size and version as for the synchronous options. Submission into an async group uses the same xtc_batch_submit path and the same handle rules.

At each iteration boundary the driver builds the ready vector and calls

int submit(const xtc_batch_item_t *items, size_t count, void *user,


           xtc_batch_exec_t *out_exec);
The application launches ONE operation covering all count members and fills out_exec with a completion descriptor whose kind is one of:
The work was done inline in submit; the engine does not park and calls collect at once (lets a CPU group use the async API uniformly).
A pollable completion fence fd in out_exec->fence_fd (sync_file / dma-fence / eventfd), awaited through the same loop readiness path as xtc_accel_wait_fence. The application owns the fd; the engine never dups or closes it. The wait ends on the fence, on exec_timeout_ns, on close(XTC_BATCH_CANCEL) (status XTC_BATCH_EXEC_ABORTED), or when the driver is killed; mail sent to the driver does not end it. A bad or closed fd, or a build without accelerator support, is reported as a negative exec_status (see below), never as success. NATIVE ONLY: the simulator has no fd-readiness hook, so a FENCE group is not sim-reachable (use FUTURE for deterministic coverage).
An xtc_future_t in out_exec->future that the application resolves from any thread when its own completion fires. Pass the READ half (the future); the engine takes responsibility for consuming it exactly once, via the normal wait on completion or by discarding it on timeout or teardown. The application keeps the PROMISE half and resolves or drops it; it must NOT also wait on or free the future handle it handed over. Resolving it with a non-OK status, or dropping it, is reported to collect as that negative status. Resolving it after the group was destroyed is safe. This half-and-half split keeps the token leak-free and double-free-free across teardown. FUTURE is the sim-deterministic path.
A synchronous closure out_exec->run(out_exec->arg) run on the xtc_blocking pool; the driver parks while a pool thread runs it (coarse offload, one pool thread held). exec_timeout_ns does not apply (a pool thread cannot be interrupted), and neither close nor a kill ends the wait early. If the pool refuses the closure (xtc_blocking_run returns an error without running it), collect sees that error as a negative exec_status.

A nonzero submit return fails that iteration's members (no operation is outstanding). On XTC_OK the engine moves the ready members in flight, awaits the token, then calls

int collect(const xtc_batch_item_t *items, xtc_batch_result_t *results,


            size_t count, int exec_status, void *user);
on the driver with the SAME frozen vector and the raw exec status. collect reads device outputs and sets each result state (CONTINUE, DONE, BLOCKED, or ERROR) exactly as a synchronous step would, so one member emitting end-of-stream becomes DONE while the rest CONTINUE. collect reads only the vectors it is handed, never live group state. It runs exactly once per outstanding operation, including on teardown, so the application always learns the operation's fate. Only after XTC_BATCH_EXEC_OK is the operation known to have finished; after any other status it MAY STILL BE RUNNING on the device, and the application must keep its buffers until its own runtime reports the operation done (for example by handing them to the completion handler of the device event it already waits on).

exec_status is one of the non-negative engine sentinels XTC_BATCH_EXEC_OK (the token fired), XTC_BATCH_EXEC_TIMEOUT (exec_timeout_ns elapsed first; the engine's wait ends at that deadline even across spurious wakes), or XTC_BATCH_EXEC_ABORTED (the group was cancel-closed, or the driver killed or torn down, while the operation was outstanding), or a NEGATIVE XTC_E_* error: the engine could not await the operation at all (a bad or closed fence fd gives XTC_E_INVAL, a build without accelerator support XTC_E_NOSYS, a refused blocking closure or a failed or dropped device promise its error code, an unknown exec kind XTC_E_INVAL). A negative status fails every member of the vector with that code and counts in offload_errors.

The engine never auto-retries a side-effecting operation, and it never resubmits a member while its previous operation may still be running. After XTC_BATCH_EXEC_OK, collect decides: CONTINUE keeps a member for the next iteration, DONE or ERROR ends it. After XTC_BATCH_EXEC_TIMEOUT, DONE and ERROR are honoured, and a member left CONTINUE, BLOCKED or UNSET ends with XTC_E_AGAIN (the device may still own its buffers). After XTC_BATCH_EXEC_ABORTED every member ends XTC_E_ABORTED; after a negative status every member ends with that status.

The per-member device-output location and per-sequence cache slot are NOT new struct fields: the request pointer the application submitted is its per-sequence handle and carries both. complete(handle, request, outcome, user) firing is the application's cue to release that sequence's resources. libxtc owns no model state; the abstraction stops at the completion token.

CANCELLATION AT A BOUNDARY. A cancel (xtc_batch_cancel, xtc_batch_cancel_mt, or close(XTC_BATCH_CANCEL)) on a member whose operation is outstanding is ACCEPTED, not refused: it records the cancellation and does not interrupt the operation mid-flight. If cancel_hint is set it fires on the driver as soon as the driver observes the cancel while the operation is still outstanding (the cancel wakes the driver's wait), once per member, so the application may signal its own runtime to abort the kernel; a forced teardown fires it for every in-flight member before collect. Mutators return XTC_E_DEADLK during the hint, as during collect. The cancel is consumed at the completion boundary: collect still runs and the member then ends XTC_E_ABORTED exactly once. A close(XTC_BATCH_CANCEL) also ends a FENCE or FUTURE wait at once (status XTC_BATCH_EXEC_ABORTED); an owner or cross-loop cancel of one member does not end the wait. Cancellation is iteration-granular, never mid-kernel.

EXACTLY ONCE ACROSS TEARDOWN. Each in-flight member is finalized by exactly one call, whether the operation completed normally, timed out, was cancelled, or the driver was killed while parked on the token. A forced teardown runs collect once with XTC_BATCH_EXEC_ABORTED before finalizing, so a member is never completed twice. A kill of the driver ends a FENCE wait at once; it ends a FUTURE wait at once on builds whose process layer wakes parked fibers on kill (otherwise at the device's completion or exec_timeout_ns). A BLOCKING wait always lasts until the closure returns.

xtc_batch_get_async_stats reports offloads, offload_timeouts, offload_errors, mt_queued, and mt_rejections through a separate xtc_batch_async_stats_t (size and version checked like the synchronous statistics); the synchronous xtc_batch_stats_t is unchanged. On a synchronous group these counters are zero.

xtc_batch_submit_mt, xtc_batch_cancel_mt, and xtc_batch_resume_mt are callable from ANY loop or a plain OS thread, unlike the owner-loop-only xtc_batch_submit/cancel/resume. They are enabled by setting mt_queue_cap > 0 in xtc_batch_async_opts_t; a group created with mt_queue_cap == 0 (or a synchronous group created with xtc_batch_create, whose options cannot carry the field) returns XTC_E_NOSYS from these calls. Cross-loop admission is orthogonal to the synchronous-versus- async step: an mt-enabled async group with a plain SYNC step works the same.

mt_queue_cap bounds a lock-free MPSC admission ring and two preallocated pools of admission records (mt_queue_cap SUBMIT records and mt_queue_cap control records for cancel_mt/resume_mt), all charged through the loop's resource accountant at create. The three mt calls allocate nothing: they claim a record from their pool, stamp a handle, and push onto the ring; a full ring or exhausted pool returns XTC_E_AGAIN and counts mt_rejections. mt_queue_cap (pending hand-ins) is distinct from capacity (admitted live requests). A hand-in that waits for a slot keeps its SUBMIT record, so mt_queue_cap also bounds hand-ins accepted but not yet admitted. Held hand-ins never take a control record: a cancel_mt or resume_mt is refused only while mt_queue_cap other controls are themselves still undrained, so a group whose slots are all BLOCKED behind held hand-ins can always be resumed or cancelled from another loop.

There is no blocking submit_mt. On XTC_E_AGAIN the caller retries: a fiber yields (xtc_yield) or sleeps (xtc_proc_sleep) between attempts, a plain thread sleeps or yields its CPU. The ring empties at the driver's next iteration boundary, and records return to the pool as hand-ins are admitted, so a short back-off (tens of microseconds, growing) avoids burning a core.

HANDLE RETURNED IMMEDIATELY, ADMISSION AT A BOUNDARY. xtc_batch_submit_mt stamps a group-global monotonic handle and returns it at once, so the caller has a usable handle before the request is admitted. The driver drains the ring at its next iteration boundary and only then creates the waiting slot and subjects it to the usual capacity, resource, and FCFS admission. A hand-in that passes the ring but finds every slot taken is held, in arrival order, and takes the next freed slot before any later submission; it is never rejected after the handle is issued, and an owner-loop xtc_batch_submit returns XTC_E_AGAIN while any hand-in is held (FCFS). Every XTC_OK from xtc_batch_submit_mt is followed by exactly one complete callback. Handles from the owner-loop and mt paths share one monotonic space and are never reused; concurrent exhaustion never wraps.

A CANCEL IS NEVER LOST. xtc_batch_submit_mt returns the handle only after its record is on the ring, and the ring is drained in the order slots were reserved, so a cancel_mt or resume_mt for that handle -- from any producer that was given it -- is always drained after the submit. It is applied to the request wherever it is: held, waiting (it never runs), active or in flight. A cancel or resume for a handle that has already completed (the client-disconnect race) is discarded; nothing is retained for it, so any number of such races cost nothing.

CROSS-LOOP RESULTS (REPLY-FUTURE PATTERN). Completion still runs the group's complete callback on the DRIVER loop; the engine does not invent a cross-loop result channel. An mt submitter that needs the result on its own loop passes a request that carries an xtc_future_t (created with xtc_future_new_pair) which the complete callback resolves with xtc_promise_set; the submitter awaits that future on its own loop. The request pointer is the consumer's per-sequence handle and carries whatever outbound state (device-output location, KV-cache slot id) the application needs.

Mutation is permitted during externally serialized pre-run setup and, once started, only from the actual owner loop. Send commands to an owner process using existing messaging APIs for cross-loop submission. A physical thread identity is not a loop identity, especially under deterministic simulation.

XTC_BATCH_DRAIN stops admission and waits for all accepted requests to finish; BLOCKED requests require explicit resume or cancellation. XTC_BATCH_CANCEL also cancels all nonterminal requests. Close is idempotent and may escalate DRAIN to CANCEL, never downgrade it. Close always signals boundary work, including when the driver is empty or all members are BLOCKED.

The required lifetime order is close, run until the driver ends, destroy, then loop or executor finalization. Even closing an empty group before its first run requires execution of its driver. Stopping a loop or executor is not draining: destroy returns XTC_E_AGAIN while the driver remains present. After a stopped run, route close through an owner process and run again.

With cross-loop producers the same order holds and extends to the ring. Once xtc_batch_close has returned, every xtc_batch_submit_mt returns XTC_E_AGAIN without issuing a handle. A submit_mt that races close either gets XTC_E_AGAIN or an XTC_OK handle that completes exactly once: XTC_BATCH_DRAIN runs it, XTC_BATCH_CANCEL ends it XTC_E_ABORTED. The driver does not finish until every such hand-in has been drained. cancel_mt and resume_mt stay accepted while the driver runs (a cancel during a drain still matters) and return XTC_E_INVAL once it has finished. Freedom from a freed-memory access is not a property the ring enforces across teardown; it rests on the caller obligation below. The admission ring and record pool are closed and freed only in destroy, after the driver is gone, and the group heap itself is not freed until then. The caller must ensure no producer is still issuing mt calls when destroy runs (the owner-outlives-group discipline); the loop that owns the group must outlive every loop that submits to it.

Destroy requires a closed group, no borrowed requests or callback, and driver absence from the process table. An exit hook flag or alive==0 is insufficient: other exit hooks may still be parked. Destroy and statistics additionally allow externally synchronized quiescent calls after executor return, with the loop still allocated. These are not general thread-safe operations; a NULL current loop does not prove quiescence. Calling loop/exec fini with a live group violates the lifetime contract and does not promise automatic terminal delivery.

Normal shutdown uses batch_close, not an asynchronous driver kill. An idle notify waiter need not wake on kill alone. A later normal batch signal permits masked waiter unlink before pending kill delivery. There is no hidden polling thread, periodic wake, real clock read, or unseeded randomness in the engine.

The placement helpers are advisory glue, not a scheduler. There is no automatic migration of a live group: a group stays on its creation loop. They only help a consumer choose a loop at create time and shard across a fixed set of groups it created itself. They are available only when built with --enable-sched-policy; disabled builds return XTC_E_NOSYS (and xtc_batch_loop_for_device still writes 0 to out_loop_idx).

xtc_batch_loop_for_device maps a probed accelerator device (xtc_accel_probe, see xtc_accel(3)) to the executor loop index whose carrier thread runs on the same NUMA node as the device, best-effort. It reads the device's NUMA node from static sysfs topology: for a renderD* node from /sys/class/drm/<name>/device/numa_node, for an accel* node from /sys/class/accel/<name>/device/numa_node (dev->name is the short label, e.g. "renderD128" or "accel0"). On success it writes a valid loop index in [0, xtc_exec_n_loops) to out_loop_idx and returns XTC_OK. When the device numa_node is -1 or unknown, the executor's loops record no known node, or the host reports a single NUMA node, it writes 0 (the loop-0 fallback) and still returns XTC_OK: absence of topology is not an error, matching xtc_accel's convention. A carrier records its node when it starts, so a loop that has not run has no known node: called before the executor has run -- which is when a group can be created -- the helper currently always returns loop 0. It never returns an out-of-range index. A NULL argument returns XTC_E_INVAL. The read is a one-shot static-topology probe and touches no sim-nondeterministic source, so it is safe to call under the deterministic simulator; it is not meant for a per-iteration hot path.

xtc_batch_router_create, xtc_batch_route, and xtc_batch_router_destroy are a keyed-shard router: N groups plus a modulo, nothing more. The caller creates one group per shard loop (xtc_batch_create or xtc_batch_create_async per loop, typically placed with xtc_batch_loop_for_device) and hands the array to xtc_batch_router_create, which copies the pointer array (it borrows the groups; it does not own them) and returns an opaque router. xtc_batch_route returns groups[key % n], a stable key-to-shard mapping; the same key always returns the same group. The router does not submit: the caller does xtc_batch_submit (or xtc_batch_submit_mt) on the group that xtc_batch_route returns. The router adds no cross-group runtime coupling. xtc_batch_router_destroy frees only the router, never the groups; the caller owns and must separately close and destroy each group on its owner loop, following the normal batch lifetime contract above. xtc_batch_router_create returns XTC_E_INVAL on a NULL array/out or n <= 0 (or a NULL group in the array), XTC_E_NOMEM on allocation failure, and XTC_E_NOSYS in a disabled build; xtc_batch_route returns NULL on a NULL router.

The snapshot includes queued, ready, blocked, reserved_units, iterations, completed, cancelled, failed, rejections, max_vector, and total_members. Queued means waiting for admission; ready and blocked count active members. Queued includes cross-loop hand-ins held for a slot. Iteration and membership totals measure vector calls and their occupancy. Terminal counters advance after completion delivery returns (or after an interrupted delivery is retired). Rejections count owner-loop xtc_batch_submit calls that failed after owner/callback validation; refused mt calls count in the async statistics' mt_rejections instead, and an accepted hand-in is never counted as a rejection. Cumulative counters saturate at UINT64_MAX. Per-request timing belongs in application instrumentation.

XTC_E_INVAL indicates malformed arguments, descriptor size, foreign ownership, foreign class, invalid close mode, or an unavailable handle. XTC_E_VERSION indicates an unsupported descriptor version. XTC_E_RANGE indicates charge, allocation arithmetic, or handle exhaustion. XTC_E_AGAIN indicates full/closed admission, creation after first execution, or destruction before shutdown has completed. XTC_E_DEADLK indicates mutation during a callback. XTC_E_NOMEM indicates allocation failure during create.

xtc_accel(3), xtc_exec(3), xtc_proc(3), xtc_sched(3), xtc_sync(3)

2026 libxtc

View the mdoc source