xtc_proc(3)

---

xtc_proc(3)

BEAM-style lightweight processes with mailboxes

XTC_PROC(3) Library Functions Manual XTC_PROC(3)

xtc_proc_spawn, xtc_proc_spawn_link, xtc_proc_spawn_monitor, xtc_self, xtc_proc_set_userdata, xtc_proc_userdata, xtc_proc_set_class, xtc_send, xtc_send_owned, xtc_recv, xtc_recv_match, xtc_recv_correlate, xtc_proc_wait_fd, xtc_proc_sleep, xtc_exit_self, xtc_exit_pid, xtc_exit_pid_deadline, xtc_proc_wake, xtc_link, xtc_unlink, xtc_proc_trap_exit, xtc_monitor, xtc_proc_mailbox_stats, xtc_fault_guard_install, xtc_proc_recovery_arm, xtc_proc_critical_enter, xtc_proc_at_exit, xtc_down_decode xtc_down_decode_ex — BEAM-style lightweight processes with mailboxes

#include <xtc.h>
#include <xtc_proc.h>

int
xtc_proc_spawn(xtc_loop_t *loop, xtc_proc_fn fn, void *arg, const xtc_proc_opts_t *opts, xtc_pid_t *out_pid);

int
xtc_proc_spawn_link(xtc_loop_t *loop, xtc_proc_fn fn, void *arg, const xtc_proc_opts_t *opts, xtc_pid_t *out_pid);

int
xtc_proc_spawn_monitor(xtc_loop_t *loop, xtc_proc_fn fn, void *arg, const xtc_proc_opts_t *opts, xtc_pid_t *out_pid, uint64_t *out_ref);

xtc_pid_t
xtc_self(void);

int
xtc_send(xtc_pid_t target, const void *data, size_t size);

int
xtc_send_owned(xtc_pid_t target, void *buf, size_t size);

int
xtc_recv(void **out, size_t *out_size, int64_t timeout_ns);

int
xtc_recv_match(xtc_match_fn match_fn, void *user_data, void **out, size_t *out_size, int64_t timeout_ns);

int
xtc_recv_correlate(const void *corr_value, size_t corr_size, int n_expected, xtc_msg_t *out_msgs, int *out_n, int64_t timeout_ns);

int
xtc_proc_wait_fd(int fd, uint32_t interest, int64_t timeout_ns, uint32_t *out_revents);

int
xtc_proc_sleep(int64_t ns);

int
xtc_exit_self(int reason);

int
xtc_exit_pid(xtc_pid_t pid, int reason);

int
xtc_exit_pid_deadline(xtc_pid_t pid, int reason, int64_t timeout_ns, int *out_status);

int
xtc_proc_wake(xtc_pid_t pid);

int
xtc_link(xtc_pid_t other);

int
xtc_unlink(xtc_pid_t other);

int
xtc_proc_trap_exit(int on);

int
xtc_monitor(xtc_pid_t target, uint64_t *out_ref);

int
xtc_proc_mailbox_stats(xtc_pid_t pid, xtc_mailbox_stats_t *out);

int
xtc_fault_guard_install(void);

int
xtc_proc_recovery_arm(void);

void
xtc_proc_recovery_disarm(void);

void
xtc_proc_critical_enter(void);

void
xtc_proc_critical_leave(void);

int
xtc_proc_at_exit(void (*fn)(void *), void *arg);

struct xtc_mctx *
xtc_proc_mctx(void);

int
xtc_down_decode(const void *msg, size_t len, xtc_pid_t *out_pid, int *out_reason);

int
xtc_down_decode_ex(const void *msg, size_t len, xtc_down_info_t *out);

xtc processes are BEAM-style lightweight actors: each has an identity (xtc_pid_t), a private mailbox, and a stack-backed coroutine that runs co-operatively inside an xtc_loop(3). Communication is by message passing. Failures are isolated by xtc_supervisor(3) or by manual link/monitor.

() creates a new process running fn (arg). The process runs on loop co-operatively with all other procs and tasks on that loop. opts is optional and may set the mailbox capacity, initial stack size, or per-proc resource accountant.

A process is pinned to loop for its lifetime: it runs only on that loop and is never migrated by work-stealing. This affinity is guaranteed and requires no opt-in -- only raw xtc_exec(3) tasks are stealable. Embedders that pin a worker to one loop (e.g. a PostgreSQL backend per loop) can rely on it.

()

() appends an envelope holding a copy of data [0 .. size) to the mailbox of target. It copies the payload once: a payload of at most 256 bytes into a pooled envelope (copied out again by the receive), a larger one into a buffer the receive then hands over without a second copy. Returns XTC_OK on success, XTC_E_INVAL if the target is dead, or XTC_E_AGAIN if the target's mailbox is full and bounded. Sending is asynchronous: the send returns immediately whether or not the receiver is currently scheduled. If the message was queued but waking a parked receiver failed, the send returns that error (e.g. XTC_E_NOMEM) rather than XTC_OK; the message IS delivered, so do not resend it; call xtc_proc_wake() to retry the wake.

() sends without copying: it moves buf itself into the mailbox, and the receiver's xtc_recv() (or xtc_recv_match(), xtc_recv_correlate()) returns that same pointer. buf must come from libxtc's allocator ((), (), (), or a buffer libxtc handed out to be freed with xtc_free(), such as a received message). On XTC_OK the buffer belongs to the library and then to the receiver, who frees it with xtc_free(); the sender must not touch it again. On any error the sender still owns it, unchanged, so a full mailbox (XTC_E_AGAIN) can be retried with the same buffer. buf may be NULL only with a zero size. Use it for payloads above a few kilobytes, or whenever the sender builds the message in a fresh buffer anyway: on one loop it moves a 4 KiB message about 4x faster than xtc_send(), and a 64 KiB one about 10x. Across loops a small owned message costs a little more than a copied one (two allocations freed on another thread); the crossover is between 4 KiB and 64 KiB.

Mailboxes are bounded by mailbox_cap (default 4096) so that a fast sender cannot drive the receiver out of memory. A caller that ignores an XTC_E_AGAIN return silently loses the message. Senders therefore MUST handle it: retry later, shed and account the load, apply end-to-end flow control, or treat it as fatal on a must-deliver path.

() delivers the next message from the calling proc's mailbox in arrival order. If the mailbox is empty, timeout_ns controls behaviour:

timeout_ns < 0
Block forever.
timeout_ns == 0
Return XTC_E_AGAIN immediately if no message is queued.
timeout_ns > 0
Block up to that many nanoseconds, then return XTC_E_AGAIN on timeout.
The delivered buffer is owned by the caller and must be released with (). It is the sender's own buffer for an xtc_send_owned() message and the envelope's payload buffer for an xtc_send() above 256 bytes (no copy at receive); a smaller payload is copied out. Release it with xtc_free(), NOT free(3): it comes from libxtc's own allocator, which an embedder may have replaced, so releasing it through the C library would be a mismatched free.

() performs selective receive: the callback match_fn is invoked for each envelope in arrival order; the first one for which it returns 1 is delivered. Non-matching envelopes are parked in a save queue. Every call tests the whole save queue against its predicate first, so a call whose predicate value differs from the previous call's (even through the same user_data pointer) still finds an envelope saved earlier. Releases before 1.55.1 skipped saved envelopes when match_fn and user_data were the same pointers as the previous call, which made xtc_recv_correlate() miss a reply that arrived before its request was waited on.

() collects n_expected messages whose first corr_size bytes match corr_value. Useful for fork-join patterns where the parent assigns each child a correlation id and waits for replies tagged with that id. The collected messages are written to out_msgs; *out_n is set to the count actually delivered (which may be less than n_expected if the timeout fires first).

() blocks until any of:

  • fd becomes ready in any interest direction (XTC_IO_READABLE | XTC_IO_WRITABLE | XTC_IO_HUP | XTC_IO_ERR),
  • a message arrives in the calling proc's mailbox (XTC_WAIT_MAILBOX)
  • the timeout elapses (XTC_WAIT_TIMEOUT)
  • the proc is killed via xtc_exit_pid().
*out_revents receives a bitmask of which conditions fired. This is the canonical way to write an idle-CPU-zero server: the proc sleeps until something real happens.

The return code distinguishes the timeout from every other wake: XTC_E_AGAIN is returned ONLY when the timeout actually fired with nothing else, and in that case *out_revents carries XTC_WAIT_TIMEOUT. A wake for any other reason -- including an explicit () that sets no readiness bit -- returns XTC_OK, possibly with *out_revents of zero, so a caller must re-check its own condition on XTC_OK rather than assume a bit is set. An earlier release reported XTC_E_AGAIN for every bit-less wake whenever a timeout had merely been SUPPLIED, which made an explicit wake indistinguishable from an expiry.

A kill delivered while the proc is parked here ends the park and unwinds the fiber, and the park's scheduler resources (the fd registration and the timeout timer) are released before the unwind completes, so a killed waiter does not leave the fd registered.

A closed fd is a programming error and returns XTC_E_INVAL (POSIX; it is probed before the fd is registered). Releases before 1.50 reported it as XTC_E_INTERNAL on epoll, and as a timeout or an XTC_IO_ERR wake on io_uring.

() parks the calling proc on a timer for at least ns nanoseconds. The loop runs other work meanwhile -- it does not block the thread -- and, unlike a timed xtc_recv(), it does not touch the mailbox. Returns XTC_E_INVAL if not called from a proc.

() terminates the calling process; cleanup runs and any links / monitors fire.

() terminates another process by pid. Asynchronous: it sets the kill and makes the target runnable wherever it is parked (xtc_proc_sleep(), xtc_recv(), xtc_proc_wait_fd(), and the sync primitives), so the target acts on it promptly rather than when its park would have ended. A park site delivers the kill at its own safe point, after it has released what it holds; one that does not, delivers it at the target's next xtc_recv(), xtc_proc_sleep(), or plain yield. A proc parked in () is not woken; it sees the kill after the await returns. A kill inside a masked region is latched as before. It is fire-and-forget: XTC_OK means the kill flag was set, not that the target ever acted on it.

() is the same kill, but waits up to timeout_ns and REPORTS the outcome in *out_status (which may be NULL):

The target observed the kill and is gone (or already was). A pid that was never issued is not DELIVERED: xtc_exit_pid_deadline() returns XTC_E_INVAL for it, as xtc_exit_pid() does. Releases before 1.50 returned XTC_OK with XTC_KILL_DELIVERED for an unknown pid, and XTC_E_INVAL for a target still running its at-exit hooks.
The target is inside xtc_uncancelable() with the kill latched: it has SEEN the kill and cannot act on it. The fiber is wedged in a critical section; a supervisor should escalate, not retry (a second xtc_exit_pid() is a no-op -- first call wins).
Not masked, but has not reached a delivery point yet: a CPU-bound loop that never yields, or a park outlasting the deadline. Usually means "wait longer".

The distinction matters because masking makes cancellation but cannot make a stuck critical section : without the report, a supervisor cannot tell a fiber that is unwinding from one that will never honor the kill, and has to guess after a fixed timeout. The same state is readable from outside the fiber via mask_depth and mask_deferred in xtc_proc_info_t (xtc_inspect(3)), which answers "is this fiber stuck in a critical section, or merely slow?" without sending a kill at all. The caller must not be the target (that is ()); it returns XTC_E_INVAL if it is.

Cancellation in libxtc guarantees : xtc_scope(3) finalizers and xtc_proc_at_exit() hooks run on an async kill, and xtc_uncancelable() ensures a resource acquired in a masked region always registers its release before cancellation is observed.

It guarantees nothing about . Releasing a lock does not undo a half-finished mutation. A fiber killed mid-update of state that OTHER fibers keep using -- a partially written log record, a page updated but not yet accounted for, an index entry without its counterpart -- leaves that state broken while remaining perfectly valid-looking to every other fiber in the address space.

So the stance is explicit: . Wrapping the dangerous window in () is right when the window always completes, and it is exactly wrong for a wedged fiber -- the mask then defers the kill forever, which is the hang you were trying to break.

For that case the correct escalation is to take down a larger unit whose shared state can be discarded wholesale and rebuilt from a durable log: the process, not the fiber. A single address space gives no smaller recovery boundary. This is not a workaround for a missing call -- it is the same trade a crash-only database makes when a worker is stuck and must die now, and it stays the right final backstop even where a per-fiber kill is available. Use () and the mask_deferred snapshot to decide WHEN to escalate; do not use them to justify killing a fiber that cannot be safely killed.

() resumes a process parked in xtc_proc_wait_fd() or xtc_recv(), from ANY OS thread -- including one libxtc does not manage (an embedder's I/O-worker thread, a foreign carrier). It is the explicit "poke the target loop" primitive: after a foreign thread makes a watched condition true (writes a self-pipe an fd-park watches, sets an embedder latch, completes an async read), calling xtc_proc_wake() nudges the target loop out of its I/O wait so the parked proc re-evaluates its condition. It delivers no message and asserts no wake cause; the woken proc simply re-checks (its xtc_proc_wait_fd() re-checks fd readiness and the mailbox, its xtc_recv() re-checks the mailbox), so a spurious wake is always safe. Relying on fd readiness alone to wake a loop is correct for a condition libxtc itself produces, but when the readiness is produced by a fully foreign thread an embedder should pair it with xtc_proc_wake() for a guaranteed, race-free resume. Returns XTC_OK (including the no-op cases: the target is not parked, already runnable, or gone), or XTC_E_INVAL for XTC_PID_NONE.

() sets whether the calling proc traps exits (default off) and returns the previous setting, or XTC_E_INVAL off a proc.

This is a behaviour change in 1.55.1. Earlier releases never killed a linked peer; every peer received an ‘EXIT’ message for every exit, normal included. Code that links and then waits for that message must call (1) first. XTC_HAVE_PROC_TRAP_EXIT is defined when the call exists.

() creates a unidirectional monitor. The watcher receives a ‘DOWN’ message ({kind='D', ref, pid, reason}) when the monitored process exits. DOWN and EXIT messages are not subject to mailbox_cap: they are bounded by the number of monitors and links, so a watcher whose mailbox is full still receives them. Releases before 1.55.1 dropped them in that case.

() snapshots a process's mailbox into out: the current depth, the selective-receive saved count, the high-water peak, the cap, and lifetime recv_total and drop_total counts (messages accepted and rejected). It is safe to call from any thread and is the basis for an overload / head-of-line-buildup scrape.

The capacity bound applies to depth + saved, not the mailbox alone: a selective receiver that moves non-matching messages aside still counts them, so a flood of non-matching traffic cannot grow a process without limit by draining the mailbox into the save queue. A consequence is that a process selectively waiting for a message while its budget fills with non-matching ones will see further sends rejected with XTC_E_AGAIN (including the awaited one): the bound is a hard backpressure contract, so design selective waits with that in mind (prefer () and bounded timeouts).

A process can also be given a mailbox watermark via xtc_proc_opts_t.mailbox_watermark_pct (1..100; 0 disables) plus mailbox_watermark_fn and mailbox_watermark_user. When an accepted message first brings the depth to that percent of the cap, the callback fires once on the rising edge -- so an application can shed load before the hard cap rejects with XTC_E_AGAIN. The callback runs on the sender's thread outside the mailbox lock and must not block.

() installs a process-wide handler for the synchronous, fiber-directed faults SIGSEGV, SIGBUS, SIGFPE and SIGILL on an alternate signal stack.

Fault containment is PER-OS-THREAD: () must have run on every OS thread that hosts a loop before a fiber fault on that thread can be contained and turned into a DOWN. A common idiom is to install it at the entry of one long-lived fiber per loop thread (a supervisor); the call is idempotent, so repeated installs are harmless. A loop thread on which the guard was never installed lets a fiber fault take down the whole process rather than containing it. () arms a recovery frame for the calling process (used like sigsetjmp(3): it returns 0 normally and the fault signal number when a contained fault returns control there). On such a fault the handler identifies the faulting process -- it is the faulting thread's running process -- and, if it armed a recovery frame and is not in a critical section, unwinds only that fiber back to the frame; siblings on the loop are untouched. The recovered process releases its resources and calls xtc_exit_self(), so a supervisor observes the fault as a normal ‘DOWN’. () and () bracket a region in which a fault must NOT be contained -- shared state may be torn -- so it escalates to process abort, preserving the embedder's critical-section panic semantics. The recovery frame only unwinds the call stack; locks, fds, allocations and other resources held at fault time are the recovery block's responsibility to release. Containment is POSIX-only; on Windows the calls compile but a fault keeps its process-wide disposition.

() registers a callback to run when the calling process exits -- on a normal return or a contained-fault recovery -- LIFO, outside signal context, and before the proc's monitors observe its ‘DOWN’. It is where an embedder guarantees a faulted session releases what it held: register the lock manager's release-all so no lock outlives the proc, close fds, or reset a memory context. () returns a memory context scoped to the process, created on first use and destroyed automatically at exit -- a backstop so a session's allocations are reclaimed even if its recovery block faults.

() extracts the target pid and exit reason from a ‘DOWN’ signal. The DOWN and EXIT signals are sent PACKED; a consumer that casts the message to its own unpacked mirror struct misreads the fields (the reason in particular). Use this helper rather than hand-rolling the layout.

() and () are a callback-free form of xtc_uncancelable() for code that cannot be expressed as a lambda body -- a macro pair wrapped around arbitrary straight-line code that , , , or longjmp(3) s out of the region (the shape of PostgreSQL's () / ()). They bump and drop the same per-proc mask depth xtc_uncancelable() uses, so they nest and compose with it: xtc_mask_enter() increments the depth (a kill delivered at a later park point is DEFERRED), and xtc_mask_leave() decrements it and, once the depth is back to 0, honors any kill latched while masked -- which, like the end of an xtc_uncancelable() body, may not return (it unwinds the fiber via xtc_exit_self()). N enters need N leaves; only the outermost leave can unwind. Both return XTC_OK on a proc and XTC_E_INVAL off one. Unlike xtc_uncancelable(), which brackets a synchronous body, enter and leave are separated in time and a yield may occur between them, so each independently re-anchors onto the running fiber -- a bridge that enters, parks, and later leaves stays correct even across a work-steal migration. A xtc_mask_leave() with no matching xtc_mask_enter() is a no-op, not an error (the decrement saturates at 0).

By default a proc's coroutine is to the loop it was spawned on: it is never work-stolen, and xtc_self() and the recovery frame always run on that one carrier thread. Setting migratable to 1 in xtc_proc_opts_t makes the coroutine work-stealable: on a multi-loop xtc_exec(3) executor, an idle loop may steal a proc to rebalance runnable work across cores. A proc is stolen only at a yield/park point, never mid-instruction, so no statement ever runs half on one loop and half on another.

A zeroed xtc_proc_opts_t leaves migratable at 0 (pinned) -- byte-identical to the historical behavior.

Migration preserves everything that defines the proc: its pid, (), its links / monitors and the DOWN signal, its recovery frame, and its mailbox. The carrier loop can change, and only at a yield point. This is proven under the deterministic simulator (test_sim_migratable): migratable procs are stolen across loops while xtc_self() identity and DOWN delivery are asserted to survive every migration, with byte-identical replay.

The one caller obligation: a migratable proc must not hold thread-affine state across a yield and then assume it is still on the same OS thread after resuming. The runtime carries the proc's own identity across the move; it cannot know about thread-local or CPU-affine state. If the proc reads no such state across a yield (the common case), it is unconditionally safe to make migratable.

When a consumer DOES keep per-proc state that must track the running proc across a migration, hang it off the proc itself with () (read back with ()): the pointer belongs to the proc, not the loop or OS thread, so it rides along across a carrier change exactly like the mailbox and xtc_self(). Both operate on the calling proc (keyed implicitly by xtc_self()), are O(1), allocation-free, and lock-free. The pointer is NULL by default; the runtime never dereferences or frees it (the consumer owns whatever it points to, the same lifetime contract as xtc_proc_at_exit()'s arg); it is dropped when the proc exits. xtc_proc_set_userdata() returns XTC_E_INVAL when called off a proc, and xtc_proc_userdata() returns NULL there.

() places the calling proc's task in a proportional-share scheduling class (created with xtc_exec_class_create(3) on the proc's loop), the runtime-side equivalent of setting xtc_proc_opts_t.sched_class at spawn. The class weights the proc's CPU share against the other classes on its loop; a NULL handle resets it to the default (implicit) class. Returns XTC_E_INVAL called off a proc or with a handle that is not on the calling proc's loop. See xtc_exec(3) for the proportional-share model (INSPIRED BY Glommio).

A consumer that caches CARRIER-AFFINE state across a yield -- e.g. an fd/eventfd added to a wait set that is itself per-loop, so the registration must be re-armed if the fiber resumes elsewhere -- detects a migration with xtc_exec(3)'s (): it reads the CURRENT resume's carrier (re-established every resume, like xtc_self()), so capturing it before a yield and comparing after tells the consumer whether it moved:

int before = xtc_exec_loop_id();
xtc_yield();                      /* may resume on a different loop */
if (xtc_exec_loop_id() != before)
        /* re-arm any per-loop wait registration here */

This needs no dedicated "you migrated" callback: the carrier id is already a plain, O(1), thread-local read, so the check costs one comparison at exactly the safe points a consumer already controls (its own yield/park call sites) -- there is no separate hook to remember to wire up, and no case where a resume can happen without the consumer's own code running first. A proc that never registers carrier-affine state (the common case, and true of the mailbox, userdata, and recovery frame this section covers) needs no such check at all.

All entry points return XTC_OK on success. Negative values are stable XTC_E_* codes; see xtc_strerror(3).

A request-reply pattern with selective receive and timeouts:

static void server(void *arg) {
    void *msg; size_t sz;
    for (;;) {
        if (xtc_recv(&msg, &sz, 5000LL * 1000 * 1000) != XTC_OK)
            continue;
        /* ... handle msg, reply via xtc_send to msg->reply_to ... */
        free(msg);
    }
}

xtc_pid_t pid;
xtc_proc_spawn(loop, server, NULL, NULL, &pid);

xtc_chan(3), xtc_loop(3), xtc_supervisor(3), xtc_svr(3), xtc_strerror(3), xtc(7)

xtc_proc_spawn appeared in xtc 0.1.

October 23, 2026 Debian

View the mdoc source