Processes and messages

Addressable, mailbox-owning units with private state -- the Erlang/BEAM model, in C.

---
  1. Spawn, send, receive
  2. Three things to notice
  3. Selective receive
  4. What you have learned

A bare coroutine computes and returns. A process is a coroutine with an identity (a xtc_pid_t) and a mailbox: other processes address it by pid and communicate only by sending it messages. Nothing is shared; there are no locks on a process’s state because no one else can touch it. This is the Erlang/BEAM model, in C.

Spawn, send, receive

xtc_proc_spawn(loop, fn, arg, opts, &pid) starts fn(arg) as a process and gives you back its pid. Inside a process, xtc_self() returns your own pid, xtc_send(pid, data, size) copies size bytes into another process’s mailbox, and xtc_recv(&buf, &size, timeout_ns) blocks (suspends the fiber) until a message arrives or the timeout elapses.

Here is a two-process ping/pong. pong waits for a number and replies with one more; ping kicks it off and bounces the number back until it reaches a limit.

sequenceDiagram
    participant P as ping
    participant Q as pong
    P->>Q: {from: ping, n: 0}
    Q->>P: {from: pong, n: 1}
    P->>Q: {from: ping, n: 2}
    Q->>P: {from: pong, n: 3}
    P->>Q: {from: ping, n: 4}
    Note over Q: n >= ROUNDS, done

Each arrow is one xtc_send into the target’s mailbox; each process sits in xtc_recv until a message arrives. No shared memory, no locks.


#include <stdio.h>
#include <stdint.h>
#include <string.h>

#include "xtc.h"
#include "xtc_loop.h"
#include "xtc_proc.h"

#define ROUNDS 4

/* Messages are plain bytes.  xtc_recv gives no sender, so we carry the
 * reply address in the payload -- the usual libxtc idiom. */
struct msg {
	xtc_pid_t from;
	int       n;
};

static void
pong(void *arg)
{
	void  *raw;
	size_t sz;

	(void)arg;
	for (;;) {
		struct msg req, reply;

		if (xtc_recv(&raw, &sz, 1000LL * 1000 * 1000) != XTC_OK)
			return;                 /* timed out: no partner left */
		if (sz != sizeof req) {
			xtc_free(raw);
			continue;
		}
		memcpy(&req, raw, sizeof req);
		xtc_free(raw);              /* received buffers are ours to free */

		if (req.n >= ROUNDS) {
			printf("pong: reached %d, done\n", req.n);
			return;
		}
		reply.from = xtc_self();
		reply.n    = req.n + 1;
		(void)xtc_send(req.from, &reply, sizeof reply);
	}
}

static void
ping(void *arg)
{
	xtc_pid_t peer = *(xtc_pid_t *)arg;
	struct msg first = { xtc_self(), 0 };
	void  *raw;
	size_t sz;

	(void)xtc_send(peer, &first, sizeof first);
	for (;;) {
		struct msg r;

		if (xtc_recv(&raw, &sz, 1000LL * 1000 * 1000) != XTC_OK)
			return;
		memcpy(&r, raw, sizeof r);
		xtc_free(raw);
		printf("ping: got %d\n", r.n);
		if (r.n >= ROUNDS)
			return;
		r.from = xtc_self();
		(void)xtc_send(peer, &r, sizeof r);
	}
}

int
main(void)
{
	xtc_loop_t *loop;
	xtc_pid_t   pong_pid;

	if (xtc_loop_init(&loop) != XTC_OK)
		return 1;
	if (xtc_proc_spawn(loop, pong, NULL, NULL, &pong_pid) != XTC_OK)
		return 1;
	if (xtc_proc_spawn(loop, ping, &pong_pid, NULL, NULL) != XTC_OK)
		return 1;
	(void)xtc_loop_run(loop);
	(void)xtc_loop_fini(loop);
	return 0;
}

Tested source: docs/_includes/snippets/03_ping_pong.c

ping: got 1
ping: got 3
pong: reached 4, done

Three things to notice

Messages are copies. xtc_send copies the bytes into the recipient’s mailbox. The sender and receiver never share the buffer, so there is nothing to lock and no lifetime to coordinate across processes.

A received buffer is yours to free. xtc_recv hands you a heap buffer that you own. Release it with xtc_free – not plain free. libxtc may be running under a custom allocator (an embedder like PostgreSQL installs one), and freeing an allocator-supplied buffer with the C library free is a mismatched-free bug. Every libxtc call that returns a caller-owned buffer documents xtc_free; that man page lists them.

There is no sender field. xtc_recv does not tell you who sent the message. If you need to reply, put your own pid in the payload – that is what the from field in the example does. This keeps the mailbox a plain byte queue and lets you design your own protocols on top.

Shared state behind a mutex. The C default is a struct guarded by a pthread_mutex. It is faster for a single hot counter, but it does not compose: every new invariant adds another lock, lock order becomes a global proof obligation, and a thread that dies holding a lock wedges everyone. The process model trades a little copy cost for the property that state has exactly one owner and failure is contained to that owner. libxtc still ships mutexes, rwlocks, RCU, and a lock manager (Locks and synchronization) for the cases that genuinely want shared memory – but the default unit of concurrency is the shared-nothing process.

Selective receive

Sometimes a process wants the next message that matches a predicate, leaving others in the mailbox for later. xtc_recv_match(match_fn, user_data, &buf, &size, timeout) scans the mailbox and returns the first message for which match_fn returns non-zero, preserving the arrival order of the rest. This is how you implement a request/response correlation (pull the reply with your request id) without draining unrelated traffic. See xtc_proc(3).

What you have learned

  • A process is an addressable, mailbox-owning coroutine with private state.
  • xtc_send copies; xtc_recv / xtc_recv_match receive; received buffers are freed with xtc_free.
  • Replies carry the sender pid in the payload by convention.

Processes let things run independently. The next chapter is about what happens when one of them fails, and how to build systems that recover: links, monitors, and supervisors.

← Fibers and the event loop · Next: Links, monitors, and supervisors →