xtc_usdt(3)

---

xtc_usdt(3)

static probes for DTrace, perf, bpftrace and SystemTap

XTC_USDT(3) Library Functions Manual XTC_USDT(3)

xtc_usdt — static probes for DTrace, perf, bpftrace and SystemTap

library “libxtc”

./dist/configure [--enable-dtrace[=yes | sdt | auto | no]] [DTRACE=path] [DTRACEFLAGS=flags]


meson setup -Ddtrace=no | auto | yes | sdt


dtrace -n ‘xtc$target:::probe-name { ... }’ -c program


bpftrace -e ‘usdt:./program:xtc:probe__name { ... }’

libxtc can be built with a set of user-level statically defined tracing (USDT) probes in its scheduler, mailbox, timer, I/O, lock and supervision paths. The probes are defined once, as the xtc provider in src/xtc_probes.d, and serve two tracer families from one build:

  • DTrace on FreeBSD, illumos (OmniOS, SmartOS) and macOS, which see them as xtc.
  • perf, bpftrace, bcc and SystemTap on Linux, which read the ELF notes the same probes produce and see them as (double underscores; DTrace shows each ‘__’ as a hyphen).

The probes are off by default. In a build without them every probe macro expands to nothing and none of its arguments is evaluated: the library's machine code is identical to a build that has no probe sites at all.

=no
The default. No probes.
=yes
Requires a dtrace(1) program. ‘dtrace -h’ generates the probe header. Except on macOS, ‘dtrace -G’ then rewrites each probe call site in the library objects to no-ops and emits the probe table into xtc_probes.o, which is archived into libxtc.a and linked into the shared library. Every object with a probe site references an anchor symbol in xtc_probes.o, so a program linked statically against libxtc.a always carries the probe table. On Linux this mode works with the systemtap dtrace script, which also emits the is-enabled semaphores.
=sdt
Linux, header-only: the systemtap <sys/sdt.h> (package systemtap-sdt-dev or systemtap-sdt-devel) emits the notes directly; no tool runs at build time. There are no semaphores, so () is always true and probe arguments are computed at every site; they are cheap by design (see Cost).
=auto
if a working dtrace is found, else sdt if a systemtap <sys/sdt.h> is, else no.

Tested: FreeBSD with yes, Linux with sdt. Linux with yes (the systemtap dtrace script), macOS and illumos follow the same recipe but were not run when the probes were added.

DTRACE names the program and DTRACEFLAGS passes flags to it (for example ‘DTRACEFLAGS=-64’ on illumos), as in PostgreSQL's configure. On Windows the option is refused; ETW is the analogue there and is not provided.

The single-file amalgamation (xtc.c) has no configure step. Compile it with -DXTC_PROBES_SDT on Linux to get the notes. For DTrace, run ‘dtrace -h -s src/xtc_probes.d’ into xtc_probes_gen.h, rename its XTC_ macros to XTC_DT_ the way dist/xtc_probes.sh does, save it as xtc_probes_dtrace.h, compile with -DXTC_HAVE_DTRACE, and on FreeBSD or illumos run ‘dtrace -G -s src/xtc_probes.d -o probes.o xtc.o’ and link probes.o next to xtc.o.

pid
A libxtc pid in one 64-bit argument: ‘loop_id << 48 | local_id << 32 | gen’.
task
The fiber's xtc_task_t * as an integer. It is stable for the fiber's life and is the key that joins fiber-park, fiber-wake, fiber-resume and the run probes. Key D variables on it rather than on the thread: a migratable fiber can park on one OS thread and resume on another.
loop
The executor loop index, or -1 for a loop that is not part of an xtc_exec(3) executor.

No probe argument reads a clock. Take time from the tracer (D timestamp, bpftrace nsecs). That keeps the probes out of the deterministic simulator's determinism guard, and under the simulator the deadline_ns and now_ns arguments of the timer probes are virtual time.

(pid, parent)
A process was created (xtc_proc_spawn(3)). parent is the spawning process, 0 if it was spawned from outside one.
(pid, task)
The process began running on its fiber.
(pid, reason)
The process finished with exit reason, before links and monitors are notified.
(target, reason)
xtc_exit_pid(3) was called on a live process.
(from, to, bytes)
A message was queued for delivery (xtc_send(3)).
(pid, from, bytes)
A process took a message from its mailbox.
(task, reason, object)
A fiber is about to yield because it must wait. reason is one of 1 mailbox (xtc_recv(3)), 2 sleep (xtc_proc_sleep(3)), 3 fd (xtc_proc_wait_fd(3)), object is the fd), 4 aio (xtc_aio(3)); object is the request, matching aio-submit), 5 blocking pool (object is the work item), matching blocking-start), 6 sync primitive (amutex, arwlock, semaphore, notify, barrier, gate; object is the primitive's internal lock), 7 lwlock (object is the lock), 8 lock manager (object is the manager), 9 future (object is the future). A fiber may park and resume several times for one wait: it rechecks its condition after every resume.
(task, reason, object)
The fiber is running again after the matching fiber-park. The time between the two is time spent parked plus time spent runnable but not yet scheduled.
(task, loop, how)
A waker targeted the fiber. how is 0 when the waker made it runnable on this thread's loop, 1 when it posted it to another loop's inbox, and 2 when the fiber was not parked (the wake was latched for a park still in progress, or the fiber was already runnable). A fiber-park followed by no fiber-wake means the wake source never fired; one followed by a wake with how 0 or 1 and no fiber-resume means the wake was lost after it was generated.
(task, loop, lane)
The scheduler is about to run a task quantum. lane is its scheduling class, -1 for the default lane.
(task, loop, verdict)
The quantum returned: 0 done, 1 rescheduled (yield), 2 parked.
(loop, timeout_ns, nevents)
A blocking poll of the loop's I/O backend returned nevents events; -1 timeout_ns waits forever. Non-blocking fairness polls are not reported.
(loop, fd, flags, task)
An I/O event was dispatched to the task waiting for it (fd is -1 for an AIO completion).
(thief, victim, task)
A loop took a migratable task from another loop's queue.
(timer, deadline_ns, task)
A timer was armed: task is the fiber a park timer will wake, 0 for an xtc_timer_set(3) callback timer.
(timer, deadline_ns, now_ns)
A timer fired; now_ns - deadline_ns is its lateness on the libxtc monotonic clock.
(timer)
A pending timer was cancelled before it fired.
(aio, op, fd, len)
A file operation reached the native AIO backend. op: 0 pread, 1 pwrite, 2 fsync, 3 fdatasync, 4 preadv, 5 pwritev. len is 0 for the vector and sync operations. An operation the backend refused falls back to the blocking pool and fires the blocking probes instead.
(aio, op, res)
The submitting fiber saw the result: bytes, or a negative errno.
(work, qlen)
Work was queued for the blocking pool (xtc_blocking_run(3)); qlen is the queue length after it.
(work)
A pool thread began running it.
(work, result)
The pool thread finished it.
(lock, kind, mode, waiter)
A caller blocked on a contended lock: it is about to sleep on a condition variable or park its fiber. kind: 1 xtc_amutex(3), 2 arwlock, 3 xtc_lwlock(3), 4 xtc_lockmgr(3) (then lock is the locked object, not the manager). mode: 0 for an amutex; 0 shared, 1 exclusive for an arwlock; the xtc_lwlock_mode_t or xtc_lock_mode_t otherwise. waiter is the parking fiber's task, 0 for a waiting OS thread. Uncontended acquisitions do not fire; trace those with the pid provider or a uprobe on the function.
(lock, kind, mode, waiter)
The caller woke up: it holds the lock, will recheck and may wait again (one start/done pair per sleep), or timed out.
(sup, child, reason)
A supervisor is restarting after child exited with reason (xtc_sup(3)).
(involuntary)
A preemption timer tick arrived (xtc_preempt(3)): 1 if it preempted the running fiber in signal context, 0 if it was deferred to the next cooperative yield check. Fires in a signal handler.
(bytes)
A libxtc heap allocation failed.
(slab, bytes)
A slab cache mapped a new chunk (xtc_slab(3)).
(what)
Under the deterministic simulator, a sim-reachable path called a nondeterministic primitive (what is a string naming it). The run aborts right after unless strict mode is off.
(site, fired)
Under the simulator, a buggify site was reached for the first time in the run and its seeded decision made (site is a string).

The provider name, probe names, argument order and the numeric codes above are an interface: they change only in a minor release, with a note in the release notes, and an existing probe's arguments are never reordered. New probes and new codes may be added. The module and function components of a probe (the dtrace(1) mod and func fields) are private: they name libxtc's internal functions and can change in any release. The D stability attributes in src/xtc_probes.d say the same: Evolving for the provider, names and arguments, Private for module and function.

Disabled, a DTrace probe is a few no-op instructions where the call used to be, plus the argument setup the compiler emitted for it; an SDT probe is one . Probes sit on the slow paths: park, wake, a contended lock, a timer firing, a failed allocation, a supervisor restart. Fast paths (an uncontended lock, a send to a running process, a slab hit) carry none. Arguments are registers and plain loads, never a call or a clock read; wrap anything more expensive in (NAME). A build without probes contains none of this.

src/xtc_probes.d
The probe definitions.
src/inc/xtc_probes.h
The macros at each probe site (library-internal).
dist/xtc_probes.sh
The ‘dtrace -h’ and ‘dtrace -G’ steps.
tools/dtrace/*.d
D scripts: park latency by reason, lock contention by lock with stacks, timer lateness, AIO latency per operation, stranded fibers.
tools/bpftrace/*.bt
The same for bpftrace on Linux.

Count messages and park reasons in a running program:

dtrace -qn 'xtc$target:::msg-send { @["sent"] = count(); }
    xtc$target:::fiber-park { @why[arg1] = count(); }' -p 1234

Park-to-resume latency by reason:

dtrace -qs tools/dtrace/xtc_park_latency.d -c ./server

The same on Linux with bpftrace, the binary linked against libxtc.a built with --enable-dtrace=sdt:

bpftrace -e 'usdt:./server:xtc:fiber__park { @[arg1] = count(); }'

List the probes perf sees and record one. ‘perf probe’ names the event with each double underscore collapsed to one, so record ‘sdt_xtc:msg_send’, not ‘sdt_xtc:msg__send’:

perf buildid-cache --add ./server
perf list 'sdt_xtc:*'
perf probe -x ./server sdt_xtc:msg__send
perf record -e sdt_xtc:msg_send -g ./server

perf 7.2 cannot enable a probe that has more than one site in the binary (for example ‘lock__wait__start’, which fires from four lock paths): the second site gets the same collapsed event name and the kernel refuses it. Use bpftrace or SystemTap for those probes.

dtrace(1), xtc_stats(3), xtc_tail(3), xtc_trace(3), xtc(7)

The probes first appeared in libxtc 1.59.0.

On FreeBSD and illumos, ‘dtrace -l’ and ‘dtrace -n’ on a user process need root, or the dtrace_user privilege on illumos. On macOS, System Integrity Protection restricts DTrace on system binaries; your own programs are traceable once DTrace is allowed. On Linux, perf and bpftrace need CAP_PERFMON or root. A program that loads libxtc as a shared library registers its probes at load time; one linked statically carries them in its own image.

October 22, 2026 Debian

View the mdoc source