Your First Cluster

This chapter takes you from a fresh checkout to a running two-node Dynomite cluster fronting Valkey, and then proves the point of the whole project: a value written through one node is readable through the other. Everything here uses dynomited, the server binary. If you want the library instead, see Your First Embedded Engine.

We build a two-datacenter cluster: one node in dc1, one in dc2, each fronting its own local Valkey. The two nodes share a token, so each is a full replica of the other -- which is exactly what makes the cross-node read at the end work. This is the shape of the shipped node1.yml / node2.yml sample pair, and you can start from those files directly.

Step 1: enter the dev shell

$ cd dynomite
$ nix develop

You now have valkey-server, valkey-cli, and a way to run dynomited (cargo run -p dynomited -- ...) on your PATH.

Step 2: start two backends

Each node fronts its own backend. This is the single most common first-time mistake, so it gets an admonition:

Start two Valkey instances on distinct ports:

$ valkey-server --port 11371 --daemonize yes
$ valkey-server --port 11370 --daemonize yes

Sanity-check them:

$ valkey-cli -p 11371 ping
PONG
$ valkey-cli -p 11370 ping
PONG

Step 3: write the node configs

A dynomited config is one YAML pool stanza. Here is node1.yml, in dc1, fronting the Valkey on 11371:

dyn_o_mite:
    datacenter: dc1
    rack: rack1
    listen: 127.0.0.1:8102
    dyn_listen: 127.0.0.1:8101
    dyn_seeds:
        - 127.0.0.1:8113:rack1:dc2:101134286
    dyn_seed_provider: simple_provider
    tokens: '101134286'
    servers:
        - 127.0.0.1:11371:1
    data_store: 0
    stats_listen: 127.0.0.1:33331
    preconnect: true

And node2.yml, in dc2, fronting the Valkey on 11370:

dyn_o_mite:
    datacenter: dc2
    rack: rack1
    listen: 127.0.0.1:8114
    dyn_listen: 127.0.0.1:8113
    dyn_seeds:
        - 127.0.0.1:8101:rack1:dc1:101134286
    dyn_seed_provider: simple_provider
    tokens: '101134286'
    servers:
        - 127.0.0.1:11370:1
    data_store: 0
    stats_listen: 127.0.0.1:33333
    preconnect: true

These are the shipped samples under crates/dynomited/conf/ (node1.yml, node2.yml); you can copy them verbatim.

What each knob does

The pool key (dyn_o_mite) is just the pool name; it can be anything. Inside it:

datacenter / rack

Where this node sits in the physical hierarchy from Concepts. One rack per DC here means one full copy of the ring per DC -- so the two nodes are replicas of each other across the two DCs.

listen

The client plane: the address your Valkey clients connect to. Node 1 serves clients on 8102, node 2 on 8114. Clients speak plain RESP here.

dyn_listen

The peer plane: the address other dynomited nodes connect to for gossip and replication (the DNODE protocol). Node 1 listens on 8101, node 2 on 8113. Clients never touch this port.

dyn_seeds

The bootstrap peer list: host:dyn_port:rack:dc:token. Node 1 seeds node 2's peer address and vice versa. Once gossip converges, the full membership is learned from these seeds.

tokens

This node's position(s) on the ring. Both nodes claim the same token (101134286), so each owns the same slice in its own rack -- i.e. they are replicas. Different tokens would make them shard partners instead.

servers

The backend datastore: host:port:weight [name]. Node 1 fronts Valkey on 11371, node 2 on 11370 -- distinct backends, per the warning above.

data_store

0 selects the Valkey / RESP backend (alias valkey/redis). 1 is Memcache, 2 is Dyniak. See the dynomited(8) backend list.

stats_listen

The HTTP stats endpoint for this node. Distinct per node so both can run on one host.

preconnect

Open the backend connection at startup rather than on first request, so a misconfigured backend fails loudly and early.

The exhaustive field list, plus consistency levels, bucket types, and hinted handoff, is in Configuration.

Step 4: validate before you launch

Always dry-run the config first. The --test-conf flag parses and validates, then exits without binding anything:

$ cargo run -p dynomited -- --test-conf --conf-file node1.yml
$ cargo run -p dynomited -- --test-conf --conf-file node2.yml

A clean exit means the YAML is well-formed and internally consistent (no duplicate bucket-type names, valid consistency strings, resolvable listen addresses, and so on). Fix anything it reports before moving on.

Step 5: launch both nodes

In two terminals (both inside nix develop):

$ cargo run -p dynomited -- --conf-file node1.yml

$ cargo run -p dynomited -- --conf-file node2.yml

Each logs its bound client and peer addresses and then a gossip round as the two discover each other. Give it a second or two to converge.

Step 6: write to one node, read from the other

This is the payoff. Set a key through node 1's client port (8102):

$ valkey-cli -p 8102 set greeting "hello from dc1"
OK

Now read it through node 2's client port (8114) -- a different node, in a different datacenter, fronting a different backend:

$ valkey-cli -p 8114 get greeting
"hello from dc1"

The value crossed the cluster. Node 1 coordinated the write, routed it over the ring, and replicated it to node 2's replica over the peer plane; node 2 served the read from its own backend. The client did none of that -- it spoke plain RESP to whichever node was nearest.

You can confirm the two backends really do each hold a copy:

$ valkey-cli -p 11371 get greeting
"hello from dc1"
$ valkey-cli -p 11370 get greeting
"hello from dc1"

Two distinct Valkey instances, same value -- that is replication, not a shared backend.

Step 7: watch the stats

Each node exposes an HTTP stats endpoint on its stats_listen address:

$ curl -s http://127.0.0.1:33331/ | head
$ curl -s http://127.0.0.1:33333/ | head

You will see per-pool and per-server counters -- request rates, forward counts, peer states. These are the same metrics the Metrics chapter describes, and the fields are enumerated by dynomited --describe-stats.

What just happened

flowchart LR
  Cli1[valkey-cli :8102] -->|SET| N1(node1 dc1)
  N1 --> B1[(Valkey :11371)]
  N1 -->|DNODE replicate| N2(node2 dc2)
  N2 --> B2[(Valkey :11370)]
  Cli2[valkey-cli :8114] -->|GET| N2

A write on node 1 replicated across the peer plane to node 2; a read on node 2 served the replicated value from its own backend.

You built a shared-nothing, cross-DC replicated cluster in front of an unmodified Valkey, and your client never learned the topology. That is the Dynomite bet from Why Dynomite? made concrete.

Where to next

• Turn consistency up: set read_consistency / write_consistency in the pool stanza and re-run. Start with DC_ONE (what these samples use by default) and move to DC_QUORUM. See Configuration and Replication and Consistency.
• Add a third node in a second rack of dc1 for a local replica; the shipped redis_rack1_node.yml / redis_rack2_node.yml / redis_rack3_node.yml samples show a three-rack single-DC layout with DC_SAFE_QUORUM already configured.
• Break something on purpose: stop one node and watch gossip eject it, then restart it and watch it rejoin. The mechanics are in Membership and Gossip and Failure Handling.
• Front a different backend: set data_store: 1 and point servers at a memcached instance from the flake.
• For the operator's view -- daemonizing, log formats, pid files, admin operations -- read dynomited(8) and Running dynomited.