Guides

Eight things you will want once the mesh is up, each with the commands and the config that do it — and, where something is fiddly or unproven, the sentence that says so.

Everything here is what the daemon does today. Anything planned and not built is marked as such.

• Names — why vps.mesh works, and why every peer also has an IPv4 address
• Services — publishing a port under a name, and reaching a box that will never join
• Binding a service to the mesh — the strongest access control here
• Announcing services — letting peers see what you offer
• Several meshes — a private one and one shared with friends
• Invites and membership — admitting a device, renewing, revoking
• Relays and reachability — when a relay is needed and what asks your router
• Bandwidth and node mode — what a node carries for other people

1. Names

There is nothing to set up. The daemon runs a resolver for the mesh and registers it with the system when it starts, so every device on the mesh is <device>.mesh:

ssh root@vps.mesh
ping6 laptop.mesh
resolvectl query vps.mesh        # names shrooms0 as the link that answered

That resolver is authoritative for one suffix and answers nothing else — no forwarding, no recursion, no upstream. A VPN that quietly becomes the system resolver is a surprise nobody asked for and a privacy leak besides, so .mesh is scoped to the interface (resolvectl domain shrooms0 '~mesh') and every other name keeps going wherever it went before.

When names do not resolve, the registration is the thing to check. Serving DNS and being asked are different, and this shipped once doing only the first:

resolvectl status shrooms0       # expect DNS Servers: <overlay> and Domain: ~mesh

Port 53 needs CAP_NET_BIND_SERVICE, which the packaged unit grants. A failure there is logged and never fatal: losing names is smaller than losing tunnels.

On a host without systemd-resolved, /etc/hosts is the fallback — sudo shrooms hosts --write once, or manage_hosts = "true" to let the daemon keep it current. It is off by default because a VPN editing a file that cloud-init and NetworkManager also touch should be deliberate. It needs root, it cannot do <service>.<device>.mesh, and it does not exist on Android, which is why the resolver replaces it.

Why each peer also has an IPv4 address

The overlay is IPv6-only, because addresses are derived rather than allocated and that needs 128 bits. A browser does not care. Chromium probes for IPv6 connectivity and stops sending AAAA queries when the probe fails, and our prefix is a ULA with no global route — so on a v4-only wifi network the probe fails and the browser asks only about IPv4. Counted from the Android app's DNS counters during one page load in Brave:

arrived 24 · answered 0 · refused 0 · forwarded 4
unanswerable 5 · ipv4-only 15 · other-type 5

Fifteen A queries, five of type 65, and not one AAAA. The resolver answered every one correctly — no records, because there were none to give — and the browser concluded the name was unusable. That is why a synthetic IPv4 address exists per peer (ADR-021): an A answer in 198.18.0.0/15 that maps 1:1 to the peer's overlay address and is translated at the tun.

shrooms status
network       fd3b:ffe9:f81::/48                       peers 4 (4 up)
self          laptop  fd3b:ffe9:f81:81a7:18bc:69b1:9bb:7e69
ipv4          198.18.44.9                              for clients that ask only for A records

2. Services

A machine can publish what it runs, so it is reached by name rather than by a port number somebody has to remember. On the machine that runs them:

# /etc/shrooms/config.toml on nas
services = ["immich:2283", "jellyfin:8096"]

sudo shrooms reload            # re-reads the config; applies services
curl http://immich.nas.mesh    # from any other device, phone included
curl http://immich.nas.mesh:2283   # the declared port also works

The name is <service>.<device>.mesh. grafana:443->3000 publishes 443 and forwards to 3000 when the two should differ. Two services with the same name, or the same published port, are refused when the config loads rather than three quarters applied.

How the bare name works

Every service on a device shares that device's one overlay address, so port 80 cannot simply belong to one of them — something has to say which service a connection wants. Both protocols that matter in a browser already do: HTTP puts the name in the Host header and TLS in the SNI extension, before anything else on the wire. So the daemon listens on 80 and 443 on the overlay address and forwards by the name asked for. Nothing is terminated and no certificate is involved: the ClientHello is cleartext by construction, the name is read out of it, and the bytes go on untouched to the application, which does its own TLS exactly as before.

443 is not served at all unless a service says it speaks TLS, so a browser's https attempt is refused immediately and it falls back to http, rather than hanging while a plain HTTP server answers a ClientHello:

services = ["jellyfin:8096", "vault:8200/tls"]

The limit is worth being plain about: it works for HTTP and TLS, because nothing else announces the name it dialled. ssh, syncthing and databases still need <service>.<device>.mesh:<port>, which always works.

It is not just a DNS entry either. A great many self-hosted applications bind 0.0.0.0 and nothing else, so pointing a name at the overlay address would resolve perfectly and connect to nothing. The daemon listens on the overlay address and forwards to 127.0.0.1, which is where the application actually is. If something already holds that port on the overlay address, the daemon says so and stays out of the way rather than taking it.

shrooms status
…
services published here
NAME      REACHABLE AS              FORWARDS TO       CONNS
immich    http://immich.nas.mesh    127.0.0.1:2283    3
jellyfin  http://jellyfin.nas.mesh  unavailable: connection refused  0

Pointing at a device that will never run a mesh node

The target does not have to be this machine. A service can point at anything the publishing device can reach, which makes that device a gateway for hardware that will never join — a Home Assistant box, a printer, a NAS web interface:

# on jimmy-crib, which is on the same LAN as 192.168.0.116
services = ["ha:8080->192.168.0.116:80"]

Then http://ha.jimmy-crib.mesh from any device on the mesh, including the phone on mobile data. Nothing is installed on the Home Assistant box and it needs no configuration; it sees an ordinary connection from jimmy-crib's LAN address.

Publish it on a port other than 80. ha:80->192.168.0.116:80 would work, but the service takes port 80 on the overlay address and the shared-port name router cannot then have it — so every other service on that device loses its bare name. Publishing on 8080 leaves 80 to the router, and you get both http://ha.jimmy-crib.mesh and http://ha.jimmy-crib.mesh:8080.

3. Binding a service to the mesh instead

services forwards a mesh connection to a local port, which means the application still listens where it always did — usually on 127.0.0.1, often with no authentication. There is a stronger arrangement, and it needs no forwarding at all: bind the service to the mesh address. Only members can route to that prefix, so the bind itself is the access control (ADR-026).

sshd, for instance. Take the address from shrooms status:

shrooms status | head -2
network       fd3b:ffe9:f81::/48
self          laptop  fd3b:ffe9:f81:81a7:18bc:69b1:9bb:7e69

# /etc/ssh/sshd_config.d/mesh.conf
ListenAddress fd3b:ffe9:f81:81a7:18bc:69b1:9bb:7e69

Now ssh laptop.mesh works from your phone on mobile data, and ssh is not listening on your LAN, on café wifi, or on the internet at all. No forwarder is involved, the name router is not in the path, and the application needs no awareness of the mesh.

One wrinkle worth knowing before it bites you

That address exists only while the daemon is running, and a service that binds an address which is not there yet fails to start. Two ways round it. Order the service after the mesh:

# /etc/systemd/system/ssh.service.d/mesh.conf
[Unit]
After=shrooms.service

or let the kernel accept the bind regardless, which also survives the daemon restarting underneath it:

echo 'net.ipv6.ip_nonlocal_bind=1' | sudo tee /etc/sysctl.d/99-shrooms.conf
sudo sysctl --system

Peers are not told until you say so

A bound port is discovered rather than declared, so shrooms bound shows exactly what would be announced before anything is:

shrooms bound
MESH     WOULD ANNOUNCE  REACHED AS
default  ssh:22          laptop.mesh:22
default  dev:3000        laptop.mesh:3000

2 would be announced with announce_bound = "true".
They are already reachable by every member; this is about being told.

announce_bound = "true"     # list them in every member's roster

Only sockets on exactly this mesh's overlay address count. One on :: is reachable from every network the machine is on, and listing it as mesh-only would be a lie. The daemon's own ports are excluded: the name router on 80 and 443 and the resolver on 53 are plumbing rather than something you offer. Names are advisory — a short table turns 22 into ssh, and anything unknown is announced by its number as port-4711:4711.

4. Announcing services

Nothing about what a device publishes is announced by default. Each node knows what it runs and nothing about what anyone else runs, which is why a name for a service that does not exist still resolves — to the machine that would run it — where an HTTP request gets a 404 listing the names that do. Turning the announcement on (ADR-023) lets peers show what the mesh offers:

services          = ["immich:2283", "jellyfin:8096"]
announce_services = "true"     # this mesh's peers may see the names
announce_bound    = "true"     # and the ports bound to its address

shrooms status
…
services offered by peers
MESH     NAME      TRY
default  ssh       nas.mesh:22
default  immich    http://immich.nas.mesh
default  jellyfin  http://jellyfin.nas.mesh  (unreachable now)

The list travels in its own control message rather than on the announce, sealed the same way and sent every few minutes. Announces are padded to 512 or 1024 bytes and a credential already forced the larger size; a service list would compete for that budget and be trimmed exactly when there is most to say.

What it does and does not disclose

An announcement is readable by every member and by nobody else — a passive observer on the shard sees a fixed-size ciphertext. And members can already enumerate your services: they listen on the device's overlay address at known ports, that address is in the announce, and scanning a single /128 for a few hundred common ports is seconds of work. So this buys discoverability, not access.

What it does add is intent, in names. "immich", "home-assistant", "jellyfin" tells a reader what you run and what to try, which a port scan does not. On a mesh of your own machines that is worth nothing. On a mesh shared with other people it is an inventory you did not mean to hand over, and the people on it are exactly the ones for whom a list is useful — which is the whole reason it is off by default.

5. Several meshes

A node can belong to several networks at once — your own machines, and one shared with somebody else — without either being able to see the other (ADR-015). Mint a second on a node that already has one:

shrooms init --mesh shared          # a second network on this node
sudo systemctl restart shrooms
shrooms invite --mesh shared        # admit one device to it

Or join one somebody else runs, giving it a name of your own:

sudo shrooms join --invite BEGUZ-N4WOX-PYMTR-CYKWT-QBYSX-U --mesh friends

shrooms status
mesh default  fd69:bd41:d9bc:7fb7:…  fd69:bd41:d9bc::/48  shrooms0   peers 2
mesh shared   fdfb:6ad9:cb3f:2e1a:…  fdfb:6ad9:cb3f::/48  shrooms01  peers 1

Each mesh gets its own key, its own identity, its own WireGuard device, its own interface and its own UDP port — ports allocated upward from listen_port in label order, so a config that names one mesh listens exactly where it always did. Credentials, relay selection, published services and the synthetic IPv4 block are all per mesh too.

The separate identity is not hygiene. The overlay's host bits are a hash of the device key, so reusing one identity would carry the same 80-bit suffix into every mesh and let anyone in two of them correlate you. It is also forced: WireGuard allows one preshared key per peer and ours is per mesh, so a peer you share two meshes with cannot be one entry on one device.

In the config, meshes are prefixed keys rather than TOML tables:

mesh.home.key         = "P27KNQ2…"
mesh.home.relay       = "true"
mesh.home.services    = ["immich:2283"]
mesh.shared.key       = "D4R5TBD…"
mesh.shared.admin_keys = ["EGRWTGUF…", "3Y5HMGWB…"]
mesh.shared.enabled   = "false"    # a member of it, not running it right now

--mesh is how you say which one a command means: init, join, invite, admin init, admin issue and admin renew all take it, and invite refuses to guess when a node has more than one. shrooms status has no such flag — it shows every mesh, and grows a MESH column when there is more than one to show.

Names take a mesh label when they need one:

ping vps.shared.mesh     # unambiguous, and canonical
ping nas.mesh            # fine while only one mesh has a `nas`

The short form is answered only when exactly one mesh has that name, so a node with one mesh sees no change, and ambiguity removes the short name rather than silently picking a network for you. The label is local to your node and deliberately never announced: there is no authenticated channel to distribute it, so Alice calling it shared while Bob calls it alice-nas is the only thing that can be true without an authority.

Seeing them, and switching one off

shrooms mesh lists every mesh this device belongs to, running or not — which is the whole point of it. status reports what the daemon is doing, and a mesh that has been switched off has no instance to report, so it disappears from the one list that could have switched it back on.

sudo shrooms mesh
MESH     STATE  PREFIX                CREDENTIAL  RELAY  SERVICES
default  on     fd3b:ffe9:f81::/48    not needed  yes    2
test     OFF    fd7b:15fb:5ec1::/48   held

test is switched off. It keeps its key and credentials:
  sudo shrooms mesh enable test && sudo systemctl restart shrooms

Switching one off is the reversible half of leaving it: the key stays, the credential stays, it simply does not run. Leaving discards the config entry and needs a fresh invite to undo — which is why the two are different words.

It reads the config directly rather than asking the daemon, so it works when the daemon is down, which is when you most want to know what is in the file. That also means sudo: the file is full of network keys.

6. Invites and membership

Adding a device is two commands, and the network key never appears on a screen. On a machine that is already a member, with its daemon running:

laptop shrooms invite
        Invite valid for 15m0s. On the joining device:
          shrooms join --invite BEGUZ-N4WOX-PYMTR-CYKWT-QBYSX-U
        [QR code]
        Waiting...

vps    sudo shrooms join --invite BEGUZ-N4WOX-PYMTR-CYKWT-QBYSX-U --name vps
        Asking to join as "vps"...
        Enrolled. Credential serial 1786439411, expires 2026-09-10T11:03:51+02:00.

The token is 128 bits, good for one device and fifteen minutes. Both ends derive from it where to meet and what to encrypt with, so a wrong token addresses a topic nobody answers rather than a guess anyone can grind against. What comes back is the network key, the admin public keys and a credential issued to that device's keys, sealed to the device that asked (ADR-017). The phone joins the same way: scan the QR, or paste the token into the one field on the join screen.

"Used once" needs no consensus, because an invite is answered only by the machine that issued it — one machine's local decision. The price is that the inviter's daemon must be running while the other device joins, which is also the point, since it means a human is present. If nobody answers, the response was lost; run invite again. Answering twice would quietly turn a single-use token into a reusable one.

Credentials expire in thirty days

Membership is an admin-signed credential naming one device, so a leak costs that device rather than the mesh (ADR-018). Expiry is what bounds the case a gossip bus cannot help with: an attacker can suppress a revocation it cannot forge, and nobody can suppress a clock. The lifetime lives inside the signature, so a device cannot extend its own membership.

Renewal is a sweep rather than a ceremony per device. From the machine that holds the admin key, against a running node:

shrooms admin renew --dry-run
  laptop           612h0m0s left
  vps              203h0m0s left
  phone            unknown expiry

shrooms admin renew         # signs for everything inside ten days of expiry

--within widens the window, --all renews everybody however long they have left, and --life sets how long the new credentials last. Each one is handed to the local daemon to deliver over the mesh, the same way a revocation travels; a device keeps whichever credential lasts longer. Anything that could not be delivered is printed so you can hand it over by any other means — a credential is public and holds nothing secret.

To withdraw a device before its credential expires:

shrooms keys                # on the device: its public keys, hex
device  8f2c1d…
tunnel  4a90b3…

shrooms admin revoke --device 8f2c1d…
Revoked 8f2c1d…, serial 1786439411 and below.

Published. Every node that sees it verifies the signature itself,
drops the device, and passes it on — so a node that was offline
learns it from whoever is up.

The admin key is needed only to enrol and revoke, so it can live offline; it is encrypted at rest with a passphrase, and --no-passphrase exists for a file you keep on an encrypted volume. The daemon has the network key and the connection and never sees the admin key; the CLI has the admin key and never sees the network key. Neither half can admit a device by itself.

7. Relays and reachability

Two peers connect directly whenever a probe gets through. When neither can reach the other — both behind NAT, no path that answers — traffic goes through a relay, which is any mesh node with a reachable address:

relay = "true"      # in that node's config; restart it

Nothing else needs configuring. The relay advertises itself in its ordinary announce and every other node picks it up, so there are no relay addresses to distribute or keep up to date, and its IP can change freely. A relay forwards ciphertext it cannot read.

A relay is chosen only among peers that announce that they relay, are believed online, and have a candidate address that has answered a probe. An unverified candidate would blackhole every relayed packet with nothing to show for it, and unlike a direct endpoint there is no second chance: WireGuard cannot relearn a relay path from an inbound packet. Among those, the lowest device ID wins — deliberately not the lowest RTT, which each side measures differently and would therefore disagree on, and both ends must pick the same relay for relaying to work at all. relay_addr pins one and overrides discovery, which is rarely what you want.

What the status output means, per peer:

• up 1m with a plain host:port endpoint — direct, and carrying traffic.
• no handshake or stale — announced and not yet usable; shrooms paths says which candidates answered.
• an endpoint reading relay:…@128.140.55.128:51820 — going through a relay.
• offline — nothing announced recently, so nothing to probe.

Asking the router for a way in

A node behind NAT normally learns its public address by reflection: it probes a peer and the pong echoes back what that peer saw. That needs a peer outside your NAT. On a mesh whose members are all in one house there is none, so every announced candidate is a LAN address and the mesh works from the sofa and not from the street.

So the daemon asks the router at startup — PCP first, then NAT-PMP — and treats whatever comes back as one more candidate (ADR-024). On 2026-08-12 a laptop behind a domestic NAT was granted 178.213.45.235:51821 over NAT-PMP and announced it; a phone on mobile data dialled it and established a direct tunnel, on both meshes, with no relay. That is not hole punching. The router was asked for a mapping and gave one.

journalctl -u shrooms | grep "port map"
… port mapped by the router  mesh=default external=178.213.45.235:51821 proto=natpmp lifetime=2h0m0s

port_mapping = "false"    # if you would rather it did not ask

A wrong answer is cheap, because every candidate is probed before it is used: an address from a confused router simply fails to probe and is discarded. A mapping is a lease rather than a fact — a router that reboots forgets everything — so it is renewed at half the granted lifetime, and a router that refuses is asked again in half an hour rather than constantly. UPnP-IGD is in the design and is not implemented; PCP and NAT-PMP are.

8. Bandwidth and node mode

A node joins a public, shared rendezvous cluster, and by default it relays for that cluster. That is the neighbourly setting and it is not free. Measured on a home connection, idle — no VPN traffic at all — over ten minutes each:

	Core (default)	Edge
received	15.6 MB/h	1.9 MB/h
sent	4.7 MB/h	1.6 MB/h
total	20.3 MB/h — 0.49 GB/day	3.4 MB/h — 0.08 GB/day
connections opened per 10 min	139	90

Almost none of that is yours. Of 745 messages the Core node handled, 693 belonged to another application entirely and 14 were on this mesh's shard. A Core node subscribes to every shard in the cluster, so it carries the cluster, not your mesh.

By comparison shrooms's own traffic is nothing: a 512- or 1024-byte announce every 45s — the larger size once a mesh uses credentials — a 104-byte probe per working path every 5s, and a WireGuard keepalive every 25s. A three-node mesh sits well under 1 MB/h. The rendezvous relay costs 20–30× the protocol it exists to serve.

mode = "Edge"      # subscribe and forward nothing
mode = "Core"      # relay for the network (default)

Edge uses filter and lightpush instead of gossipsub. It was verified receiving: a Core publisher's message reached an Edge subscriber 281 ms later, same message hash. Nothing else in the config changes and the mesh behaves identically — this is about what you carry for other people, not about how your own traffic moves.

Use Edge on anything metered or battery-powered, and keep Core where bandwidth is flat and the machine is always on. A VPS is the right place to contribute relay capacity; a phone is not — half a gigabyte a day of other people's traffic on a mobile plan is the whole argument. On Android the setting is on the main screen rather than in a config file.

Where to go next