RoboHub

This page was generated from the source of RoboHub. Edit the source, not this page.

Secure telemetry

One owner for telemetry, and every link between the parts that produce, carry and
use it authenticated, fresh and bounded in time. This page is the design, and all
of it is implemented; architecture/security is the shorter account of who
may do what, and architecture/reading follows one reading through it.

The parts and who trusts whom

 board ──frames──▶ gateway ──outbox──▶ mesh ──/mesh/api──▶ aquarium, energy, ...
   K_dev, session     K_gw, RP of keeper   owner, RP of keeper     RP of keeper
                                  ▲               ▲                       ▲
                                  └───── short-lived tokens from keeper ──┘
  • keeper is the only issuer. It holds the one ES256 (P-256) private key and
    publishes the public key. Nothing else reads its files or holds its key.
  • mesh is the only owner of telemetry: it alone ingests from gateways, stores
    readings, names sources and actuators, and brokers commands. No other
    application opens its database, and no other application has a telemetry store.
  • gateway carries frames between boards and mesh. It authenticates boards by
    their device key and mesh by keeper's tokens.
  • aquarium, energy and any later application hold domain state only. They read
    sources and send actions through /mesh/api, with a token keeper issued.

Users: one session per application

A browser signs in at keeper and comes back to an application with a one-use
ticket. The application then keeps its own session:

  • the cookie is named for the application's audience (rho_session_<aud>), so
    two applications on one host do not overwrite each other;
  • its value is a keeper-signed session token for that audience, valid for at most
    eight hours, verified with keeper's public key; there is no shared session file;
  • the token's audience must be the application's own; a token for another
    application is refused.

Services: short-lived tokens

A service that calls another (an application calling mesh, mesh calling a
gateway) holds a client credential keeper registered for it: an id and a secret,
stored by keeper as a hash and by the service in its workspace with mode 0600. It
exchanges the credential at keeper for a token:

POST /auth/service-token   client_id, client_secret, audience, scopes
→ { token, expires_in }    ES256, aud = the callee, scp = granted scopes, ttl ≤ 10 min

The caller caches the token and renews it at 80 % of its lifetime. A callee
verifies it offline with keeper's public key; nothing is looked up per request.

Every relying party, generated by Rho's auth middleware:

  • removes X-Rho-Subject, X-Rho-Scopes and X-Rho-Aud from every incoming
    request before anything else, exempt path or not, and sets them only from a
    token it verified;
  • requires the token's audience to be its own;
  • grants a route group only to the scopes it declares (mesh:<project>,
    gateway:<id>:read, gateway:<id>:command).

Gateway and mesh

The gateway is a relying party of keeper with the audience gateway. It knows its
own id (GATEWAY_ID, the mesh's name), and the scopes name it:
gateway:<id>:read for telemetry, the outbox and diagnostics, gateway:<id>:command
for switching and commands; gateway:<id>:*, gateway:* and * hold both. There
is no static token. Mesh may be issued gateway:*, but asks each gateway's token for
gateway:<id>:*, so a token taken from the network opens that gateway and no other.
A gateway reached over a network serves HTTPS when the mesh names its certificate
and key (gateway_tls_cert, gateway_tls_key).

The outbox keeps a cursor per consumer (the token's subject) and retains
records by age and size, rather than deleting what the first consumer
acknowledged. In practice mesh is the one consumer, and this makes a second one
(a backup, a debugger) harmless.

Boards and gateways: authenticated frames

Keys

  • Every gateway has a 256-bit gateway key K_gw, kept as gateway.key in the
    mesh's folder. Mesh makes it the first time it is needed and installs it with
    the gateway (the same deployment that starts it over SSH or ADB), in a file
    created 0600 on the gateway's host before anything is written to it, with the
    node manifest nodes.txt beside it.
  • Every board has a 64-bit id uid, minted by Mesh (node.<n>.id in the plan)
    and built into its firmware; a mesh linked below has one as its parent's relay
    (relay_uid).
  • The device key is derived, not stored: K_dev = HMAC-SHA256(K_gw, "rh-dev1" ‖ uid)[0:16]. Mesh derives it to build the firmware; the gateway
    derives it when the board says hello. It crosses a network only to a Linux host
    node, which fetches it from Mesh over HTTPS with its keeper credential
    (architecture/enrollment).
  • That is the key's epoch 0. A rotated key is a later epoch n:
    K_dev(n) = HMAC-SHA256(K_gw, "rh-dev2" ‖ uid ‖ n as u32 LE)[0:16]. The node
    manifest says which epoch each board should be at; see
    key rotation.
  • The gateway admits only the boards its node manifest lists and does not mark
    revoked; without a manifest it admits any board whose key derives.

Session handshake

board   → HELLO   uid(8) ‖ nonce_b(8) ‖ tag(8)
                  tag = HMAC-SHA256(K_dev, "rh-hello1" ‖ uid ‖ nonce_b)[0:8]
gateway → SESSION uid(8) ‖ session_id(2) ‖ nonce_g(8) ‖ lifetime_s(2) ‖ tag(8)
                  tag = HMAC-SHA256(K_dev, "rh-session1" ‖ uid ‖ nonce_b ‖ session_id ‖ nonce_g ‖ lifetime_s)[0:8]
both    K_up   = HMAC-SHA256(K_dev, "rh-up1"   ‖ nonce_b ‖ nonce_g)[0:16]
        K_down = HMAC-SHA256(K_dev, "rh-down1" ‖ nonce_b ‖ nonce_g)[0:16]

HELLO is message 0x0011 and SESSION 0x0012; neither carries firmware,
platform or time. A session lasts lifetime_s (one hour) and then the board says
hello again. The gateway keeps its live sessions and their counters on disk, so a
restart neither loses them nor accepts a frame recorded before it, and it does
not answer a hello whose nonce it has just seen from that board.

Secured frame

The framing stays what it is (COBS, the six-byte header, the check), so a parser
finds boundaries as before. A secured frame adds a trailer inside the payload area:

header(6) ‖ session_id(2) ‖ counter(4) ‖ payload ‖ tag(8) ‖ check(2)
tag = HMAC-SHA256(K_dir, msg_id(2) ‖ tx_id(2) ‖ session_id ‖ counter ‖ payload)[0:8]

A secured frame's message id has its high bit set; the tag covers the message id
and transaction id of the header, not its length.

The receiver drops a frame whose session is unknown or expired, whose counter is
not greater than the last one accepted in that direction, or whose tag does not
match. Truncating HMAC-SHA256 to 64 bits is standard for a low-rate link; at the
rates a sensor sends, forging a tag by guessing is not a practical attack within
a session's lifetime.

This provides authenticity, integrity, replay protection and freshness. It does
not encrypt: readings and commands are not secret, but they must not be forged or
replayed. Encryption (an AEAD with the same keys and counters) can be added
without changing the handshake.

Sessions across a gateway restart (SESSION_RESET)

The gateway keeps its sessions on disk, with each counter whenever it has moved 64
past what was kept, and restores a session's counters 128 past the kept marks, so
no frame recorded before a restart opens after it. A board does not see the
restart and goes on counting below that mark, so the gateway asks it, sealed under
the restored session's K_down with a counter past anything the board accepted:

gateway → SESSION_RESET  [reason(1)]       message 0x0016, reason 1 = the gateway restarted

on the serial link as soon as it is open, and through a relay (inside RELAYED)
or on an uplink connection on the board's first refused frame from it — at most
once in 10 s per session. The board drops the session and says HELLO at once; its
first frame under the new session ends the restored one. Unsealed, or under
another session, the board refuses it like any frame.

Key rotation (REKEY)

A board's key is rotated without flashing it. Mesh's Rotate key raises the
board's epoch in the plan and the manifest. Nothing in a HELLO says which epoch it
was made under, so the gateway tries the manifest's epoch for that uid, then the
one before it (the grace epoch) while the board has not confirmed the new one;
the session records the epoch it was opened under. A board whose session is under
the grace epoch is sent, sealed under that session's K_down:

gateway → REKEY      new_epoch(4) ‖ wrapped(16)                         message 0x0014
                     wrapped = K_dev(new) XOR HMAC-SHA256(K_dev(current), "rh-wrap1" ‖ session_id ‖ counter)[0:16]
board   → REKEY_ACK  new_epoch(4) ‖ proof(8)                            message 0x0015
                     proof = HMAC-SHA256(K_dev(new), "rh-rekey-ok" ‖ uid ‖ new_epoch)[0:8]

with session_id and counter the REKEY frame's own and every integer
little-endian. Both are only ever sent sealed. The board stores the new key, with
its gateway's key scope (RH_KEY_SCOPE, so a board later moved to another mesh
ignores it), in its keystore's inactive slot (robohub/keystore.h: two A/B slots with a CRC and a
sequence number; a Linux host node rewrites its key file, <epoch>:<hex>, 0600,
atomically), answers under the current session, keeps the current key for the
rest of the session and uses the new one from its next HELLO. The gateway checks
the proof and records the board's confirmed epoch with its sessions; from then
on it refuses the old epoch for that uid and drops any other session that uid
holds under it. A HELLO that authenticates under the manifest's epoch confirms it
as well, so a firmware built after a rotation — which carries the current epoch's
key and RH_DEVICE_KEY_EPOCH — needs no REKEY. The gateway writes rekey.txt
beside the manifest (uid|epoch|status|time, status confirmed or cannot) and
serves it as GET /api/v1/keys, which is what Mesh shows. A board whose platform
has no key store answers epoch 0 and a zero proof: it cannot rotate, which Mesh
shows on the board, and it stays in its grace epoch until it is reflashed.

Revocation after a compromise is: rotate, and once the board has confirmed, the
stolen old key no longer opens a session. The caveat is that there is no forward
secrecy: whoever already holds the old key and recorded the REKEY frame can unwrap
the new key. A key exchange (X25519) would remove that and is out of scope; for a
key known to be stolen, reflash the board as well, or rotate while no one can be
listening. Only one epoch of grace is kept, so Mesh does not rotate a board again
before it confirmed the last rotation, and a board two epochs behind its manifest
is refused.

A relay link (gateway to gateway) uses the same frames. The relay is a board of
its parent, with uid its relay_uid, and its key is derived from the parent's
K_gw; the parent's manifest lists it as a relay with the areas it may
announce.

Live UI: HTMX and SSE

Applications stay small by staying server-rendered, as build/reactive-ui
lays down: a page opens one SSE connection, the server sends HTML fragments, and
HTMX swaps them in. Nothing in a browser polls, and nothing in a browser talks to
mesh or a gateway.

mesh ──SSE (service token, per project)──▶ app server ──SSE fragments──▶ browser (HTMX)
     ◀──POST action (service token)───────            ◀──hx-post───────
  • Mesh streams per project. /mesh/api/projects/{project}/telemetry/stream
    serves that project's events only, with a snapshot of current values first, so
    a reconnect starts from the present.
  • An application relays, it does not store. Its server holds one
    subscription to mesh's feed for its project with its service token, renders the events
    that concern its widgets as fragments, and serves them on its own page stream.
    The browser never sees a mesh URL or a token.
  • Actions are forms. A button or switch is an hx-post to the application,
    which checks the user's session and forwards the action to mesh with its own
    service token. Mesh brokers it to the gateway, and the new state comes back on
    the stream. No state is guessed in the page.

The relay is the runtime's rho_sse_follow: it follows a remote SSE stream into a
local file with a service token, for as long as the application runs, and the
application serves its own page stream from files (sse_events). When the
installation runs the MQTT broker, Mesh also relays each project's feed to the topic
mesh/<project>/telemetry, and an application's gateway provider can follow that
instead (transport: auto | bus | http in its binding).

Source identity

A source or actuator is named by (gateway_id, uid, signal), which survives a
reordered plan, a renamed folder and a renumbered node. Mesh keeps the mutable
facts (area, node, label, the firmware's module.id) in a telemetry_source
table and keys readings by the source; the numeric area and node remain only as
routing on the wire.

Where it is implemented

Part of the designCode
Header stripping, per-application sessions, audience and scope checksRho's auth middleware, generated into every relying party
Service tokens and the client registrythe keeper (/auth/service-token) and the rho auth component
One telemetry owner, streamed per projectMesh, with its store as the mesh-telemetry port
Applications reading Mesh with service tokens and relaying its feedrobohub-provider-mesh and the widgets module
The gateway as a relying party; the outbox with a cursor per consumerrobohub-gateway (security.rho, outbox.rho)
Authenticated framesrobohub-transport (secure.h), the gateway (secure.rho), Mesh's firmware generation, relay links
Key epochs and REKEYrobohub-transport (secure.c: unwrap, store, ack), robohub-common's keystore.h and each platform's store, the gateway (keys.rho), Mesh (node.<n>.epoch, Rotate key)
Source identityMesh's telemetry_source table; readings keyed by source