# Deployment

> Run the recorder as a systemd service behind a caching reverse proxy, and size the machine.

The recorder is one binary with no external dependencies. In production you run it as a service, bind it to localhost, and put a reverse proxy in front for TLS, compression and caching.

## Build and install

```bash
cargo build --release -p flowscope-recorder
sudo install -m 0755 target/release/flowscope-recorder /usr/local/bin/
sudo useradd --system --home /var/lib/flowscope --create-home flowscope
```

## systemd

```text title="/etc/systemd/system/flowscope-recorder.service"
[Unit]
Description=Flowscope recorder
Wants=network-online.target
After=network-online.target

[Service]
User=flowscope
Group=flowscope
Environment=FLOWSCOPE_RECORD_MARKETS=Aggregate:BTC,Aggregate:ETH
Environment=FLOWSCOPE_RECORD_DIR=/var/lib/flowscope
Environment=FLOWSCOPE_RECORD_LISTEN=127.0.0.1:8787
Environment=FLOWSCOPE_RECORD_RETENTION_DAYS=365
Environment=RUST_LOG=info
ExecStart=/usr/local/bin/flowscope-recorder
# The recorder saves everything on SIGINT. SIGTERM would skip the final save.
KillSignal=SIGINT
TimeoutStopSec=60
Restart=always
RestartSec=5
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/flowscope
PrivateTmp=true

[Install]
WantedBy=multi-user.target
```

```bash
sudo systemctl daemon-reload
sudo systemctl enable --now flowscope-recorder
journalctl -u flowscope-recorder -f
```

> [!IMPORTANT]
> Keep `KillSignal=SIGINT`. The recorder listens for an interrupt to end open Hyperliquid runs and save every market. With the default SIGTERM it stops without that final save and loses up to one save interval.

## Reverse proxy with caching

The recorder's HTTP server is deliberately minimal: one request per connection, no keep-alive, no TLS, no compression, no authentication. A reverse proxy adds all of that and absorbs repeat traffic.

The recorder already tells caches what they may keep:

| Response | `Cache-Control` |
|---|---|
| A past day's segment | `public, max-age=31536000, immutable` |
| Today's segment, day lists, `/v1/markets` | `no-cache` |
| `/v1/hl/…` | `no-cache` |
| `/health`, errors | `no-store` |

So a cache that honours these headers stores past days forever and revalidates the rest. Past-day segments are the bulk of the traffic: a client opening a week of 1-minute history fetches seven segments, six of them immutable.

### nginx

```text title="/etc/nginx/conf.d/flowscope.conf"
proxy_cache_path /var/cache/nginx/flowscope levels=1:2 keys_zone=flowscope:10m
                 max_size=20g inactive=30d use_temp_path=off;

server {
    listen 443 ssl http2;
    server_name data.example.com;

    ssl_certificate     /etc/letsencrypt/live/data.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/data.example.com/privkey.pem;

    gzip on;
    gzip_types application/octet-stream text/plain;

    location / {
        proxy_pass http://127.0.0.1:8787;
        proxy_http_version 1.1;
        proxy_set_header Connection "";

        proxy_cache flowscope;
        proxy_cache_lock on;
        proxy_cache_use_stale updating error timeout;
        add_header X-Cache-Status $upstream_cache_status always;
    }
}
```

The recorder sends `Access-Control-Allow-Origin: *` itself, so the proxy does not need to add CORS headers. See [Caching and CORS](/docs/api/caching-and-cors).

### Caddy

```text title="Caddyfile"
data.example.com {
    encode gzip zstd
    reverse_proxy 127.0.0.1:8787
}
```

Caddy terminates TLS and compresses. It does not cache by itself; put a CDN in front if you need an edge cache.

### A CDN

Any CDN that respects `Cache-Control` works. Past days are served from the edge with an infinite lifetime; today's segment and the Hyperliquid endpoints pass through to the recorder.

Flowscope clients do not read a recorder directly: they take every market and its history from a data hub. The hub (`flowscope-hub`) runs this same recorder inside and takes the same options and `FLOWSCOPE_RECORD_*` variables after its own, so on a server you run it in place of `flowscope-recorder`, behind the same proxy. The proxy must also pass websocket upgrades for `/v1/ws` (Caddy does this by default; in nginx add `proxy_set_header Upgrade $http_upgrade` and `Connection "upgrade"` for that path). Then point clients at the hub's public URL:

```bash
FLOWSCOPE_HUB_URL=https://hub.example.com FLOWSCOPE_HUB_KEY=fsk_… ./target/release/flowscope
```

A hub asks for keys unless started with `--open` (local use only): `flowscope-hub keys add --tier app --owner NAME` prints one. The website passes `/hub` to the hub for the browser app: `flowscope-site serve --hub-url http://127.0.0.1:8787 --hub-key KEY` (or `FLOWSCOPE_SITE_HUB_URL` and `FLOWSCOPE_SITE_HUB_KEY`). A bare recorder behind the proxy still serves its history to operators and your own tools.

## Health checks

`GET /health` answers `200 ok` with `Cache-Control: no-store`. Use it for load balancers and uptime checks.

```bash
curl -fsS https://data.example.com/health
```

For a deeper check, look at the recorder's log: once a minute it reports per-market coverage (full, partial, loading, unavailable, missing minutes and whether the current minute is live) and Hyperliquid status.

## Sizing

One recorder process handles a few hundred markets on 2–4 cores. For comparison, the engine runs sixty heatmap charts on one core in the simulator. Memory is bounded by the engine's budget of 3 GB, shared by all markets.

| Item | Per market-day | 500 markets, 1 year |
|---|---|---|
| Minute footprints (raw) | ~0.2 MB (BTC, measured), ~0.1 MB (quiet altcoin) | ~20–35 GB raw, ~6–10 GB with zstd |
| Heatmap runs at full resolution | 20–100 MB (BTC), 1–5 MB (altcoin), estimated | 90 days ≈ 1–3 TB |
| Websocket ingress | ~2–5 GB (BTC, all streams), estimated | ~100–300 Mbit/s per shard set |

Heatmap runs are not stored by the recorder today beyond its exit snapshot; the row shows what serving them will cost. The zstd figure is for the planned compressed archive. See [Architecture](/docs/recorder/architecture).

Serving is static-file traffic. A client opening a 1-minute chart over a week of BTC downloads about 1.5 MB of footprints, well under 1 MB compressed.

### Memory

The engine's memory budget is fixed at 3 GB for all recorded markets together, plus up to 64 MB of level runs and two days of footprint minutes per Hyperliquid coin. Leave headroom above that for the operating system and the page cache that serves segments.

## Upgrades

Stop the service, replace the binary, start it again. Day segments of version 1 are still read by current builds. The minutes while it was down are not recorded; nothing is backfilled from venue trade history.