Skip to content

Relay Tuning ​

The Hub protects its abuse-prone entry points with per-surface rate limiters, enforces per-user relay bandwidth quotas, a concurrent-stream cap, and a per-user bandwidth throttle, and records relay observability metrics you can query for capacity planning. All of it is operator-configurable.

Rate limiting ​

Each surface has its own rate limiter (a single login-grade limiter is wrong for everything but login). Five surfaces use a per-worker in-memory limiter; login alone is shared and DB-backed (see Per-worker vs global below). The rate_limit section of config/server.php sets each surface's {max, window} plus a shared key-count cap; every value is env-overridable. Absent keys fall back to the defaults below.

Env overrideDefault (max / window s)Surface / key
PHLIX_HUB_RATELIMIT_CAP10000Max distinct keys tracked per limiter (memory ceiling)
PHLIX_HUB_RATELIMIT_LOGIN_MAX / _LOGIN_WINDOW5 / 900Login attempts, keyed by identity
PHLIX_HUB_RATELIMIT_PROXY_MAX / _PROXY_WINDOW600 / 60Relay proxy, keyed proxy:{userId} (checked after the auth gate; generous so normal HLS/DASH segment bursts never trip)
PHLIX_HUB_RATELIMIT_HEARTBEAT_MAX / _HEARTBEAT_WINDOW30 / 60Server heartbeat, keyed heartbeat:{serverId} (after the enrollment JWT is validated)
PHLIX_HUB_RATELIMIT_JWKS_MAX / _JWKS_WINDOW120 / 60/.well-known/jwks.json, keyed jwks:{ip}
PHLIX_HUB_RATELIMIT_RELAY_CONNECT_MAX / _RELAY_CONNECT_WINDOW10 / 60:8802 server relay-connect handshake (WebSocket), keyed by IP
PHLIX_HUB_RATELIMIT_CLIENT_MOUNT_MAX / _CLIENT_MOUNT_WINDOW30 / 60:8803 client-mount handshake (WebSocket), keyed by IP (before auth)

What a client sees when it trips ​

  • HTTP surfaces (proxy, heartbeat, JWKS, login) return 429 Too Many Requests with a Retry-After header and body {"error":"Too Many Requests","code":"rate_limited"}.
  • WebSocket handshakes (:8802 relay-connect, :8803 client-mount) can't send an HTTP status after the upgrade, so they reject the connection with WS close code 1013 (Try Again Later). The peer should back off and retry.

Per-worker vs global (important) ​

Most thresholds are enforced per worker process, not globally:

  • The :8802 and :8803 relay workers are count=1, so for relay_connect and client_mount the per-worker limit is the global limit — these are the primary DoS surfaces, so this is intentional.
  • proxy, heartbeat, and jwks run across HUB_WORKERS HTTP workers (default 2), so their effective soft-global limit is roughly max × HUB_WORKERS. Size your override accordingly, or reduce HUB_WORKERS. A strict global cap for these would require a shared store (Redis/DB) and is planned future work.

login is the exception — it is genuinely global. Its bucket is backed by the shared login_rate_limit DB table (migration 040_login_rate_limit) — the one DB-backed profile — so every HTTP worker shares one counter per key. This means the 5 attempts / 900 s login budget is actually 5/900, not the ~5 × HUB_WORKERS / 900 (≈20/900 with HUB_WORKERS=4, where the first 429 landed near attempt ~9) it was when login was worker-local like the other surfaces. This closes the one surface where the per-worker weakening was a real brute-force concern (HB-4.6 "Option B"). The login_rate_limit table holds one row per bucket key (an INSERT … ON DUPLICATE KEY UPDATE counter with a TTL-driven reset_at); a bounded sweep on each recorded attempt reclaims expired rows, so it stays small with no operator maintenance.

Do not set PROXY_MAX too low

The proxy limiter is keyed per user and sized generously on purpose: a single HLS/DASH playback session fires many short segment requests in a burst. A too-low PROXY_MAX will break normal playback. Test with a real multi-segment stream before lowering it.

Per-user bandwidth quotas & concurrent-stream cap ​

The relay proxy meters the real streamed bytes delivered to each client and rolls them up per user (table relay_user_quotas). Two byte caps and one concurrency cap are enforced; 0 means unlimited (the default for existing rows):

ColumnMeaningEnforcement
quota_bytes_inMonthly download cap (bytes streamed to the user)Over cap → 503 quota.exceeded at proxy admission
quota_bytes_outMonthly upload cap (request bytes from the user)Over cap → 503 quota.exceeded
max_concurrent_streamsMax simultaneous relay streams for the userOver cap → 503 stream.limit, request never occupies a slot

Concurrent-stream cap is per HTTP worker

The concurrent-stream counter is held in memory in the HTTP worker that admits the stream, so the cap is enforced per worker — the effective soft-global limit is max_concurrent_streams × HUB_WORKERS. A strict global cap needs a shared store (future work). Byte-cap accounting, by contrast, is persisted to the DB and is global.

Managing quotas over HTTP ​

MethodPathWhoNotes
GET/api/v1/me/bandwidthAny authenticated userRead your own current-period usage + caps
GET/api/v1/admin/users/{id}/bandwidthAdminRead any user's usage + caps
PUT/api/v1/admin/users/{id}/quotaAdminSet a user's download/upload byte caps + concurrent-stream cap

PUT .../quota validates its body (non-negative integers; byte caps ≤ 1 PiB; max_concurrent_streams ≤ 1000; 0 = unlimited) and records the change in the audit log (user.quota.set). The response is the user's rollup ({user_id, bytes_in, bytes_out, quota_bytes_in, quota_bytes_out, max_concurrent_streams, throttle_bps}) — a zeroed rollup with unlimited caps if the user has no row yet. (throttle_bps reflects the per-user throttle documented below; the two bandwidth GETs surface it too.)

Requires migration 038_relay_user_quotas_concurrency (adds max_concurrent_streams).

Per-user bandwidth throttle ​

Independent of the monthly byte caps above, each user has a relay bandwidth throttle — a hard cap on the rate (not the monthly total) at which the Hub relays their remote stream. Enforcement is a per-connection token bucket applied on both relay transports: the native-client WebSocket relay path (:8803 client mount), where frames are paced to the user's cap as they are delivered, and the browser HTTP-over-relay proxy (streaming response) path, where response-body fragments are paced through the same bucket before each send(). The same durable per-user cap and the same 0 = Unlimited bypass (no token bucket, no pacing) apply on both paths. Because the bucket is per connection/channel, other users multiplexed over the same server tunnel are unaffected — one throttled viewer never slows anyone else on that tunnel.

Levelthrottle_bps
Unlimited0 (bypasses throttling entirely — no token bucket, no pacing)
1 Mbps1000000
3 Mbps3000000 (default)
5 Mbps5000000
10 Mbps10000000
20 Mbps20000000
50 Mbps50000000

Every user starts at the 3 Mbps default; 0 = Unlimited turns the throttle off for that user completely (the relay send path is byte-identical to an unthrottled connection).

Durable per-user setting — not a monthly quota

The throttle is a durable per-user setting stored in its own table (relay_user_settings, migration 043_relay_user_settings). It persists indefinitely and does not reset each month. This is deliberately distinct from the monthly byte-cap quota above, which is period-scoped (rolled up per calendar month in relay_user_quotas). Setting a throttle never touches a user's quota, and setting a quota never touches the throttle.

Raise the cap for HD / 4K viewers

Because 3 Mbps is the enforced default, an unconfigured user's remote stream is capped at 3 Mbps — comfortable for SD/720p but tight for 1080p and too low for 4K. Raise the level (or set Unlimited) per user for viewers who need higher quality.

Managing the throttle over HTTP ​

MethodPathWhoNotes
PUT/api/v1/admin/users/{id}/throttleAdminSet a user's relay bandwidth throttle level

PUT .../throttle is admin-gated and records the change in the audit log (user.throttle.set). Its body is {"throttle_bps": <level>} where <level> must be one of the allow-listed values in the table above (0, or 1/3/5/10/20/50 Mbps expressed in bps); any other value — negative, fractional, non-numeric, or off-list — is rejected with 400 invalid_throttle. On success it returns the same bandwidth rollup as GET .../bandwidth (now including throttle_bps). The current throttle is also surfaced on both GET /api/v1/admin/users/{id}/bandwidth and GET /api/v1/me/bandwidth.

Requires migration 043_relay_user_settings (the durable per-user throttle store).

From the admin console (Users page) ​

Both the monthly quota and the throttle are editable without touching the API, from the Users page of the Hub's web admin console (/app/admin/users). Every user row has a Relay action that opens a Relay limits dialog with two sections:

  • Bandwidth throttle — a dropdown of the fixed levels above (Unlimited / 1 / 3 / 5 / 10 / 20 / 50 Mbps). Choosing Unlimited sends throttle_bps = 0.
  • Monthly quota — download and upload caps entered in GiB (converted to bytes at 1024³; 0 = unlimited), plus a max concurrent streams field (0 = unlimited). The dialog also shows the bytes already used this period.

Save issues only the PUTs for the section(s) you actually changed (PUT …/throttle and/or PUT …/quota), so an unchanged section is never re-written or re-audited. The client enforces the same bounds as the API (byte caps ≤ 1 PiB, streams ≤ 1000; the throttle is limited to the allow-listed levels) and blocks an out-of-range save before it reaches the Hub.

Hub-only control

The Relay action renders only in the Hub admin console (phlixConfig.app === 'hub'); the media server does not serve these endpoints, so the control never appears in the server's admin UI. It shipped in @phlix/ui 0.98.30, and the Hub's web-ui/package.json now pins a later release, so a Hub built from current sources has it. If you are running an older deployment, it appears once the pin is ≥ 0.98.30 and the SPA bundle under public/assets/app/ is rebuilt. The underlying HTTP API (above) works regardless of the SPA version.

Relay observability metrics ​

The Hub records relay metrics into the time-bucketed metrics_rollup table (there is no/metrics scrape endpoint — see Monitoring & Alerting). Query them with SQL for capacity planning and alerting:

ColumnTypeMeaning
relay_pending_requestsgaugeIn-flight proxied requests
relay_reply_dropscounterReply frames dropped (no matching in-flight request)
relay_error_503counter503s returned by the proxy (offline server / no tunnel)
relay_error_504counter504s (relay reply timeouts)
relay_cancelscounterHTTP_CANCEL frames the Hub sent a server after a client abandoned a request
relay_decode_buffer_bytesgaugeFrame-decoder buffer high-water
relay_latency_h_le_10 … _h_gt_5000histogramPer-request latency buckets (first-byte + total observations)

relay_cancels (migration 039_relay_cancel_metric) counts cancellations sent when a browser abandons an in-flight stream. The HTTP_CANCEL = 0x12 frame is advisory (hub → server only, no response); the server-side stop-work half lives in phlix-server.

Metrics are collected per worker and flushed on a throttle; the DB retention prune runs from the single count=1 relay worker only (so retention DELETEs don't multiply across workers).

See Also ​

BSD-3-Clause