Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 mrhid6
|
8528f14ed7 | ||
|
mrhid6
|
df1d9658f5 | ||
|
mrhid6
|
9f9b384481 |
@@ -67,6 +67,15 @@ jobs:
|
||||
--set mongo.enabled=false \
|
||||
--set server.env.mongoUri=mongodb://mongo.example.com:27017/vantage > /dev/null
|
||||
|
||||
- name: Render with the Traefik ingress
|
||||
run: |
|
||||
helm template test "$CHART_DIR" \
|
||||
--set ingress.enabled=true \
|
||||
--set ingress.web.host=vantage.example.com \
|
||||
--set ingress.grpc.host=agents.example.com \
|
||||
--set ingress.tls.certResolver=letsencrypt \
|
||||
--set server.env.grpcHost=agents.example.com:443 > /dev/null
|
||||
|
||||
# The guards are load-bearing, so their absence is a regression the
|
||||
# same way a broken render is. Each of these must fail.
|
||||
- name: Check the guards still refuse bad values
|
||||
@@ -88,6 +97,16 @@ jobs:
|
||||
--set redis.enabled=false
|
||||
refuses "multiple replicas on a ReadWriteOnce volume" \
|
||||
--set server.replicaCount=2 --set server.persistence.enabled=true
|
||||
refuses "ingress with no web host" \
|
||||
--set ingress.enabled=true
|
||||
refuses "gRPC ingress with no host" \
|
||||
--set ingress.enabled=true \
|
||||
--set ingress.web.host=vantage.example.com \
|
||||
--set server.env.grpcHost=agents.example.com:443
|
||||
refuses "gRPC ingress while grpcHost is still in-cluster" \
|
||||
--set ingress.enabled=true \
|
||||
--set ingress.web.host=vantage.example.com \
|
||||
--set ingress.grpc.host=agents.example.com
|
||||
|
||||
- name: Read the chart version
|
||||
id: chart
|
||||
@@ -126,20 +145,36 @@ jobs:
|
||||
# github.server_url is this Gitea instance, so the registry
|
||||
# host needs no variable of its own and cannot drift from it.
|
||||
REGISTRY: ${{ github.server_url }}/api/packages/${{ github.repository_owner }}/helm/api/charts
|
||||
# The same pair server-deploy.yml uses for `docker login`.
|
||||
# RELEASE_TOKEN, not REGISTRY_PASSWORD: the latter is named in
|
||||
# the docs but set by no workflow, and an unset secret becomes
|
||||
# an empty password, which Gitea reports as "Failed to
|
||||
# authenticate user" rather than as a missing credential.
|
||||
REGISTRY_USER: ${{ secrets.REGISTRY_USER }}
|
||||
REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD }}
|
||||
REGISTRY_TOKEN: ${{ secrets.RELEASE_TOKEN }}
|
||||
CHART_VERSION: ${{ steps.chart.outputs.version }}
|
||||
run: |
|
||||
set -eu
|
||||
PKG="dist/vantage-${CHART_VERSION}.tgz"
|
||||
test -f "$PKG"
|
||||
|
||||
# Checked explicitly, because the failure it prevents is a
|
||||
# 401 that looks like a permissions problem on the token that
|
||||
# was never sent.
|
||||
if [ -z "${REGISTRY_USER}" ] || [ -z "${REGISTRY_TOKEN}" ]; then
|
||||
echo "REGISTRY_USER or RELEASE_TOKEN is not set on this repository."
|
||||
echo "RELEASE_TOKEN needs the write:package scope to publish a chart."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "publishing to ${REGISTRY} as ${REGISTRY_USER}"
|
||||
|
||||
# --fail-with-body so an HTTP error is a failed step with the
|
||||
# server's explanation, rather than a green run that published
|
||||
# nothing. A repeated version is rejected by the registry;
|
||||
# that is the intended behaviour, not something to retry past.
|
||||
curl --fail-with-body -sS \
|
||||
--user "${REGISTRY_USER}:${REGISTRY_PASSWORD}" \
|
||||
--user "${REGISTRY_USER}:${REGISTRY_TOKEN}" \
|
||||
-X POST \
|
||||
--upload-file "$PKG" \
|
||||
"$REGISTRY"
|
||||
|
||||
@@ -606,6 +606,25 @@ Windows: MSI built by CI (WiX), or `installer/setup.ps1` registering the agent a
|
||||
| `SITE_ORIGIN` | yes in practice | comma-separated allowed origins; unset refuses every cross-origin browser request |
|
||||
| `TRUST_PROXY` | no | only `true` behind a proxy that overwrites `X-Forwarded-For`, or clients spoof past the rate limiter |
|
||||
|
||||
### Ingress (Helm, Traefik)
|
||||
|
||||
`ingress.enabled` publishes **two** hostnames, because the two audiences arrive over different protocols:
|
||||
|
||||
| Values | Route |
|
||||
| --------------------- | ----------------------------------------------------------------------------------------- |
|
||||
| `ingress.web.host` | browsers → `web:3000`. Everything, including `/api` — see below |
|
||||
| `ingress.grpc.host` | agents → a dedicated `<release>-server-grpc` Service on 9090, annotated `serversscheme: h2c` |
|
||||
|
||||
**The server's HTTP port is deliberately not publishable.** `web` already proxies `/api`, `/auth` and the install scripts to it (`web/next.config.ts`), so a second route would be a second front door to the same API with none of that routing — and the console WebSocket and ESO token path would then exist at two addresses with different behaviour.
|
||||
|
||||
**The gRPC route needs its own Service.** The server terminates no TLS; it speaks plain h2c and always has, with TLS terminated by whatever sits in front. Traefik will not use h2c to a backend unless the *Service* says so, and that annotation applies to every port on the Service — so annotating the shared two-port `<release>-server` would force h2c on its HTTP port too.
|
||||
|
||||
**`server.env.grpcHost` is not derived from `ingress.grpc.host`, and the chart refuses to render if they disagree.** Agents dial whatever `grpcHost` says, and it is baked into every install one-liner; left pointing at the in-cluster Service while agents arrive through the ingress, every install succeeds and every agent then fails to connect, with nothing in the control plane explaining why. Guessing at the port (443? 9090?) would be worse than stopping.
|
||||
|
||||
TLS is `ingress.tls.secretName` / `grpcSecretName` (pre-existing certificates) **or** `certResolver` (Traefik ACME). Setting neither while `tls.enabled` produces a TLS router with no certificate, so Traefik serves its self-signed default — which looks valid and is trusted by nothing. NOTES.txt warns on install rather than the chart failing, since it is a real if unusual choice behind another terminator.
|
||||
|
||||
---
|
||||
|
||||
`deploy/docker-compose.yml` runs four services: `redis`, `guacd`, `server` (8080 + 9090), `web` (3000). MongoDB is external. `deploy/docker-compose.site.yml` adds five more — `site` (3003), `sitesvc` (8082), `admin` (8083), `adminsite` (3004) and `docsite` (3005) — and is only used on vantage.hostxtra.co.uk.
|
||||
|
||||
`docsite` is the odd one: a **static** build served by `nginx:alpine-slim`, not a Node runtime, and it listens on `80` rather than `3000`. It is reached at **`vantage.hostxtra.co.uk/docs`** — a path on the marketing host, routed by its own Nginx Proxy Manager location, which must sort **above** the catch-all forwarding to `site:3003` or Next answers the 404. A path and not a subdomain because `*.vantage.hostxtra.co.uk` is the per-tenant instance namespace and `APP_ROOT_LABEL` would read a `docs.` label as a tenant slug. NPM forwards the **full** path upstream — it does not strip `/docs` — so `DOCS_BASE_URL`, the proxy location and the directory the image copies the build into (`/usr/share/nginx/html/docs`) must all agree. When they do not, the HTML loads and every asset 404s.
|
||||
@@ -730,7 +749,7 @@ Two jobs' worth of work in one, split by trigger. Any push or PR touching `deplo
|
||||
|
||||
Publishing runs only on a `chart/v*` tag, to the Gitea Helm registry at `/api/packages/<owner>/helm/api/charts`. **`Chart.yaml` is the source of truth for the version**; the tag only selects which one to publish, and a tag that disagrees with `Chart.yaml` fails rather than stamping over it — the alternative leaves the repository disagreeing with what shipped. A version already in the registry is rejected by Gitea, which is intended: published chart versions are immutable.
|
||||
|
||||
The registry host comes from `github.server_url`, so it cannot drift from the instance the workflow is running on, and it reuses `REGISTRY_USER` / `REGISTRY_PASSWORD` — the same `write:packages` token the images use.
|
||||
The registry host comes from `github.server_url`, so it cannot drift from the instance the workflow is running on. It authenticates with `REGISTRY_USER` + **`RELEASE_TOKEN`** — the pair `server-deploy.yml` actually uses for `docker login`. `REGISTRY_PASSWORD` is listed in the secrets table below but set by no workflow; passing an unset secret yields an empty password and Gitea answers `401 Failed to authenticate user`, which reads like a scope problem on a token that was never sent. The publish step therefore checks both are non-empty before it calls curl. `RELEASE_TOKEN` needs `write:package` in addition to `write:release`.
|
||||
|
||||
```bash
|
||||
helm repo add vantage https://gitea.hostxtra.co.uk/api/packages/mrhid6/helm
|
||||
@@ -749,9 +768,9 @@ git push origin main # server + web deploy
|
||||
|
||||
| Name | Type | Value |
|
||||
| ----------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `RELEASE_TOKEN` | Secret | Gitea API token, `write:release` |
|
||||
| `REGISTRY_USER` | Secret | Gitea username |
|
||||
| `REGISTRY_PASSWORD` | Secret | Gitea token, `write:packages` |
|
||||
| `RELEASE_TOKEN` | Secret | Gitea API token. Needs `write:release` (agent releases), `write:package` (container images and the Helm chart). **This is the only token any workflow authenticates with** — `docker login` and the chart publish both pair it with `REGISTRY_USER` |
|
||||
| `REGISTRY_USER` | Secret | Gitea username. Must own `RELEASE_TOKEN`, or basic auth is rejected |
|
||||
| ~~`REGISTRY_PASSWORD`~~ | — | **Not used.** Named here historically; no workflow reads it. Referencing an unset secret yields an empty password and a `401 Failed to authenticate user` that looks like a token scope problem. Use `RELEASE_TOKEN` |
|
||||
| `DOCKER_HOST` | Variable | registry host used for image tags |
|
||||
| `API_URL` | **not** a CI variable | `web` reads it at **runtime**, from the container environment — `next.config.ts` is evaluated when `server.js` boots in standalone mode, and the rewrites it feeds are server-side, never browser-side. Default `http://localhost:8080`; compose sets `http://server:8080`. `NEXT_PUBLIC_API_URL` is still honoured as a fallback for existing deployments. |
|
||||
| `SITE_API_URL` | Variable | **browser-reachable** sitesvc URL, baked into the `site` image. Required — if empty, both forms report "not connected" and submit nowhere. Must also be in sitesvc's `SITE_ORIGIN`. |
|
||||
|
||||
@@ -2,5 +2,5 @@ apiVersion: v2
|
||||
name: vantage
|
||||
description: Helm chart for the Vantage stack (Redis, MongoDB, guacd, server, web)
|
||||
type: application
|
||||
version: 0.1.0
|
||||
appVersion: "1.0.0"
|
||||
version: 1.0.2
|
||||
appVersion: "1.0.2"
|
||||
|
||||
@@ -34,6 +34,27 @@ Scaling (server.replicaCount / web.replicaCount):
|
||||
the pods skip them. Its logs are kept: kubectl logs job/{{ .Release.Name }}-migrate
|
||||
{{- end }}
|
||||
|
||||
{{- if .Values.ingress.enabled }}
|
||||
|
||||
Ingress (Traefik):
|
||||
- UI and API: https://{{ .Values.ingress.web.host }}
|
||||
Everything browsers need goes here; web proxies /api, /auth and the install
|
||||
scripts to the server, so the server's HTTP port is not published separately.
|
||||
{{- if .Values.ingress.grpc.enabled }}
|
||||
- Agents: {{ .Values.ingress.grpc.host }} (gRPC, h2c behind TLS)
|
||||
Agents dial server.env.grpcHost, currently {{ tpl .Values.server.env.grpcHost . }}.
|
||||
Point DNS for both hostnames at the Traefik load balancer.
|
||||
{{- if not .Values.ingress.tls.enabled }}
|
||||
- WARNING: ingress.tls.enabled is false. Agent tokens and session cookies
|
||||
would cross the network in clear.
|
||||
{{- else if and (not .Values.ingress.tls.certResolver) (not .Values.ingress.tls.secretName) }}
|
||||
- WARNING: TLS is on but neither ingress.tls.secretName nor
|
||||
ingress.tls.certResolver is set, so Traefik will serve its self-signed
|
||||
default certificate, which no agent and no browser will trust.
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
By default the server/web/guacd services are ClusterIP only (no host port publishing,
|
||||
unlike the original docker-compose file). To expose them externally, set
|
||||
server.service.type / web.service.type / guacd.service.type to NodePort or LoadBalancer,
|
||||
|
||||
@@ -0,0 +1,132 @@
|
||||
{{- if .Values.ingress.enabled }}
|
||||
{{/*
|
||||
Two hostnames, because the two audiences arrive over different protocols.
|
||||
|
||||
Browsers reach `web`, and only `web`: it proxies /api, /auth and the install
|
||||
scripts through to the server itself (see web/next.config.ts), so publishing the
|
||||
server's HTTP port separately would be a second front door to the same API with
|
||||
none of the same routing.
|
||||
|
||||
Agents reach the server's gRPC port, which is plain h2c — the server holds no
|
||||
certificates of its own, TLS has always been terminated by whatever sits in
|
||||
front. Traefik will not speak h2c to a backend unless told to, and it is told
|
||||
per Service, which is why the gRPC route gets a Service of its own below rather
|
||||
than reusing the two-port one. Annotating the shared Service would force h2c on
|
||||
its HTTP port too.
|
||||
*/}}
|
||||
{{- $tls := .Values.ingress.tls }}
|
||||
{{- $webHost := required "ingress.enabled requires ingress.web.host" .Values.ingress.web.host }}
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-web
|
||||
labels:
|
||||
{{- include "vantage.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: web
|
||||
annotations:
|
||||
traefik.ingress.kubernetes.io/router.entrypoints: {{ .Values.ingress.entrypoint | quote }}
|
||||
{{- if and $tls.enabled $tls.certResolver }}
|
||||
traefik.ingress.kubernetes.io/router.tls: "true"
|
||||
traefik.ingress.kubernetes.io/router.tls.certresolver: {{ $tls.certResolver | quote }}
|
||||
{{- else if $tls.enabled }}
|
||||
traefik.ingress.kubernetes.io/router.tls: "true"
|
||||
{{- end }}
|
||||
{{- with .Values.ingress.annotations }}
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
{{- if .Values.ingress.className }}
|
||||
ingressClassName: {{ .Values.ingress.className }}
|
||||
{{- end }}
|
||||
{{- if and $tls.enabled $tls.secretName }}
|
||||
tls:
|
||||
- hosts:
|
||||
- {{ $webHost | quote }}
|
||||
secretName: {{ $tls.secretName }}
|
||||
{{- end }}
|
||||
rules:
|
||||
- host: {{ $webHost | quote }}
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: {{ .Release.Name }}-web
|
||||
port:
|
||||
number: {{ .Values.web.service.port }}
|
||||
{{- if .Values.ingress.grpc.enabled }}
|
||||
{{- $grpcHost := required "ingress.grpc.enabled requires ingress.grpc.host" .Values.ingress.grpc.host }}
|
||||
{{/*
|
||||
GRPC_HOST is what an agent is told to dial, and it is baked into every install
|
||||
one-liner. Left pointing at the in-cluster Service while agents are expected to
|
||||
arrive through the ingress, every install would succeed and every agent would
|
||||
fail to connect — with nothing in the control plane saying why.
|
||||
*/}}
|
||||
{{- $grpcEnv := tpl .Values.server.env.grpcHost . }}
|
||||
{{- if contains (printf "%s-server" .Release.Name) $grpcEnv }}
|
||||
{{- fail (printf "ingress.grpc.enabled routes agents through %s, but server.env.grpcHost is still the in-cluster address %q. Agents dial the value of grpcHost, so set it to the public gRPC address (for example %q)." $grpcHost $grpcEnv (printf "%s:443" $grpcHost)) }}
|
||||
{{- end }}
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-server-grpc
|
||||
labels:
|
||||
{{- include "vantage.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: server
|
||||
annotations:
|
||||
# The server speaks h2c: it terminates no TLS itself. Without this Traefik
|
||||
# dials the backend as HTTP/1.1 and every agent handshake fails.
|
||||
traefik.ingress.kubernetes.io/service.serversscheme: h2c
|
||||
spec:
|
||||
type: ClusterIP
|
||||
selector:
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
app.kubernetes.io/component: server
|
||||
ports:
|
||||
- name: grpc
|
||||
port: {{ .Values.server.service.grpcPort }}
|
||||
targetPort: {{ .Values.server.service.grpcPort }}
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-grpc
|
||||
labels:
|
||||
{{- include "vantage.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: server
|
||||
annotations:
|
||||
traefik.ingress.kubernetes.io/router.entrypoints: {{ .Values.ingress.entrypoint | quote }}
|
||||
{{- if and $tls.enabled $tls.certResolver }}
|
||||
traefik.ingress.kubernetes.io/router.tls: "true"
|
||||
traefik.ingress.kubernetes.io/router.tls.certresolver: {{ $tls.certResolver | quote }}
|
||||
{{- else if $tls.enabled }}
|
||||
traefik.ingress.kubernetes.io/router.tls: "true"
|
||||
{{- end }}
|
||||
{{- with .Values.ingress.grpc.annotations }}
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
{{- if .Values.ingress.className }}
|
||||
ingressClassName: {{ .Values.ingress.className }}
|
||||
{{- end }}
|
||||
{{- if and $tls.enabled $tls.grpcSecretName }}
|
||||
tls:
|
||||
- hosts:
|
||||
- {{ $grpcHost | quote }}
|
||||
secretName: {{ $tls.grpcSecretName }}
|
||||
{{- end }}
|
||||
rules:
|
||||
- host: {{ $grpcHost | quote }}
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: {{ .Release.Name }}-server-grpc
|
||||
port:
|
||||
number: {{ .Values.server.service.grpcPort }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
@@ -108,4 +108,37 @@ web:
|
||||
env:
|
||||
apiUrl: "http://{{ .Release.Name }}-server:8080"
|
||||
|
||||
# Traefik ingress. Two hostnames, because the two audiences arrive differently:
|
||||
# browsers reach `web` (which proxies /api, /auth and the install scripts to the
|
||||
# server), and agents reach the server's gRPC port directly.
|
||||
#
|
||||
# Publishing the server's HTTP port is deliberately not offered — it would be a
|
||||
# second door to the same API, bypassing the routing web already performs.
|
||||
ingress:
|
||||
enabled: false
|
||||
className: traefik
|
||||
# Traefik entrypoint name. `websecure` is the default TLS entrypoint in the
|
||||
# official chart; installs that renamed it must say so here.
|
||||
entrypoint: websecure
|
||||
# Applied to the web router only. Middlewares, rate limits, IP allow lists.
|
||||
annotations: {}
|
||||
web:
|
||||
# Required when ingress.enabled. The hostname users open in a browser.
|
||||
host: ""
|
||||
grpc:
|
||||
# Agents dial this. Turning it off means agents reach gRPC some other
|
||||
# way — a LoadBalancer Service, a node port, or an in-cluster path.
|
||||
enabled: true
|
||||
host: ""
|
||||
annotations: {}
|
||||
tls:
|
||||
enabled: true
|
||||
# Either name pre-existing certificate Secrets, or leave both empty and
|
||||
# set certResolver to have Traefik obtain them (ACME). Setting neither
|
||||
# produces a TLS router with no certificate, which serves Traefik's
|
||||
# self-signed default — valid-looking and trusted by nothing.
|
||||
secretName: ""
|
||||
grpcSecretName: ""
|
||||
certResolver: ""
|
||||
|
||||
imagePullSecrets: []
|
||||
|
||||
Reference in New Issue
Block a user