feat: Added ingress to chart
This commit is contained in:
@@ -67,6 +67,15 @@ jobs:
|
||||
--set mongo.enabled=false \
|
||||
--set server.env.mongoUri=mongodb://mongo.example.com:27017/vantage > /dev/null
|
||||
|
||||
- name: Render with the Traefik ingress
|
||||
run: |
|
||||
helm template test "$CHART_DIR" \
|
||||
--set ingress.enabled=true \
|
||||
--set ingress.web.host=vantage.example.com \
|
||||
--set ingress.grpc.host=agents.example.com \
|
||||
--set ingress.tls.certResolver=letsencrypt \
|
||||
--set server.env.grpcHost=agents.example.com:443 > /dev/null
|
||||
|
||||
# The guards are load-bearing, so their absence is a regression the
|
||||
# same way a broken render is. Each of these must fail.
|
||||
- name: Check the guards still refuse bad values
|
||||
@@ -88,6 +97,16 @@ jobs:
|
||||
--set redis.enabled=false
|
||||
refuses "multiple replicas on a ReadWriteOnce volume" \
|
||||
--set server.replicaCount=2 --set server.persistence.enabled=true
|
||||
refuses "ingress with no web host" \
|
||||
--set ingress.enabled=true
|
||||
refuses "gRPC ingress with no host" \
|
||||
--set ingress.enabled=true \
|
||||
--set ingress.web.host=vantage.example.com \
|
||||
--set server.env.grpcHost=agents.example.com:443
|
||||
refuses "gRPC ingress while grpcHost is still in-cluster" \
|
||||
--set ingress.enabled=true \
|
||||
--set ingress.web.host=vantage.example.com \
|
||||
--set ingress.grpc.host=agents.example.com
|
||||
|
||||
- name: Read the chart version
|
||||
id: chart
|
||||
|
||||
@@ -606,6 +606,25 @@ Windows: MSI built by CI (WiX), or `installer/setup.ps1` registering the agent a
|
||||
| `SITE_ORIGIN` | yes in practice | comma-separated allowed origins; unset refuses every cross-origin browser request |
|
||||
| `TRUST_PROXY` | no | only `true` behind a proxy that overwrites `X-Forwarded-For`, or clients spoof past the rate limiter |
|
||||
|
||||
### Ingress (Helm, Traefik)
|
||||
|
||||
`ingress.enabled` publishes **two** hostnames, because the two audiences arrive over different protocols:
|
||||
|
||||
| Values | Route |
|
||||
| --------------------- | ----------------------------------------------------------------------------------------- |
|
||||
| `ingress.web.host` | browsers → `web:3000`. Everything, including `/api` — see below |
|
||||
| `ingress.grpc.host` | agents → a dedicated `<release>-server-grpc` Service on 9090, annotated `serversscheme: h2c` |
|
||||
|
||||
**The server's HTTP port is deliberately not publishable.** `web` already proxies `/api`, `/auth` and the install scripts to it (`web/next.config.ts`), so a second route would be a second front door to the same API with none of that routing — and the console WebSocket and ESO token path would then exist at two addresses with different behaviour.
|
||||
|
||||
**The gRPC route needs its own Service.** The server terminates no TLS; it speaks plain h2c and always has, with TLS terminated by whatever sits in front. Traefik will not use h2c to a backend unless the *Service* says so, and that annotation applies to every port on the Service — so annotating the shared two-port `<release>-server` would force h2c on its HTTP port too.
|
||||
|
||||
**`server.env.grpcHost` is not derived from `ingress.grpc.host`, and the chart refuses to render if they disagree.** Agents dial whatever `grpcHost` says, and it is baked into every install one-liner; left pointing at the in-cluster Service while agents arrive through the ingress, every install succeeds and every agent then fails to connect, with nothing in the control plane explaining why. Guessing at the port (443? 9090?) would be worse than stopping.
|
||||
|
||||
TLS is `ingress.tls.secretName` / `grpcSecretName` (pre-existing certificates) **or** `certResolver` (Traefik ACME). Setting neither while `tls.enabled` produces a TLS router with no certificate, so Traefik serves its self-signed default — which looks valid and is trusted by nothing. NOTES.txt warns on install rather than the chart failing, since it is a real if unusual choice behind another terminator.
|
||||
|
||||
---
|
||||
|
||||
`deploy/docker-compose.yml` runs four services: `redis`, `guacd`, `server` (8080 + 9090), `web` (3000). MongoDB is external. `deploy/docker-compose.site.yml` adds five more — `site` (3003), `sitesvc` (8082), `admin` (8083), `adminsite` (3004) and `docsite` (3005) — and is only used on vantage.hostxtra.co.uk.
|
||||
|
||||
`docsite` is the odd one: a **static** build served by `nginx:alpine-slim`, not a Node runtime, and it listens on `80` rather than `3000`. It is reached at **`vantage.hostxtra.co.uk/docs`** — a path on the marketing host, routed by its own Nginx Proxy Manager location, which must sort **above** the catch-all forwarding to `site:3003` or Next answers the 404. A path and not a subdomain because `*.vantage.hostxtra.co.uk` is the per-tenant instance namespace and `APP_ROOT_LABEL` would read a `docs.` label as a tenant slug. NPM forwards the **full** path upstream — it does not strip `/docs` — so `DOCS_BASE_URL`, the proxy location and the directory the image copies the build into (`/usr/share/nginx/html/docs`) must all agree. When they do not, the HTML loads and every asset 404s.
|
||||
|
||||
@@ -2,5 +2,5 @@ apiVersion: v2
|
||||
name: vantage
|
||||
description: Helm chart for the Vantage stack (Redis, MongoDB, guacd, server, web)
|
||||
type: application
|
||||
version: 1.0.1
|
||||
appVersion: "1.0.1"
|
||||
version: 1.0.2
|
||||
appVersion: "1.0.2"
|
||||
|
||||
@@ -34,6 +34,27 @@ Scaling (server.replicaCount / web.replicaCount):
|
||||
the pods skip them. Its logs are kept: kubectl logs job/{{ .Release.Name }}-migrate
|
||||
{{- end }}
|
||||
|
||||
{{- if .Values.ingress.enabled }}
|
||||
|
||||
Ingress (Traefik):
|
||||
- UI and API: https://{{ .Values.ingress.web.host }}
|
||||
Everything browsers need goes here; web proxies /api, /auth and the install
|
||||
scripts to the server, so the server's HTTP port is not published separately.
|
||||
{{- if .Values.ingress.grpc.enabled }}
|
||||
- Agents: {{ .Values.ingress.grpc.host }} (gRPC, h2c behind TLS)
|
||||
Agents dial server.env.grpcHost, currently {{ tpl .Values.server.env.grpcHost . }}.
|
||||
Point DNS for both hostnames at the Traefik load balancer.
|
||||
{{- if not .Values.ingress.tls.enabled }}
|
||||
- WARNING: ingress.tls.enabled is false. Agent tokens and session cookies
|
||||
would cross the network in clear.
|
||||
{{- else if and (not .Values.ingress.tls.certResolver) (not .Values.ingress.tls.secretName) }}
|
||||
- WARNING: TLS is on but neither ingress.tls.secretName nor
|
||||
ingress.tls.certResolver is set, so Traefik will serve its self-signed
|
||||
default certificate, which no agent and no browser will trust.
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
By default the server/web/guacd services are ClusterIP only (no host port publishing,
|
||||
unlike the original docker-compose file). To expose them externally, set
|
||||
server.service.type / web.service.type / guacd.service.type to NodePort or LoadBalancer,
|
||||
|
||||
@@ -0,0 +1,132 @@
|
||||
{{- if .Values.ingress.enabled }}
|
||||
{{/*
|
||||
Two hostnames, because the two audiences arrive over different protocols.
|
||||
|
||||
Browsers reach `web`, and only `web`: it proxies /api, /auth and the install
|
||||
scripts through to the server itself (see web/next.config.ts), so publishing the
|
||||
server's HTTP port separately would be a second front door to the same API with
|
||||
none of the same routing.
|
||||
|
||||
Agents reach the server's gRPC port, which is plain h2c — the server holds no
|
||||
certificates of its own, TLS has always been terminated by whatever sits in
|
||||
front. Traefik will not speak h2c to a backend unless told to, and it is told
|
||||
per Service, which is why the gRPC route gets a Service of its own below rather
|
||||
than reusing the two-port one. Annotating the shared Service would force h2c on
|
||||
its HTTP port too.
|
||||
*/}}
|
||||
{{- $tls := .Values.ingress.tls }}
|
||||
{{- $webHost := required "ingress.enabled requires ingress.web.host" .Values.ingress.web.host }}
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-web
|
||||
labels:
|
||||
{{- include "vantage.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: web
|
||||
annotations:
|
||||
traefik.ingress.kubernetes.io/router.entrypoints: {{ .Values.ingress.entrypoint | quote }}
|
||||
{{- if and $tls.enabled $tls.certResolver }}
|
||||
traefik.ingress.kubernetes.io/router.tls: "true"
|
||||
traefik.ingress.kubernetes.io/router.tls.certresolver: {{ $tls.certResolver | quote }}
|
||||
{{- else if $tls.enabled }}
|
||||
traefik.ingress.kubernetes.io/router.tls: "true"
|
||||
{{- end }}
|
||||
{{- with .Values.ingress.annotations }}
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
{{- if .Values.ingress.className }}
|
||||
ingressClassName: {{ .Values.ingress.className }}
|
||||
{{- end }}
|
||||
{{- if and $tls.enabled $tls.secretName }}
|
||||
tls:
|
||||
- hosts:
|
||||
- {{ $webHost | quote }}
|
||||
secretName: {{ $tls.secretName }}
|
||||
{{- end }}
|
||||
rules:
|
||||
- host: {{ $webHost | quote }}
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: {{ .Release.Name }}-web
|
||||
port:
|
||||
number: {{ .Values.web.service.port }}
|
||||
{{- if .Values.ingress.grpc.enabled }}
|
||||
{{- $grpcHost := required "ingress.grpc.enabled requires ingress.grpc.host" .Values.ingress.grpc.host }}
|
||||
{{/*
|
||||
GRPC_HOST is what an agent is told to dial, and it is baked into every install
|
||||
one-liner. Left pointing at the in-cluster Service while agents are expected to
|
||||
arrive through the ingress, every install would succeed and every agent would
|
||||
fail to connect — with nothing in the control plane saying why.
|
||||
*/}}
|
||||
{{- $grpcEnv := tpl .Values.server.env.grpcHost . }}
|
||||
{{- if contains (printf "%s-server" .Release.Name) $grpcEnv }}
|
||||
{{- fail (printf "ingress.grpc.enabled routes agents through %s, but server.env.grpcHost is still the in-cluster address %q. Agents dial the value of grpcHost, so set it to the public gRPC address (for example %q)." $grpcHost $grpcEnv (printf "%s:443" $grpcHost)) }}
|
||||
{{- end }}
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-server-grpc
|
||||
labels:
|
||||
{{- include "vantage.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: server
|
||||
annotations:
|
||||
# The server speaks h2c: it terminates no TLS itself. Without this Traefik
|
||||
# dials the backend as HTTP/1.1 and every agent handshake fails.
|
||||
traefik.ingress.kubernetes.io/service.serversscheme: h2c
|
||||
spec:
|
||||
type: ClusterIP
|
||||
selector:
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
app.kubernetes.io/component: server
|
||||
ports:
|
||||
- name: grpc
|
||||
port: {{ .Values.server.service.grpcPort }}
|
||||
targetPort: {{ .Values.server.service.grpcPort }}
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-grpc
|
||||
labels:
|
||||
{{- include "vantage.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: server
|
||||
annotations:
|
||||
traefik.ingress.kubernetes.io/router.entrypoints: {{ .Values.ingress.entrypoint | quote }}
|
||||
{{- if and $tls.enabled $tls.certResolver }}
|
||||
traefik.ingress.kubernetes.io/router.tls: "true"
|
||||
traefik.ingress.kubernetes.io/router.tls.certresolver: {{ $tls.certResolver | quote }}
|
||||
{{- else if $tls.enabled }}
|
||||
traefik.ingress.kubernetes.io/router.tls: "true"
|
||||
{{- end }}
|
||||
{{- with .Values.ingress.grpc.annotations }}
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
{{- if .Values.ingress.className }}
|
||||
ingressClassName: {{ .Values.ingress.className }}
|
||||
{{- end }}
|
||||
{{- if and $tls.enabled $tls.grpcSecretName }}
|
||||
tls:
|
||||
- hosts:
|
||||
- {{ $grpcHost | quote }}
|
||||
secretName: {{ $tls.grpcSecretName }}
|
||||
{{- end }}
|
||||
rules:
|
||||
- host: {{ $grpcHost | quote }}
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: {{ .Release.Name }}-server-grpc
|
||||
port:
|
||||
number: {{ .Values.server.service.grpcPort }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
@@ -108,4 +108,37 @@ web:
|
||||
env:
|
||||
apiUrl: "http://{{ .Release.Name }}-server:8080"
|
||||
|
||||
# Traefik ingress. Two hostnames, because the two audiences arrive differently:
|
||||
# browsers reach `web` (which proxies /api, /auth and the install scripts to the
|
||||
# server), and agents reach the server's gRPC port directly.
|
||||
#
|
||||
# Publishing the server's HTTP port is deliberately not offered — it would be a
|
||||
# second door to the same API, bypassing the routing web already performs.
|
||||
ingress:
|
||||
enabled: false
|
||||
className: traefik
|
||||
# Traefik entrypoint name. `websecure` is the default TLS entrypoint in the
|
||||
# official chart; installs that renamed it must say so here.
|
||||
entrypoint: websecure
|
||||
# Applied to the web router only. Middlewares, rate limits, IP allow lists.
|
||||
annotations: {}
|
||||
web:
|
||||
# Required when ingress.enabled. The hostname users open in a browser.
|
||||
host: ""
|
||||
grpc:
|
||||
# Agents dial this. Turning it off means agents reach gRPC some other
|
||||
# way — a LoadBalancer Service, a node port, or an in-cluster path.
|
||||
enabled: true
|
||||
host: ""
|
||||
annotations: {}
|
||||
tls:
|
||||
enabled: true
|
||||
# Either name pre-existing certificate Secrets, or leave both empty and
|
||||
# set certResolver to have Traefik obtain them (ACME). Setting neither
|
||||
# produces a TLS router with no certificate, which serves Traefik's
|
||||
# self-signed default — valid-looking and trusted by nothing.
|
||||
secretName: ""
|
||||
grpcSecretName: ""
|
||||
certResolver: ""
|
||||
|
||||
imagePullSecrets: []
|
||||
|
||||
Reference in New Issue
Block a user