k8s¶
Gridweave splits in two: the web services (platform, gateway, credit, faucet, metering, webui) run in Docker on the master, and the compute — Ray, GPU workers, model endpoints — runs on Kubernetes (K3s). This folder sets up the Kubernetes side. deploy.sh applies it once when a cluster is first stood up; you never run these by hand.
How the two halves connect¶
Two seams, both defined here:
- Traffic in. Kubernetes' built-in Traefik is the public HTTPS front door. It gets TLS certificates automatically (Let's Encrypt) and routes
platform.gridweave.io→ the platform service andapp.gridweave.io→ the webui. Those services are Docker containers, not Kubernetes pods — sotraefik/services.yamlpoints Traefik at them by the master'shost:port. - Compute out. The gateway and kuberay reach the Ray head over three fixed NodePorts —
30001(client),30002(dashboard),30003(Serve) — the ports baked into theirRAY_HEAD_ADDRESS/RAY_SERVE_PORT.
The manifests¶
| File | What it sets up |
|---|---|
namespace.yaml |
The gridweave namespace everything lives in. |
gridweave-job-crd.yaml, gridweave-endpoint-crd.yaml |
Teach Kubernetes about GridweaveJob / GridweaveEndpoint records. These live in Kubernetes' own storage (etcd), which is why the gateway needs no database of its own — see gridweave_controller. |
nvidia-device-plugin.yaml |
Tells Kubernetes which nodes have NVIDIA GPUs so it can schedule on them. Two variants — datacenter and consumer — because consumer cards lack the ECC/MIG health checks datacenter cards have (the consumer one runs with FAIL_ON_INIT_ERROR=false). |
ray-head-nodeport.yaml |
Exposes the Ray head's client / dashboard / Serve ports as the NodePorts above (30001/30002/30003). |
ray-head-networkpolicy.yaml, ray-egress-networkpolicy.yaml |
Firewall rules: who may reach the Ray head, and where workers may connect out. |
endpoint-exec-rbac.yaml |
Lets a custom endpoint's ray-proxy sidecar run commands inside the user's container — how custom-command endpoints work (feed input → run command → read output). |
traefik/helm-config.yaml |
Configures the built-in Traefik, mainly turning on automatic Let's Encrypt HTTPS. |
traefik/ingress.yaml |
The HTTPS routes — which domain goes to which service, plus an HTTP→HTTPS redirect. |
traefik/services.yaml |
The bridge that lets Traefik (in Kubernetes) reach the web services (in Docker), by the master's host IP and port. |
Observability manifests (fluent-bit, DCGM) live separately under diagnostics/.