Skip to content

1 · Deploy a cluster

Deploy Gridweave on one machine or across several — the same interactive wizard (deploy.sh) does both. The master is the control plane (needs a public IP); it can run everything itself, or spread database / storage / billing (and even the API + dashboard) onto extra machines. GPU nodes join afterward.

Timing, port, and disk details live in the architecture reference. Rough expectation: master ~10–15 min, each GPU node ~30–40 min on a fresh install.

Requirements

Master: 4 CPU, 8 GB RAM, 160 GB disk, a public IP.

GPU nodes: Linux (Ubuntu 22.04+) with GPU hardware. At least 50 GB free disk; more is recommended depending on deployed image size — a secondary disk can be mounted, see Storage.

Disk preflight: check df -h / shows the size you paid for. Ubuntu Server's installer allocates only part of the disk to the root volume by default, so a 100 GB VM often boots with a ~25 GB /. If so, expand it (online, non-destructive):

lvextend -l +100%FREE /dev/ubuntu-vg/ubuntu-lv && resize2fs /dev/ubuntu-vg/ubuntu-lv

join-node.sh detects this case and prints the exact commands for your device names.

Driver preflight (NVIDIA x86): driver 580.159.03 or newer is required — older drivers fail CUDA init with the vLLM v1 engine (torch 2.11+) and hit Xid cascades on multi-GPU. Check with nvidia-smi; if below, upgrade before joining:

wget https://us.download.nvidia.com/tesla/580.159.03/NVIDIA-Linux-x86_64-580.159.03.run
sudo bash NVIDIA-Linux-x86_64-580.159.03.run --silent --dkms --kernel-module-type=proprietary --no-opengl-files
sudo reboot

The join gate rejects older drivers with these same instructions.

Extra (non-GPU) machines: any SSH-reachable Linux box. Docker + WireGuard are installed for you.

Deploy the master

curl -fsSL https://pub-c48a651bbb2f42988602aa11bb9d9267.r2.dev/tarball/gridweave-server-latest.tar.gz | tar xz
cd core
sudo ./deploy/deploy.sh

The wizard walks you through everything — press Enter to accept the default at each prompt:

  • Public IP — auto-detected; Enter to accept (or type another).
  • External S3 (optional) — Enter keeps the bundled MinIO, or paste an AWS/R2 endpoint + keys. A deploy-time decision: switching object stores later doesn't migrate data.
  • Secrets (ADMIN_TOKEN, JWT_SECRET, POSTGRES_PASSWORD, S3_SECRET_KEY) — auto-generated and saved to deploy/.env. No manual editing needed.
  • On-chain audit ledger (CometBFT)? [y/N] — y anchors every credit/job/endpoint event on an append-only chain (see the Chain Activity page); N skips the chain node entirely.
  • Stripe payments (optional) — paste your Stripe secret key and users can buy credits by card; the webhook is provisioned automatically (guide). Enter to skip. Either way it's stored in the platform and stays manageable from the web UI → Configuration → Payments.
  • Signup email (SMTP) (optional) — Enter to skip; tokens are then shown on screen at signup. Either way it's stored in the platform and stays editable from the web UI → Configuration → Email — applies live, no restart.
  • How many machines besides this master?
  • 0 → single machine. Everything runs here.
  • N → multi-VM. For each machine you give its IP, SSH user, and either a password or an SSH key (if the VM doesn't have your key yet, the wizard installs it for you — you enter that VM's password once). Then you choose where each service runs (API · dashboard · database · storage · billing); Enter keeps it on the master. Type back at any prompt to fix a value.

When it finishes, the summary prints your URLs, the exact ports to open in each box's cloud firewall / security-group, and the admin token + a ready-to-paste join-key command. Save the admin token — you'll use it in every guide that follows (it's also in /opt/gridweave/deploy/.env).

⚠️ deploy.sh recreates a clean slate every run — fresh secrets, wiped databases. To ship a new version onto a running cluster, use update instead.

Advanced config (optional)

The wizard auto-fills deploy/.env. To override anything, create/edit it before deploying (cp deploy/.env.example deploy/.env):

Variable Description
MASTER_IP Public IP of this master (auto-detected if blank)
ADMIN_TOKEN / JWT_SECRET / POSTGRES_PASSWORD / S3_SECRET_KEY auto-generated if blank
S3_ENDPOINT / S3_ACCESS_KEY / S3_SECRET_KEY point at external S3 (AWS / R2) to skip the bundled MinIO — see Bring your own S3
CHAIN_ENABLED / CHAIN_RPC enable on-chain token features
STRIPE_SECRET_KEY Stripe payments — secret key only; the webhook auto-provisions (guide)

Add GPU nodes

⚠️ Join nodes one at a time. Wait for Node joined successfully on the current node before starting the next — parallel joins race on the RayCluster update and fail with a 409.

1. Generate a join key (on master)

export ADMIN_TOKEN=<ADMIN_TOKEN_FROM_SUMMARY_OR_ENV>
curl -X POST http://<master-ip>:8800/v1/nodes/join-keys \
  -H "Authorization: Bearer ${ADMIN_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"label": "my-gpu-node", "owner": "org"}'

# Returns: {"join_key": "...", "expires_in_hours": 24, "vpn_ip": "10.100.0.x",
#           "join_command": "sudo MASTER_IP=<master-ip> bash /opt/gridweave/deploy/join-node.sh <join-key>"}

owner: "org" for organization resources, "provider-<name>" for provider-hosted endpoints. A provider's join_key doubles as their web-UI / SDK login token — send it to the GPU node owner and tell them to keep it. For pre-release testing, SKIP_R2=1 prevents R2 tarballs from overwriting local code.

2. Join (on the GPU machine, Ubuntu 22.04+)

curl -fsSL https://pub-c48a651bbb2f42988602aa11bb9d9267.r2.dev/tarball/gridweave-worker-latest.tar.gz | tar xz
sudo MASTER_IP=<master-ip> bash core/deploy/join-node.sh <join-key>

This validates the key, brings up the WireGuard tunnel, joins K3s, builds the worker image into containerd, cleans the build cache, and registers the GPU.

The script asks for the hourly rate (credits/hr) for paid workloads on this hardware:

  • 0 — org-owned / free capacity (the default for owner: "org").
  • > 0 — provider hardware: paid workloads route here and the provider earns per VRAM-hour. Must be above 0 for earnings to accrue.

A fresh join builds images for ~30–40 min and dies if your SSH session drops — run it inside tmux/screen (e.g. tmux new -s join) to make it disconnect-proof. If it does get interrupted: just re-run, it's idempotent and resumes from the images already built.

3. Verify (on master)

kubectl get nodes                    # all nodes Ready
kubectl get pods -n gridweave        # head + workers Running

Next: 2 · HTTPS — put the WebUI and API behind your own domains with auto-TLS.