gridweave-sdk¶

One Python API for a heterogeneous GPU cluster — mix NVIDIA and AMD, size GPUs by the gigabyte, run fractional or multi-GPU, and deploy models you can charge others to call. Works from a script, a notebook, or the REPL.

Install¶

# First run — the tarball also bundles the onboarding notebook:
curl -sL https://pub-c48a651bbb2f42988602aa11bb9d9267.r2.dev/tarball/gridweave-sdk.tar.gz | tar xz
cd gridweave-sdk && python3.12 -m venv .venv && source .venv/bin/activate
pip install gridweave_sdk-*.whl jupyterlab
jupyter lab onboarding.ipynb

# Or into an existing env:
V=$(curl -s https://pub-c48a651bbb2f42988602aa11bb9d9267.r2.dev/tarball/sdk-latest-version.txt)
pip install https://pub-c48a651bbb2f42988602aa11bb9d9267.r2.dev/tarball/gridweave_sdk-${V}-py3-none-any.whl

Python 3.12 required — functions are shipped to the workers with cloudpickle, which won't unpickle across versions.

import gridweave
gridweave.auth("YOUR_TOKEN", platform_url="https://platform.gridweave.io")
gridweave.resources()     # nodes, vendors, and free VRAM

Run anything, on exactly the hardware you want¶

Decorate a function, run() it — it executes on a worker, streams its stdout back, and returns what it returned. vram is how you ask for GPUs:

@gridweave.remote()                # no vram → CPU
def hello(name):
    return f"hi {name}"

@gridweave.remote(vram="4GB")      # a GPU with ≥4 GB free
def matmul():
    import torch
    x = torch.randn(4096, 4096, device="cuda")
    return (x @ x).mean().item()

gridweave.run(matmul)

The cluster is a single pool of mixed hardware, and the same knobs pin any of it:

Arg	Effect
`vram="24GB"`	a GPU with ≥24 GB free — omit for CPU; ask for more than one GPU has → multi-GPU (auto `CUDA_VISIBLE_DEVICES`); two small jobs → share a GPU
`vendor="amd"` / `"nvidia"`	pin the GPU brand (default: auto)
`node="hostname"`	pin one machine (from `resources()`)
`paid=True`	run on provider hardware (paid credits) vs. the free org pool

@gridweave.remote(vram="16GB", vendor="amd")     # a 16 GB AMD GPU
def on_amd(): ...

@gridweave.remote(vram="100GB")                  # spans several GPUs automatically
def big(): ...

Serve a model — and get paid for it¶

Deploy a model behind an endpoint. vLLM models get an OpenAI-style chat interface; any HTTP container works via spec=:

ep = gridweave.serve(model="Qwen/Qwen2.5-0.5B", vram="4GB", name="qwen")
ep.chat("Explain quantum computing in one sentence.")

ep = gridweave.serve(spec={"image": "kennethreitz/httpbin", "port": 80,
                           "actions": {"echo": {"method": "POST", "path": "/post"}}},
                     name="httpbin")
ep.call("echo", data={"hello": "cluster"}).json()

gridweave.endpoints(); gridweave.stop("qwen"); gridweave.start("qwen"); gridweave.delete("qwen")

Make it public and paid and you run a tiny inference business: callers pay your price (minus a small platform fee), you pay the GPU rental while it's up, and you keep the spread — on your own hardware you keep both sides.

gridweave.serve(model="Qwen/Qwen2.5-0.5B", vram="4GB", name="qwen-paid",
                paid=True, public=True, price_per_call=0.01)   # or price_per_1m_input/output

Every vLLM endpoint is also on the OpenAI-compatible API (model id {username}/{name}), so Open WebUI, curl, or the OpenAI SDK work with no gridweave install:

curl -H "Authorization: Bearer $TOKEN" \
  -d '{"model":"you/qwen","messages":[{"role":"user","content":"hi"}]}' \
  https://platform.gridweave.io/v1/chat/completions

Distributed training¶

@gridweave.train(gpus=N) runs your function once per rank with PyTorch Distributed (NCCL/RCCL) already wired up — use Trainer/DDP as normal:

@gridweave.train(gpus=2)
def finetune():
    ...                          # build model + Trainer, then trainer.train()
    import os; return {"rank": int(os.environ.get("RANK", 0))}

gridweave.run(finetune)

Async, parallel, files, audit trail¶

h = gridweave.submit(matmul)                 # non-blocking
gridweave.status(h); gridweave.get(h)        # poll / block-for-result
gridweave.gather([gridweave.submit(matmul) for _ in range(10)])   # 10 at once

uri = gridweave.upload("data.csv"); gridweave.download(uri, "data.csv")   # cluster S3
gridweave.chain(limit=10)                    # your on-chain audit events
gridweave.chain_rpc("status")                # or verify the ledger yourself — any read-only query

Learn by doing¶

onboarding.ipynb (in the tarball) runs all of it against a live cluster — CPU / GPU / fractional / multi-GPU jobs, GPT-2 fine-tuning, vLLM + gated + S3 + custom-image serving, the OpenAI API, Open WebUI, paid endpoints, and the audit chain. Each cell is independently re-runnable.

The SDK sends its version on every call; on a major mismatch the call is rejected with an "SDK UPDATE REQUIRED" message and the exact pip install to run.