Metering Service¶
The metering service is the platform's usage ledger. When a job finishes, the gateway sends it what the job used — wall-clock time, CPU, GPU VRAM — plus the cost it was billed; metering stores one row per job and answers "how much has this user used?" and "what did the whole cluster cost this week?". Small FastAPI app on port 8300, backed by Postgres (creates its own database on first start).
Metering does not charge anyone, and it does not calculate cost. Live billing — deducting credits, paying providers, and computing the cost — is the gateway's job (see
job_gateway). Metering is the after-the-fact record for reporting and dashboards; the gateway posts to it fire-and-forget when a job completes.
Cost — recorded, not computed¶
Metering stores the cost the gateway sends, verbatim — it never calculates cost itself. That keeps one source of truth: the gateway bills the job and reports what it charged, so the ledger can't disagree with the actual bill.
The formula lives in the gateway (billing.hourly_cost): BASE_RATE + gpu_vram_gb × VRAM_RATE per hour — defaults 0 + 1, i.e. 1 credit per VRAM-GB-hour, CPU-only free. gpu_vram_gb is the reserved tier the user requested (not peak usage); gpu_memory_peak_mb is stored for reference, to help users right-size.
Where the numbers come from¶
metering.py has a small standalone measure(fn, …) helper: it runs a function and captures wall/CPU time and GPU VRAM (via pynvml). The gateway uses it inside Ray workers to collect a job's metrics, then POSTs them (with the cost) here. It's also handy for local benchmarking.
Endpoints¶
The write is admin-only (only the gateway writes); reads are open (the platform proxies them with user auth).
| Method | Path | Auth | What it does |
|---|---|---|---|
| GET | /healthz |
— | Health check |
| POST | /v1/usage/record |
admin | Record a completed job's usage + cost (gateway; idempotent per job_id) |
| GET | /v1/accounting/user/{user} |
— | One user's totals (jobs, wall-seconds, cost) |
| GET | /v1/accounting/job/{job} |
— | One job's usage record (404 if none) |
| GET | /v1/accounting/report?since=&until= |
— | Cluster-wide usage by user, highest cost first |
Writes require X-Admin-Token (ADMIN_TOKEN, fails closed if unset). record upserts on job_id, so a re-sent record updates in place instead of duplicating.
Storage¶
One table, usage_records: job_id (unique), user_id, wall_seconds, cpu_seconds, gpu_vram_gb, gpu_memory_peak_mb, cost, recorded_at — indexed by user and by time.
Configuration¶
| Variable | Default | Description |
|---|---|---|
METERING_DATABASE_URL |
— (required) | Postgres connection string (DB auto-created if missing) |
METERING_DB_POOL_SIZE |
10 | Max pooled DB connections |
ADMIN_TOKEN |
— | Shared secret for the usage-record write (unset ⇒ writes rejected) |
CORS_ORIGINS |
* |
Allowed origins |
RATE_LIMIT_PER_MINUTE |
120 | Per-IP rate limit |
Consumers (gateway, platform) reach it at METERING_URL (http://metering-service:8300).
Running & testing¶
# Unit tests need a Postgres (test compose exposes one on host port 5433):
docker compose -f docker-compose.test.yml --profile credit up postgres -d
pytest tests/metering_service/ -v --ignore=tests/metering_service/test_docker.py
# Full stack (metering runs under the `credit` profile) for the Docker test:
docker compose -f docker-compose.test.yml --profile credit up --build -d
pytest tests/metering_service/test_docker.py -v
Coverage: test_accounting (record/upsert, user summary, cluster report), test_metering (measure() metrics), test_endpoints (HTTP + admin auth, verbatim cost), test_docker (deployed). Interactive API docs at /docs and /redoc.