Credit Service¶
The credit service is the bank of the platform. It tracks how many compute credits each user has, hands out free credits over time based on how much GPU capacity the cluster has, records every transaction, and stores the pricing knobs the rest of the system reads. It's a small FastAPI app on port 8400, backed by Postgres.
It never runs jobs or touches GPUs itself. Other services call it: - the gateway asks "does this user have enough credits?" before starting a job, and tells the bank to deduct credits while the job runs; - the platform service proxies the read endpoints so users can see their balance; - Stripe (via platform) tells it to add paid credits when someone buys them.
Two kinds of credit¶
Everything in the bank is one of two types:
| Free credits | Paid credits | |
|---|---|---|
| Come from | Faucet + automatic fair-share accrual | Bought with money (Stripe) |
| Transferable to another user? | No | Yes |
| Spent on org hardware | Yes (spent first) | Yes (after free runs out) |
| Spent on a provider's hardware | No | Yes (only paid credits rent provider GPUs) |
1 credit = 1 GB-hour of GPU memory. Renting an 80 GB A100 for an hour costs 80 credits. CPU-only work is free. Balances live in credit_balances (one row per user: balance for free, paid_balance for paid); every change is appended to the credit_transactions ledger.
How credits come in¶
Free — the faucet gives new users a starting balance.
Free — fair-share accrual (the interesting part). A background loop (scheduler.py) tops up every user's free balance once a minute, based on how big the cluster is right now:
the cluster emits = total GB of VRAM across all live GPUs, per hour
each user earns = cluster total ÷ number of users (per hour)
up to a cap of = their hourly rate × 24 hours
Worked example: the cluster is 8× A100 (80 GB) = 640 GB, so it emits 640 credits/hour. Split across 10 users, each earns 64 credits/hour, capped at 1,536 (a day's worth). One A100 costs 80 credits/hour, so a user can run a single A100 for ~19 hours before running dry — and it refills at 64/hour while they're idle. The cap stops anyone hoarding weeks of compute by sitting idle.
Two dials keep it fair and busy (all in allocation_config, editable via the API):
- a floor (min_pool_size, default 10) on the "number of users" divisor, so the first user on a new cluster doesn't get everything;
- an over-allocation boost (×1.5 when the cluster is <30% used) so idle GPUs get handed out faster.
A user accrues only once flagged accrual_eligible. New free users get that flag at signup (faucet → platform /onboard seeds them at the cap); paid signups and bare-mint/system accounts stay ineligible.
Paid — Stripe. When someone buys a credit pack, Stripe calls the platform's webhook, which tells the bank to mint that many paid credits to the buyer. It's idempotent (a stripe_events table drops duplicate deliveries), so a retried webhook never double-credits.
How credits move between users¶
A logged-in user can send their paid credits to someone else with POST /v1/credits/transfer (the platform fills in from_user = you, so you can only give away your own). Free credits are locked. The transfer locks both accounts in a fixed order so two opposite transfers can't deadlock.
How credits are spent¶
The gateway does the spending, not this service. While a job or endpoint runs, the gateway calls POST /v1/credits/deduct roughly once a minute for the GPU-time used, and the bank picks which balance to draw from:
- org hardware → free credits first, then paid;
- a provider's hardware → paid credits only.
If a user runs out mid-job, the deduct is rejected and the gateway stops the workload.
Reserving capacity for paid customers¶
monetization_pct sets aside a share of the cluster for paid work on org hardware. It's one cluster-wide reserve budget (paid_reserve). Each accrual tick, monetization_pct% of the cluster's tokens flow into the reserve — capped at 24 hours' worth, exactly like a user — and the remaining (100 − pct)% is split among free users.
When a paid customer runs on org hardware and dips into paid credits, that spend draws both their paid balance and the reserve down together. If the reserve is empty, paid-on-org work is throttled until it refills — that's the reservation holding the line for the free tier. Provider hardware is separate: renting it never touches the reserve.
Set monetization_pct = 0 (the default) and the reserve is dormant — paid credits spend on org hardware freely, exactly as before. Inspect the live reserve at /v1/credits/allocation/status.
Pricing knobs¶
The bank also stores two numbers the gateway reads when money changes hands:
- gpu_pricing — a per-node hourly rental rate for provider hardware.
- revenue_config.platform_fee — the platform's cut of paid endpoint-call revenue (default 0.10, admin-editable at /v1/pricing/revenue-splits).
How that money splits is the gateway's job (documented there): providers earn 100% of GPU rental, and paid endpoint-call revenue splits deployer (1 − platform_fee) / platform platform_fee.
Endpoints¶
Reads are open (the platform enforces user auth in front). Writes require the shared X-Admin-Token — the bank fails closed if ADMIN_TOKEN is unset, so nothing mints or moves credits without it.
| Method | Path | Auth | What it does |
|---|---|---|---|
| GET | /healthz |
— | Health check |
| GET | /v1/credits/balances |
— | Every user's balance |
| GET | /v1/credits/{user}/balance |
— | One user's free + paid balance |
| GET | /v1/credits/{user}/transactions |
— | Their ledger, newest first |
| GET | /v1/credits/allocation/config |
— | Current accrual settings |
| GET | /v1/credits/allocation/status |
— | User count + live paid-reserve balance |
| GET | /v1/pricing/gpu · /gpu/{node} · /gpu/batch |
— | GPU rental rates |
| GET | /v1/pricing/revenue-splits |
— | Current platform_fee |
| POST | /v1/credits/mint |
admin | Add credits (free or paid) |
| POST | /v1/credits/deduct |
admin | Charge for GPU time |
| POST | /v1/credits/transfer |
admin | Move paid credits between users |
| POST | /v1/credits/allocation/seed/{user} |
admin | Give a user their starting balance |
| PUT | /v1/credits/allocation/config |
admin | Change accrual settings, incl. monetization_pct |
| POST·DELETE | /v1/credits/reset-all · /test-users |
admin | Ops / test cleanup |
| PUT | /v1/pricing/gpu · /revenue-splits |
admin | Set rental rates / platform fee |
Every mint, deduct, and transfer also emits an event to chain_service for the audit ledger.
Configuration¶
| Variable | Default | Description |
|---|---|---|
CREDIT_DATABASE_URL |
— (required) | Postgres connection string |
CREDIT_DB_POOL_SIZE |
10 | Max pooled DB connections |
ADMIN_TOKEN |
— | Shared secret for write endpoints (unset ⇒ writes rejected) |
GATEWAY_URL |
http://job-gateway:8200 |
Where accrual reads the live cluster size |
PLATFORM_FEE |
0.10 | Seeds revenue_config.platform_fee on first startup |
CORS_ORIGINS |
* |
Allowed origins |
RATE_LIMIT_PER_MINUTE |
120 | Per-IP rate limit |
Callers reach it at CREDIT_SERVICE_URL (http://credit-service:8400). The gateway only enforces any of this when CREDITS_ENABLED=true.
Running & testing¶
# Unit tests need a Postgres (the test compose exposes one on host port 5433):
docker compose -f docker-compose.test.yml --profile credit up postgres -d
pytest tests/credit_service/ -v --ignore=tests/credit_service/test_docker.py
# The one Docker test runs against the full stack with enforcement on:
CREDITS_ENABLED=true docker compose -f docker-compose.test.yml --profile credit up --build -d
pytest tests/credit_service/test_docker.py -v
What the tests cover: test_allocator (accrual math), test_credits / test_dual_balance / test_cross_flows (balances, free-vs-paid routing, overflow), test_scheduler (the accrual loop), test_paid_reserve (the monetization reserve), test_endpoints, test_stress (concurrent deducts/transfers), test_docker (deployed). Interactive API docs at /docs and /redoc.