When people say “run an LLM,” they almost always mean inference: send a prompt, get a completion. Scalattice is an inference network. We do not train your models on provider GPUs. Catalog weights are hosted and served for chat and completion traffic.
Tokens
Models do not see words the way you do. Text is split into tokens: pieces of words, punctuation, code. Input tokens are your prompt (and conversation history). Output tokens are the reply.
Billing on Scalattice is published per million tokens, separately for input and output. Output usually costs more because generating it is the heavy work.
Context
Every model has a max context: how many tokens of prompt + history + reply it can hold at once. Long threads burn input tokens even if the new user message is short. Trim history or summarize when you do not need the full transcript.
Streaming
Streaming returns tokens as they are produced instead of waiting for the full answer. Latency to first token drops; total work is similar. Most chat UIs should stream.
What happens on a Scalattice request
- Your app calls the OpenAI-compatible API with a key and a catalog model ID.
- Optional headers set region, security tier, and output vetting.
- The network routes to a ready provider machine that has that model loaded.
- Tokens are metered; credits debit on the developer side; providers earn their share on completed work.
If capacity is short, routing falls back according to policy. That is the point of a network instead of a single box.
Why prices move
Catalog list prices are the baseline. Demand can adjust live rates. You still see published numbers in Cloud and on the public pricing snapshot, with no opaque enterprise quote for standard catalog traffic.
Practical tips
- Measure cost per successful task, not only per million tokens.
- Shorter system prompts and less redundant history save money every day.
- Use a smaller model until quality forces an upgrade.
- Read the developers page and Cloud docs before you invent your own retry logic.