Definition
Temperature
Temperature is a setting that controls how random a language model's output is. At low temperature the model strongly favors its most likely next token, producing focused, deterministic-leaning text; at high temperature it samples more broadly, producing varied, creative, and less predictable output. It tunes the trade-off between consistency and diversity in generated responses.
Key takeaways
- Temperature is a setting that controls how random a language model's output is. At low temperature the model strongly favors its most likely next token, producing focused, deterministic-leaning text; at high temperature it samples more broadly, producing varied, creative, and less predictable output. It tunes the trade-off between consistency and diversity in generated responses.
- At each step a language model produces a probability distribution over possible next tokens.
- Planoda runs accuracy-critical features — extraction, classification, structured proposals — at low temperature so AI output stays consistent and reviewable under propose/approve.
At each step a language model produces a probability distribution over possible next tokens. Temperature reshapes that distribution before a token is sampled. A low value sharpens it, concentrating probability on the top candidates so the model almost always picks the safest continuation. A high value flattens it, giving lower-probability tokens a real chance and widening the range of plausible outputs.
The right setting depends entirely on the task. For extraction, classification, structured output, or anything that must be reliable and repeatable, low temperature is correct — you want the same well-formed answer every time. For brainstorming, creative writing, or generating varied options, raising it introduces the useful unpredictability that keeps output from feeling rote.
Temperature is one of several sampling controls and is often adjusted alongside others like top-p. A common misconception is that low temperature guarantees a single fixed answer; it heavily biases toward one but, except at zero, does not make generation fully deterministic. It also never improves factual accuracy — a confident wrong answer stays wrong at any temperature.
Planoda runs accuracy-critical features — extraction, classification, structured proposals — at low temperature so AI output stays consistent and reviewable under propose/approve.
Related terms
- Large Language Model (LLM)A large language model is an AI system trained on vast amounts of text to predict and generate language, enabling it to answer questions, summarize, write, and reason over natural language. LLMs power modern AI assistants and agents. They are probabilistic next-token predictors, which makes them remarkably capable but also prone to confident errors.
- InferenceInference is the act of running a trained model to produce output from new input — the moment a model is actually used, as opposed to trained. Every prompt sent to a language model triggers an inference pass. Inference dominates the running cost and latency of AI features, since it happens on every request while training happens once.
- Hallucination (AI)An AI hallucination is output from a language model that is fluent and confident but factually wrong, fabricated, or unsupported by the model's inputs — invented citations, fake quotes, or made-up details. It arises because models predict plausible text rather than retrieve verified facts, making hallucination an inherent risk that must be mitigated, not a bug that's simply fixed.
- System PromptA system prompt is a set of instructions given to a language model before any user input, defining its role, behavior, tone, constraints, and the tools or knowledge it should use. It frames every response in the conversation that follows. Unlike a user message, it sets the persistent ground rules the model is meant to honor throughout an interaction.
- Model GatewayA model gateway is a unified layer that sits between an application and one or more AI model providers, routing requests through a single interface. It centralizes provider abstraction, authentication, rate limiting, cost tracking, caching, and failover — so an app can swap models, enforce spending budgets, and observe usage without rewiring every call site.