Decision API Design: Typed Answers, Calibrated Confidence, and Code That Can Actually Act

Every team that has wired a chat model into a production workflow eventually hits the same wall. You write a careful prompt, ask the model to “reply in JSON,” parse the response, validate it against a schema, retry when validation fails, and then discover that the model invented a label you never defined. The decision was supposed to take four milliseconds of compute; instead it took two round trips, a retry budget, and a fallback path nobody tested. The problem was never the model’s reasoning. It was the interface.

A decision API changes the interface. Instead of generating prose and hoping it can be coerced into structure, you declare the question in code and receive a typed answer back — in one pass, with a confidence value attached. For developers building routing, gating, scoring, and monitoring systems, that shift is less about model quality and more about whether the result can be trusted inside an if statement.

What a Decision API Returns — and Why Typed Output Changes Your Architecture

A decision API inverts the usual contract. With text generation, the question lives inside the prompt as natural language and the answer comes back as prose you must interpret. With a decision endpoint, the question is part of the request schema and the answer arrives as one of a small, fixed set of types. You post state — free text, structured JSON, or up to 16 images and video frames — to a single HTTPS endpoint, along with the questions you defined in your own codebase, and every answer returns in one call.

The response vocabulary is deliberately narrow, and that narrowness is the point:

choice returns exactly one label from the set you supplied, so it feeds a switch statement without a normalisation layer. score returns a level on an ordered scale you define, which is what priority queues, severity tiers, and risk bands actually need. truth returns a calibrated probability between 0.00 and 1.00 rather than a yes/no — the distinction matters enormously when you are deciding whether to auto-approve a refund. number returns a value on a grid you specify, useful for quantities, wait-time estimates, and quote adjustments. point and box return locations inside an image, which turns visual inspections into coordinates your pipeline can crop, flag, or re-shoot.

Each of these maps cleanly onto control flow. A choice becomes a branch, a score becomes a threshold comparison, a truth becomes a gate, a number becomes an input to pricing or capacity logic, and a point or box becomes a spatial instruction. No regex, no schema-repair prompt, no “please return only valid JSON” pleading. This is the contract a decision api such as DecisionNode is built around: state in, typed answers out, with the question shape owned by the caller.

The architectural consequence runs deeper than ergonomics. When decisions return as typed values, they stop being asynchronous, best-effort side quests and become composable function calls inside a request handler. They also become testable, provided the engine is deterministic — the same request yielding the same answer means you can replay last month’s traffic against a new rule set and diff the outcomes, and your integration tests stop flaking on sampling variance.

Confidence, Latency, and Cost: The Three Numbers That Decide Whether It Ships

Typed output solves the parsing problem. The shipping problem is different: can a service act on the answer without a human in the loop? That question gets answered by three engineering constraints — calibration, latency, and unit cost — and each one has a measurable answer.

Calibration is the one teams underestimate. A confidence score is only useful if it is honest at scale: across many calls, answers returned at 0.82 confidence should be correct roughly 82% of the time. When that holds, you can design around it. Set a threshold per decision type; above it, act immediately and log the outcome. Below it, choose a defined fallback — enrich the state and re-check, route to a human queue, or take a conservative safe path such as holding an order or flagging an image for manual review. This act-or-escalate pattern replaces the binary “the model said yes” logic that makes rule-based automation brittle, and it gives product teams a real dial: raise the threshold for refunds, lower it for spam quarantine.

Latency determines placement. DecisionNode Flash runs at roughly 5 ms server-side and 19 ms end to end; DecisionNode MoE, the larger model, sits around 9 ms. Those numbers are inside the budget of a synchronous API handler, which means the decision can happen while the user is waiting rather than in a background job that reports back twenty seconds later. For throughput-heavy loops — anything above roughly 10 decisions per second, as seen in drones, robots, simulators, and machine monitoring — sessions over WebSocket keep the loop tight instead of paying per-call connection overhead.

Cost determines scope. Flash is priced at $0.035 per 1M input tokens and MoE at $0.042, with output tokens always free because there is no generated text to bill. Batch discounts and a free daily tier make offline evaluation realistic: teams can score a large replay corpus, compare engine output against historical human decisions, and tune thresholds before a single request touches production. SDKs for TypeScript, Python, Go, Ruby, and curl mean the first integration is usually an afternoon, not a sprint.

Where Decision APIs Replace Hand-Written Rules, Prompt Chains, and Manual Review

The clearest signal that a decision API fits a workflow is the presence of a long, ugly rules file or a human whose job is to look at something and pick a category. Both are decision surfaces in disguise.

In support operations, a ticket triage endpoint takes the message body plus account metadata and returns a choice over your routing labels, a score for priority, and a truth value for questions like “does this require a refund?” Confidence does the rest: high-confidence tickets route automatically, low-confidence ones land in a queue with the engine’s suggested label attached so the agent starts from a strong prior instead of a blank form.

In risk and commerce, the same shape applies to fraud screening, promo-abuse checks, and order and shipping decisions. A score output feeds a risk tier, a choice output selects a fulfilment path, and a number output estimates delay. Because confidence is attached to every answer, borderline cases can be held for review rather than approved by a coin flip — and because the engine is deterministic, a disputed decision can be reproduced exactly from the logged request.

Physical and visual operations are where the image and video inputs earn their keep. Up to 16 images or frames travel with the request, and boosters for OCR, face detection and recognition, human age estimation, AI-generated image detection, depth, and segmentation collapse what used to be a pipeline of specialised models into a single call. A production-line quality check can return a pass/fail choice plus a box around the defect; a warehouse monitoring system can return zone occupancy from a frame; an insurance intake flow can read a document with OCR and flag a suspiciously synthetic photo in the same round trip. Moderation and content gating follow the same pattern with text state.

AI agents deserve a separate mention, because they are the fastest-growing consumer of this interface. An agent that can call a decide endpoint before acting — through MCP tools, webhooks, or an SDK — gets something a chat completion cannot reliably give it: a stopping condition. The calibrated probability tells the agent whether to proceed, gather more context, or hand off. For platform teams, the practical starting point is modest: replay a week of real decisions, diff the engine’s typed answers against what humans or legacy rules chose, adjust thresholds where the confidence curve and the business cost disagree, and only then put the endpoint in the hot path.