OpenAI Decisions API Slashes Inference Time for AI Tutors
OpenAI has made its Decisions API generally available, cutting AI decision-making time from seconds to milliseconds to power real-time applications like language tutors and voice assistants.
OpenAI made its Decisions API widely available to all users on October 22, 2026. The tool dramatically reduces AI inference time, enabling faster and cheaper real-time applications like voice assistants and AI language tutors.
Announced at the company's developer day on September 29, the API is now public with new voice and visual modalities. OpenAI optimized computing resource allocation geographically to reduce global latency. Nikunj Handa, product lead for OpenAI's APIs, stated that the speed improvement is fundamental. "The big unlock with Decisions API has been the speed, shrinking things from one to three seconds to 100, 120, 180 milliseconds," Handa said. The company reconfigured its ChatGPT Luna model to create the API, focusing on accelerating inference and lowering cost rather than building a new model from scratch.
Initial customers including Box, Datadog, Ramp, and Speak have already replaced larger language models with the Decisions API to increase throughput. At Speak, an AI language tutor platform, CTO Andrew Hsu reported the API cut the time between actions from 1.4 seconds to 0.13 seconds. OpenAI also used the API internally to improve the responsiveness of its own Dots voice assistant. Handa suggested that further speed increases could unlock new use cases, such as processing every frame in a video.
TypeSafe's Jev Model Created the Super Classifier Category
The market for fast, low-cost classification intelligence was pioneered by TypeSafe, a startup now valued at $7.6 billion. Founded by a former frontier lab researcher, the company developed the Jev model, which created the category for deep-learning classifiers that act as super decision-makers. These models select quickly and return confidence ratings. The latest Jev model added image support and smart-if statements. Developers are adopting Jev models to replace larger, slower LLMs for speed and cost savings. TypeSafe's offerings are currently the most widely-queried on the OpenRouter platform.
Industry Benchmarks Show Mixed Performance Rankings
Public benchmarks present a varied landscape for evaluating decision models. Two major indices show different leaders, with OpenAI's solution not yet at the top.
| Model | AIM System One Index Rank | Artificial Analysis Decision Index Rank |
|---|---|---|
| Liquid D1 Ultra | 1 | - |
| Base Lab Elephant-1 | - | 1 |
| TypeSafe Jev | 7 | 3 |
| OpenAI Decision Model | 8 | 14 |
TypeSafe's own JevMan benchmark, where models play PacMan, tells a different story. The latest Jev model tops that ranking. Felix Wunderlich, chief growth officer at Opper AI, described JevMan as an interesting test due to its nonstop stream of quick, high-stakes decisions. TypeSafe co-founder Diogo Almeida distrusts external benchmarks and encourages developers to test models on their individual workloads.
Developers Replace LLMs for Speed and Cost Savings
The shift is driven by practical needs. Engineers are replacing large language models with specialized decision models to gain speed and reduce costs across various workflows. Handa noted that colleagues recognized Jev's massive potential, suggesting there are 'like a million use cases' for this technology. He also pointed out that OpenAI's model may have an advantage in multi-modality, particularly for image classification, where Luna shows better calibrations. Users lack a clear standard for a good decision model, as the right balance of intelligence, cost, calibration, and speed shifts with each unique workload.
OpenAI expects speed to keep climbing. Handa is looking ahead, asking if the API can reach 60 milliseconds to enable an API call for every video frame.
