OpenAI published the first benchmark results for Jalapeño on August 25, moving its Broadcom-built inference chip from a June hardware announcement into a measurable challenge to today’s AI accelerator market. The company says the chip delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than comparison systems across GPT-OSS 120B, DeepSeek R1, and Kimi K2.5 1T.
The numbers matter because OpenAI is not pitching Jalapeño as another training GPU. It is an inference chip, built for the expensive, repeated work of running trained models for products such as ChatGPT, Codex, API workloads, and multi-step agents. In that setting, the bottleneck is increasingly not only how many accelerators a company can buy. It is how many useful tokens can be served inside a fixed power envelope, with latency low enough that interactive products still feel responsive.
OpenAI’s benchmark post says Jalapeño was tested with SemiAnalysis’ InferenceX suite, using matched user experience rather than a raw chip-only metric. The company argues that performance per unit of power is the more useful standard for production AI, especially for agents, where small delays can accumulate across browsing, coding, tool use, retrieval, and verification steps.
What OpenAI says it measured
OpenAI framed the results around three public models rather than only its own internal models. On GPT-OSS 120B, the appendix lists Jalapeño at about 85,448 mixed tokens per second per kilowatt compared with 44,960 for GB200 at peak, with end-to-end latency of 1.03 seconds versus 1.80 seconds in the tested setup. On DeepSeek R1 670B, OpenAI reports about 19,641 mixed tokens per second per kilowatt versus 11,781 for GB300, with latency of 1.65 seconds versus 5.99 seconds. On Kimi K2.5 1T, the company reports about 18,195 mixed tokens per second per kilowatt versus 11,862 for GB300, with latency of 1.56 seconds versus 5.31 seconds.
Those figures are not the same as saying every AI workload will run faster or cheaper on Jalapeño. They show OpenAI trying to prove a narrower and more commercially important point: for language-model inference, a custom chip and system can keep throughput high without giving up the low latency users expect from interactive AI products.
The chip’s design is aimed at the two main phases of inference. Prefill, when the system processes the prompt, leans heavily on compute. Decode, when the model generates tokens one by one, is often constrained by memory bandwidth and the movement of state, including the KV cache. OpenAI says Jalapeño was designed to reduce data movement, keep model state local where possible, and use an integrated network so the request can move through the system with fewer latency penalties.
Why the power metric matters
For a frontier AI company, performance per watt is not an abstract benchmark preference. Data centers are increasingly constrained by power availability, grid connections, cooling, backup systems, and local permitting. If a company can serve more useful AI work from the same megawatts, it can improve capacity without waiting for every new power project, data hall, or GPU shipment to arrive.
That is why the Jalapeño results are really about operating leverage. Faster, more efficient inference can lower the cost of each answer, make agentic workflows less sluggish, and let OpenAI allocate scarce infrastructure to more users or heavier products. It also gives OpenAI a path to tune hardware, serving software, model architecture, and product behavior together instead of adapting everything to merchant silicon alone.
The company is careful not to present Jalapeño as a full replacement for Nvidia. OpenAI says it will continue to deploy accelerators from Nvidia and other partners for training and inference. Jalapeño is expected to begin deployment inside OpenAI’s own compute infrastructure by the end of 2026, with later generations already in development.
The important caveats
SemiAnalysis, which published its own technical look at Jalapeño, called the chip a generalized inference accelerator rather than a narrow OpenAI-only part. Its write-up also adds the most useful caution: the public results are based on 8k input / 1k output style benchmark runs, not the full set of long-context, multi-turn agent scenarios that stress routers, prefix caching, cache management, offload infrastructure, and serving reliability over longer sessions.
That distinction matters for real AI agents. A single-turn inference benchmark can show strong latency, throughput, and watt efficiency, but multi-step work creates different stress. A coding agent may read a large repository, call tools, revise patches, run tests, and return to earlier context. A workplace agent may move among email, Slack, documents, websites, and approval flows. Those workloads test scheduling, cache behavior, network locality, tail latency, fault handling, and policy enforcement as much as the accelerator itself.
SemiAnalysis also notes that the comparison set is still evolving. Nvidia’s newer Rubin systems are arriving, and early silicon plus early software can change quickly on both sides. OpenAI’s results are still early enough that production qualification, software maturity, real traffic, and broader model coverage will decide whether the lab advantage becomes a durable infrastructure advantage.
AI is entering the chip-design loop
One of the more interesting parts of OpenAI’s disclosure is not only that the chip runs AI models. It is that AI models helped build and program the chip. OpenAI says earlier models assisted chip design and bring-up, while newer models helped optimize and program Jalapeño. For selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than existing human-expert implementations, according to the company.
OpenAI also says Codex with GPT-Astra helped bring three open-weight models that were not part of the original production plan to high performance within two months. That claim points to a broader loop: AI systems are becoming tools for accelerator programming, kernel tuning, verification, and workload adaptation. If that holds up at production scale, the software side of custom silicon may become less of a barrier than it has been for some previous AI chip efforts.
What to watch next
The next test is deployment, not another chart. OpenAI still has to qualify Jalapeño for production, operate it at scale, prove reliability under live traffic, and show that the efficiency gains survive broader workloads than the published benchmarks. The most important evidence will be whether Jalapeño changes user-visible latency, API pricing, Codex throughput, agent reliability, or OpenAI’s ability to absorb demand spikes without simply buying more outside capacity.
For the wider chip market, Jalapeño is another sign that AI companies with enough demand may no longer treat accelerators as interchangeable rented capacity. Google, Amazon, Microsoft, Meta, and now OpenAI are all pushing some version of custom or semi-custom AI silicon. The difference is that OpenAI’s first public performance case is aimed directly at inference, where product quality, power limits, and gross margin meet every time a model answers.
That does not end Nvidia’s advantage. It does make the AI chip fight more specific. Training scale, inference latency, watts per token, memory bandwidth, networking, software maturity, and real agent workloads are becoming separate battlegrounds. Jalapeño’s first benchmark results suggest OpenAI now has a credible weapon in one of the most important ones.
