Silicon Valley’s most talked-about AI lab has stopped just buying chips and started designing them. OpenAI’s first in-house accelerator, Jalapeño, is built to run trained models — the inference stage, where an AI actually answers your prompt or drives an autonomous agent — and OpenAI claims it does so faster and more efficiently than rival hardware.
In a blog post published Tuesday, and a follow-up briefing with reporters, OpenAI hardware vice president Richard Ho pitched Jalapeño as the “best of both worlds.” His argument: most AI systems force a compromise between latency (how quickly a response begins) and throughput (how many requests you can process at once). Push one up and the other usually sags. Ho says Jalapeño delivers low latency and high throughput at the same time — the holy grail of inference serving.
Jalapeño is an Application-Specific Integrated Circuit (ASIC), co-developed with Broadcom. That’s the same playbook Google used with its TPUs and Amazon with Trainium: rather than chase a general-purpose GPU, you carve the transistors around one job and squeeze out efficiency the flexible parts can’t match. OpenAI’s chosen job is inference, and the design decisions all point that way.
The numbers OpenAI has shared are aimed squarely at hyperscale deployment rather than a spec-sheet beauty contest:
- 1.7 exaflops of 4-bit compute per 128-chip rack — low-precision math being the currency of modern inference
- 15.4 TBps of memory bandwidth per package, the kind of firehose large models need to avoid starving their compute
- 700W rated capacity, with measured sustained power sitting at or below 550W on tested workloads
Those power figures matter. Inference is where AI companies burn most of their operational energy — every chatbot reply, every agent action — so a chip that can run heavy workloads comfortably under its rated ceiling translates directly into datacenter economics.
Jalapeño first surfaced in October 2025 and was formally unveiled on June 24, 2026. Don’t expect to buy one, though. This is captive silicon: OpenAI is building Jalapeño to run its own models, not to sell to competitors or slot into your workstation. Initial deployment is planned for the end of 2026 in very small volumes, with a more meaningful rollout arriving in 2027.
The strategic read is hard to miss. By designing its own inference accelerator, OpenAI joins the growing club of AI heavyweights trying to loosen their dependence on off-the-shelf GPUs — controlling cost, supply and the exact shape of the hardware their models run on. Whether Jalapeño’s “best of both worlds” claim holds up outside OpenAI’s own benchmarks is something we’ll only judge once the racks light up.
