Browsing Tag
AI Inference
5 posts
AI inference infrastructure, model serving, token generation, inference clouds, latency, throughput, and production AI workloads.
OpenAI’s Jalapeño Benchmarks Turn Inference Power Into the AI Chip Fight
OpenAI published first benchmark results for Jalapeño, its Broadcom-built inference chip, claiming higher performance per watt and lower latency than leading commercial AI systems. The useful question is not whether it replaces Nvidia immediately, but whether custom inference silicon can turn power limits into a product advantage for ChatGPT, Codex, and agentic AI workloads.
Etched’s $1B Sohu Backlog Turns AI Inference Into the Next Chip Fight
Etched says it has raised $800 million, signed more than $1 billion in customer contracts, and started production of its Sohu-based inference racks. The startup’s transformer-specialized chip is a serious bet that AI’s next hardware fight will be won on serving models, not just training them.
OpenAI’s Jalapeño Chip Puts Inference Costs at the Center of the AI Race
OpenAI and Broadcom unveiled Jalapeño, OpenAI’s first custom inference accelerator for large language models. The chip is less about replacing Nvidia overnight than controlling the cost, latency, and supply of the compute that runs products like ChatGPT, Codex, and the API.
Qualcomm’s Modular Deal Is a $3.9 Billion Bet on AI Software Portability
Qualcomm agreed to acquire Modular in a nearly $4 billion stock deal, giving its AI data center push a software layer built around portable model deployment. The move is aimed at a practical bottleneck in AI infrastructure: making models run efficiently across CPUs, GPUs, NPUs, and custom accelerators without locking developers into one hardware stack.
Groq’s $650M Raise Makes AI Inference the New Cloud Fight
Groq raised $650 million to expand its AI inference cloud, with 13 data centers, more than five million developers, NVIDIA LPX integration, and a 200 MW capacity target by the end of 2027. The deal shows why serving AI models is becoming its own infrastructure market, separate from the training race.