Tag: AI Inference
Fal lands $1.3B and a $16B price tag on inference demand
Serverless model hosting has turned into one of the most expensive races in AI infrastructure, and investors just…
AMD’s MI355X undercuts Nvidia on one model at the cost of users
SemiAnalysis priced the two accelerators against Moonshot's Kimi K3 and found AMD ahead per token at every interactivity…
Stanford and Berkeley shrink the first pass over long prompts
TokenGate learns which tokens matter and drops the rest before attention runs in full.
Google frees the GPU engine sitting under its on-device AI runtime
ML Drift hides Metal, OpenCL, OpenGL ES and WebGPU behind one layer and already ships inside Chrome and…
Cerebras says splitting inference in two yields five times the throughput
By running prompt processing and token generation on separate hardware pools, Cerebras says it served five times more…
Volantis lands $88M to break AI’s memory wall with light
Volantis closed an $88M Series A to commercialize a photonic inference system it says sidesteps the memory wall…
Modal nears a $750M round as inference demand climbs
Accel is said to be leading a raise that would more than triple the startup's valuation in four…
Fujitsu puts a 2nm Arm chip behind Japan-built AI servers
Fujitsu will start selling its MONAKA processor and Japan-built servers in November, pitching them at buyers who want…
Alibaba previews Qwen4 architecture in a 6B-active open model
An experimental open-weight release sketches the design direction for Alibaba's next flagship.
Ramp’s free model gateway steers AI queries to the cheapest option
Ramp launched Router, a model routing API that is free through 2026 and picks the cheapest LLM for…
Kog bets software unlocks 30x faster inference on stock GPUs
The French startup says its software can push 30x faster decoding out of the GPUs enterprises already own.
d-Matrix snaps up Wallaroo to run AI across mixed silicon
Chipmaker d-Matrix acquires Wallaroo.ai to orchestrate AI inference across GPUs and its Corsair accelerators.