Kog Bets Deep GPU Engineering Can Deliver 10x LLM Speed

French startup Kog hit Hacker News' front page in May after a tech preview claiming that "extremely fast single-request decoding is possible on the standard datacenter GPUs enterprises already own." The demo ran on AMD MI300X and Nvidia H200 hardware and recorded 3,000 per-request tokens per second — but the model behind that number was Laneformer 2B, a purpose-built, now open-sourced model with roughly 2 billion parameters. Scaling that result to the large language models that agentic pipelines actually depend on is the unresolved problem Kog is now racing to solve.

For engineers running agentic workloads, the cost and latency bottleneck is real. Veteran Claude Code users already accept multi-hour waits on heavy tasks, and Anthropic charges a price premium for Claude's Fast Mode. Kog's bet is that software-level GPU engineering can close that gap without new silicon — and the May preview generated 200 concrete business leads, according to CEO Gaël Delalleau.

How the Kog Inference Engine Works

Kog calls its product the Kog Inference Engine (KIE). Delalleau positions it closer in spirit to Stanford's Hazy Research lab than to hardware-agnostic middleware like ZML, which bypasses Nvidia's CUDA to run across competing chips. Where ZML abstracts away the hardware, Kog goes the opposite direction: for each new GPU, the team commits several weeks to months of what Delalleau calls "GPU engineering research," working down to assembly language and binary code to understand the chip's actual memory-bandwidth characteristics rather than its rated specifications.

Delalleau's framing is that newer GPUs carry increasingly large memory bandwidth headroom that software has not been written to exploit. His background — solid-state physics at École Polytechnique followed by offensive cybersecurity work, including four DEFCON CTF finals — informs the methodology directly. The physics training instills reasoning from first principles about hardware constraints; the security work built the muscle for deep reverse-engineering. The practical consequence is that the approach does not generalise cheaply across chip generations.

The Gap Between Demo and Target

Kog's headline claim is 30x faster LLM inference, but the 3,000 TPS demo figure was achieved with a 2-billion-parameter model. The company's actual customers — software engineering teams and design partners building prompt-driven game and app generators — need acceleration on larger production models. Delalleau acknowledges the prospective customer base is not prepared to fine-tune small models, which is why KIE development has pivoted toward accelerating larger models since the May launch.

The near-term milestone Delalleau has committed to publicly is a 10x speed result on a first major model, expected in September. That result is also the condition he has set for beginning a Series A raise — the funding timeline is explicitly gated on the technical proof. With a team of 11 people, the per-chip research burden imposes a hard ceiling on how many GPU variants KIE can support simultaneously. Longer-term, Delalleau describes plans to feed the methodology into agent-based pipelines that would let the company support more chips and models with less per-chip overhead — an approach that parallels the software extraction thesis gaining traction across the inference layer.

Positioning and Backing

Kog's seed round was co-led by Varsity VC, whose managing partner Kamel Zeroual is Delalleau's former co-founder from Stribe — a TechCrunch50 2009 alum with no technical overlap with Kog. Public backers include Bpifrance and the French Tech 2030 programme; Scaleway provides infrastructure support. As Europe works to develop indigenous model and chip capability, a software inference layer that extracts more from domestic or allied hardware carries a policy tailwind that purely US-centric inference optimisation startups do not.

The inference optimisation layer is fragmenting into distinct bets — new silicon, CUDA abstraction, and low-level GPU microarchitecture research — each carrying a different scaling cost. Kog's per-chip research commitment is a durable moat if the September LLM result lands, but a critical bottleneck if the team stays at 11 while the GPU landscape keeps diversifying.