French startup Kog is attempting to maximize inference throughput on existing datacenter hardware through deep-level software optimization. The firm aims to bypass traditional bottlenecks by reverse-engineering GPU memory access patterns.
According to a report by TechCrunch, French startup Kog is challenging the prevailing industry assumption that conventional datacenter GPUs are inherently inefficient for agentic AI workflows. By focusing on deep-level software optimization, the firm aims to extract significantly higher inference throughput from existing hardware like the NVIDIA H200 and AMD MI300X.
Gaël Delalleau, CEO of Kog, asserts that the perceived limitations of GPUs in decoding tasks stem from underutilized memory bandwidth rather than architectural flaws. The company recently demonstrated a proof-of-concept achieving 3,000 tokens per second using the 2-billion parameter Laneformer model.
This performance metric highlights the potential for software-defined acceleration to mitigate current latency constraints in enterprise AI environments. The engineering methodology employed by Kog draws heavily from Delalleau’s background in solid-state physics and offensive cybersecurity.
By reverse-engineering GPU hardware down to the binary and assembly levels, the team seeks to exploit undocumented performance characteristics. This hands-on approach requires several weeks of dedicated research for each new chip architecture supported by the Kog Inference Engine.
The startup currently maintains a lean team of 11 researchers to manage these intensive optimization cycles. While the initial focus remains on smaller, specialized models, the company is actively transitioning its research toward larger language models.
Delalleau expects to demonstrate a 10x speed improvement on a major LLM by September, a milestone intended to validate the scalability of their approach. This target is essential for proving that the methodology is not limited to small-parameter models.
Market demand for such optimizations is driven by the high cost and latency associated with current inference modes in professional AI applications. Enterprises relying on agentic workflows often face significant delays, creating a clear commercial incentive for software-based performance gains.
Kog has already secured 200 business leads, signaling strong interest from sectors where inference speed directly correlates with operational revenue. The company realizes that prospective customers are not currently prepared to fine-tune small models, necessitating a shift toward larger, more capable architectures.
The competitive landscape includes other European players like ZML, which focuses on hardware-agnostic software to bypass CUDA dependencies. Kog distinguishes itself through a more granular focus on GPU-specific acceleration, aligning its research philosophy with the work of academic groups such as Stanford University’s Hazy Research.
This technical depth is supported by regional backing from Bpifrance and the French Tech 2030 program, providing a foundation for long-term development. The primary technical challenge remains the transition from purpose-built small models to general-purpose LLMs.
If Kog successfully demonstrates its 10x acceleration target, it could provide a viable alternative to the reliance on purpose-built inference hardware. The company plans to leverage these performance gains to secure Series A funding, positioning itself as a critical layer in the infrastructure stack for high-performance AI.
Future development will likely involve integrating these optimization techniques into broader agent-based pipelines. This evolution would allow the software to support a wider array of models and hardware configurations without requiring manual engineering for every iteration.
The success of this transition will determine whether Kog can move beyond niche performance gains to influence the broader trajectory of AI inference efficiency. By optimizing memory access patterns to saturate available bandwidth on existing datacenter GPUs, the firm aims to redefine the economics of large-scale model deployment.
The firm is currently refining its kernel-level operations to ensure that data movement between the GPU memory and the processing units is perfectly synchronized. This granular control over the data pipeline is intended to reduce the idle time typically observed during the decoding phase of transformer-based architectures.
