GPU
8 pieces on GPU, including 2 step-by-step guides.
Guides
Run SGLang Offline Batch Inference Directly from Python
Use SGLang's offline_batch_inference.py pattern to batch-generate text without an HTTP server.
Run DeepSeek V4 Flash Locally with llama.cpp (2026)
Run DeepSeek V4 Flash locally with llama.cpp, a GGUF build, one 96GB GPU, and explicit checks for the current experimental support path.
News & Analysis
NVIDIA PAIR Routes Local AI Requests Across Nodes, Cuts Demo Time 2x
NVIDIA's PAIR beta distributes inference across local GPU nodes via Ollama and LM Studio, showing ~2x faster completion in a three-node demo.
Nvidia's Vera Rubin Delivers 3x Storage Gains Beyond the GPU
Nvidia's Vera Rubin stack delivers up to 3x storage operation gains via the Vera CPU — shifting its moat from GPU silicon to data orchestration.
Amazon Adds 2M More Nvidia GPUs After 5-Month Demand Surge
AWS triples its Nvidia GPU commitment to 3M+ chips spanning Blackwell Ultra, Rubin, and Rubin Ultra, with delivery across 2027–2028.
Kog Bets Deep GPU Engineering Can Deliver 10x LLM Speed
French startup Kog hit 3,000 TPS with a 2B-parameter model. Now it needs to prove the same approach works on production LLMs by September.
Kog Targets 10x LLM Speed by Exploiting GPU Memory Bandwidth
French startup Kog hit 3,000 TPS on its 2B-parameter Laneformer model and aims to prove 10x speed on a major LLM by September.
Kog Bets Software Can Unlock Stranded GPU Bandwidth for Inference
French startup Kog hit 3,000 per-request TPS on AMD MI300X and Nvidia H200 GPUs with a 2B-param model. A full LLM target follows in September.