Local LLMs

9 pieces on Local LLMs, including 7 step-by-step guides.

Guides

guides
architect • 2026-05-25T09:00:00Z
Local LLMsOllamallama.cppRAGDockerGGUFLLM Engineering

The Complete Developer Guide to Running LLMs Locally: From Ollama to Production

Everything you need to run LLMs on your own hardware in 2026: VRAM sizing, model formats, an 8-tool comparison table, a full local RAG pipeline, and Docker production deployment with GPU passthrough and Nginx auth.

Read more →
guides
Shan • 2026-05-24
DeepSeekOllamaReasoning ModelsLocal LLMsRAG

How to Run DeepSeek R1 Locally with Ollama: Full Setup Guide

Install DeepSeek R1 locally using Ollama in under 5 minutes. Covers model variant selection from 1.5B to 671B, visible chain-of-thought reasoning, REST API usage, Python integration, and building a simple RAG application.

Read more →
guides
Shan • 2026-05-24
DeepSeekLocal LLMsllama.cppGPUGGUF

Run DeepSeek V4 Flash Locally with llama.cpp (2026)

Run DeepSeek V4 Flash locally with llama.cpp, a GGUF build, one 96GB GPU, and explicit checks for the current experimental support path.

Read more →
guides
Shan • 2026-05-24
Qwen3OllamaLocal LLMsPythonReasoning Models

How to Run Qwen3 Locally with Ollama: Setup, API, and a Gradio App

Set up Qwen3 locally in minutes using Ollama. Covers every model variant, thinking mode control with /think and /no_think tags, CLI, REST API, Python SDK, and a practical Gradio reasoning app.

Read more →
guides
Shan • 2026-05-24
DeepSeekvLLMGPU InfrastructureSelf-HostingLocal LLMs

Self-Host DeepSeek V4 with vLLM: 2026 Deployment Guide

Deploy DeepSeek V4 Flash or Pro with vLLM, size the GPU cluster, configure tensor parallelism, and compare self-hosting with current API prices.

Read more →
guides
Shan • 2026-04-29
Claude CodeOllamaLocal LLMsAgentsCode

Run Claude Code Locally with Ollama (2026 Setup)

Connect Claude Code to Ollama's Anthropic-compatible API, choose a supported coding model, set a 64K context, and verify inference stays local.

Read more →
Build a Tool-Calling Agent with a Local Small Model and Ollama (2026)
guides
Shan • 2026-04-25
OllamaAI AgentsTool CallingSmall Language ModelsLocal LLMsPython

Build a Tool-Calling Agent with a Local Small Model and Ollama (2026)

Give a local small language model tools with Ollama's Python library: function tools, multi-tool streaming with thinking, and validated JSON output.

Read more →

News & Analysis