Deep Learning
6 pieces on Deep Learning.
News & Analysis
PC-ALM Trains 1,000-Layer Networks Within 2 Points of Backprop
Sakana AI's PC-ALM keeps every update layer-local yet matches backprop on 1,000-layer residual MLPs — and runs on CPU under an MIT licence.
RLT Runs 96 Blocks Per Token With Recurrent State, No Resets
Princeton's Recurrent Looped Transformer carries full decoder state across every token with 96 logical blocks and no prompt-response boundary reset.
Why σ(x) = 1/(1+e⁻ˣ): The Derivation Behind Sigmoid
Most practitioners memorise sigmoid but skip the derivation. Here's why e appears, how the (0,1) bound is constructed, and what it means for backpropagation.
Why Transformers Look the Way They Do: Deriving Q, K, and V
A step-by-step reconstruction of the Transformer shows that queries, keys, values, and W₀ are near-inevitable answers to concrete engineering problems.
AI vs. Machine Learning vs. Deep Learning: Choosing an Approach
Understand how AI, machine learning and deep learning relate, then choose an approach using your data, evaluation needs and operating constraints.
Solving AI Hallucinations: Fact-checking Pipelines Explained
How modern architectures use strict fact-checking pipelines and grounding mechanics to mathematically force LLMs to tell the truth.