Amazon SageMaker
8 pieces on Amazon SageMaker.
News & Analysis
SageMaker HyperPod Inference Gateway Cuts TTFT P95 by Up to 98%
AWS's new EKS addon uses real-time KV cache and queue metrics to slash first-token latency by up to 98% over Kubernetes round-robin, with zero app changes.
SageMaker Prefix-Aware Routing Cuts LLM P50 TTFT by Up to 77%
Amazon SageMaker's new PREFIX_AWARE routing lifts KV cache hit rates from ~25% to ~82% on long-context workloads, cutting P50 TTFT by up to 77%.
150M-Parameter BDH-CQ Scores 29.2% on ARC-AGI-1 at $0.0007 per Task
Pathway's 150M-parameter BDH-CQ model scores 29.2% pass@2 on ARC-AGI-1 at $0.0007 per task, trained on SageMaker HyperPod with H200 GPUs.
2.4T-Parameter Qwen3.8 Runs on One Node With vLLM and NVFP4
AWS documents a single-node serving path for Qwen3.8-2.4T-A95B on a p6-b300 instance using NVFP4 quantization, cutting TTFT by nearly 60%.
SageMaker Feature Store's UpdateRecord Ends Read-Modify-Write Cycles
AWS ships UpdateRecord for SageMaker Feature Store: atomic partial writes on up to 100 features, no GetRecord required, available in all regions today.
Salesforce Gets Multi-AZ HA on SageMaker With SchedulingConfig
Salesforce's 50+ production models needed 2-AZ compliance. SageMaker's new SchedulingConfig parameter closes the gap without surrendering GPU cost efficiency.
Deepgram Brings Billing-Accurate Metrics to SageMaker AI Endpoints
Deepgram's Enhanced Metrics push billing and engine-level signals into CloudWatch via EMF stdout — no agent, sidecar, or relaxed network isolation required.
SageMaker SDK v3 Replaces Dozen Estimator Classes With Two Primitives
AWS shipped SageMaker Python SDK v3 on 26 Aug 2026, collapsing framework-specific estimators into ModelTrainer and ModelBuilder with runtime code injection.