GPUs
How fast does it serve? Throughput, latency, and picking the right GPU
Part 2 of 2 on inference engineering for AI engineers.
Welcome, AI Engineers!
👋🏽 I'm Anup. I'm an AI and Software Engineer building production Agentic AI and Generative AI systems. I work on RAG pipelines, multi-agent architectures, and multi-cloud deployments.
My newsletter, The AI Engineering Brief, covers practical AI engineering for people shipping real systems. X/Twitter/Bluesky: @anup.
Join AI engineers getting weekly insights on agents, RAG & production LLM systems
The AI Engineering Brief Newsletter
LLM Scaling
Fitting LLMs on Self-Hosted GPUs
How much VRAM does your LLM need, and which GPU should you actually rent? A free calculator covering DeepSeek, Llama, Mixtral on H100, B200, A100.
Claude Code
The Harness Is the Product
Where does product quality live in an LLM-based system? A leaked source and a detailed postmortem, both from Anthropic in the last four weeks, make the answer unusually concrete.
Agentic Architecture
Harness Engineering: The Outer System That Makes Agents Reliable
Building a good harness is what separates a good agentic implementation from a great one.
Claude Code
Write Skills Like Workstations, Not Prompts
Claude Code skills work best when you treat them as workstations, not prompts: folders with scripts, gotchas, templates, and progressive disclosure that manage the agent's attention budget at runtime.
eval driven development
Ship Prompts Like Software: Regression Testing for LLMs
Because "it seemed fine when I tested it" is not a deployment strategy.
Part 4 of 4: Evaluation-Driven Development for LLM Systems
evals
Four Ways to Grade an LLM (Without Going Broke)
Your evaluation technique should match the question you're asking, not your ambition.
Recommended Resources and Tools
Recommended Podcasts
AI for Humans
Educational, light-hearted discussions breaking down AI concepts
Google DeepMind: The Podcast
Research-focused, neuroscience-inspired models, safe and ethical AI
Latent Space
Technical updates, real-world engineering, foundational models
NVIDIA AI Podcast
In-depth interviews on AI transforming industries
Shipped
VRAM Calculator: 2026 LLM Tool
LLM VRAM Calculator
A free VRAM calculator for self-hosted LLMs that picks the smallest fitting GPU instance (H100, B200, A100 ) across FP16, FP8, FP4.
Eliza Redux: A Voice AI crisis support Agent
Voice AI crisis support chatbot that responds in under a second. Built and deployed in 90 minutes for a hackathon win. Handles natural conversation, breathing exercises, safety checks, and can locate nearby emergency services.
Side Projects
ViT: 2024 - Paper Replication
Vision Transformer (ViT) Paper PyTorch Implementation
PyTorch replication of the Vision Transformer paper. Applied transformer architecture to image classification to deepen my understanding of attention over patches.
EfficientNetB2: 2024 - Paper Replication
EfficientNetB2 Computer Vision Model
An EfficientNetB2 feature extractor computer vision model to classify images of food as pizza, steak or sushi. The model performs at 95%+ accuracy, and can classify an image at 0.03 seconds inference time per image.