AI Systems (74)

Essays and notes on AI systems, LLM architectures, and agentic workflows.

11 blogs • 60 notes • 3 videos

2026

(44)

Sep 08

NOTE Reorganizing KV cache memory can make models interactive in real-time

AI Systems #ai-systems #kv-cache #model-memory
Sep 08

VIDEO Quantization Fundamentals: How LLMs Run Efficiently with Reduced Memory and Fast Arithmetic

AI Systems #quantization #llm-inference #gpu-memory
Sep 07

NOTE Organizing a Model's KV Cache Unlocks New Interactive Abilities

AI Systems #ai-systems #kv-cache #model-memory
Aug 30

NOTE Design Multi-Context Processing Tools as Capabilities, Not Endpoints

AI Systems #ai-systems #context-management #mcp
Aug 30

NOTE Effective PII Masking Requires Layered Data Classification Beyond Regex

AI Systems #ai-systems #data-classification #layered-approach
Aug 30

NOTE Retrieving Correct Code for Agentic SDLCs Presents Three Major Problems

AI Systems #ai-systems #agentic-sdlc #ai-agent-gullibility
Aug 30

NOTE Schema-Driven Data Modeling Is Essential for Deterministic Agent Memory

AI Systems #ai-systems #agent-memory #ai-agents
Aug 28

NOTE Automated Verification Ensures End-to-End Correctness and Human Accountability

AI Systems #ai-systems #accountability #communication
Aug 26

NOTE Continuous batching achieves high GPU utilization for large language models

AI Systems #ai-systems #caching #continuous-batching
Aug 26

NOTE Tailor AI Code Generation Tradeoffs Per File Based on Module Risk

AI Systems #ai-systems #ai-agents #ai-code-generation
Aug 22

NOTE Robust Plumbing and Harness are Essential for Production-Ready AI Agents

AI Systems #ai-systems #ai-agents #backoff
Aug 20

NOTE GPU Drivers Adopt Database Wound-Wait Mutex to Prevent Deadlocks

AI Systems #ai-systems #concurrency #concurrency-control
Aug 15

NOTE Claude's Text Watermark Uses Secret Keys To Derive Randomness

AI Systems #ai-systems #claude #llm-generation
Aug 08

NOTE Using Plain Integers in MCP Tool Parameters Cuts Agent Errors

AI Systems #ai-systems #agent-errors #ai-agents
Aug 06

BLOG What AI First Engineering Orgs Look Like

AI Systems #ai-systems #ai-agents #code-review
Aug 06

NOTE Dispatch Tools Incrementally and Load Schemas Dynamically to Cut Agent Latency

AI Systems #ai-systems #agent-latency #ai-agents
Aug 05

NOTE Cloudflare increases LLM concurrency through precision reduction without quality loss

AI Systems #ai-systems #caching #cloudflare
Jul 31

BLOG Three Claude Skills I Think Every Org Should Have

AI Systems #ai-systems #databases #leadership
Jul 20

NOTE AI-Driven Development Shifts Constraints to Judgment and Saying No

AI Systems #leadership #ai-development #clock-synchronization
Jul 14

BLOG G-Eval, Explained

AI Systems #ai-systems #evaluation #llms
Jul 10

NOTE Monitor AI Agent Apologies To Improve Their Context And Performance

AI Systems #ai-systems #agent-context #ai-agent
Jul 08

NOTE Your Specific LLM Task Requires a Custom Performance Benchmark

AI Systems #ai-systems #cheaper-models #custom-benchmarks
Jun 30

NOTE How Erasure Coding Achieves S3 Durability with Reduced Storage Costs

AI Systems #distributed-systems #compute-tradeoff #data-durability
Jun 29

NOTE AI Makes Premature Optimization Effortless Leading to Over-Engineered Code

AI Systems #software-engineering #ai #caching
Jun 20

NOTE Idempotency is a must-have for reliable AI agent operations

AI Systems #ai-systems #ai-agents #clock-synchronization
Jun 10

NOTE System Design, Not Prompting, Optimizes LLM Token Budget Performance

AI Systems #ai-systems #leadership #llm-context-windows
Jun 03

NOTE Tool Calls Enable Internal Agent Logic Decomposition and Testability

AI Systems #ai-systems #agent-design #ai-agents
Jun 01

BLOG AI Workflows Need Topological Sort

AI Systems #ai-systems #ai-agents #embeddings
Jun 01

NOTE Agentic Systems Encounter Out of Memory Errors Due to Context Accumulation

AI Systems #ai-systems #agentic-systems #ai-agents
May 30

NOTE Production AI agents face classic software engineering reliability issues

AI Systems #ai-systems #ai-agents #concurrency
May 28

BLOG Embedding Models Make Or Break Your Ai App

AI Systems #ai-systems #embeddings #information-retrieval
May 28

NOTE Modern Embeddings Represent Contextual Meaning Better Than Static Word2Vec

AI Systems #ai-systems #bert #contextual-embeddings
May 19

NOTE System Design Fundamentals Remain Crucial for Robust Long-Running Agentic Applications

AI Systems #ai-systems #ai-agents #caching
May 15

BLOG What Matters in Production RAG

AI Systems #ai-systems #embeddings #llms
May 13

BLOG Structure of Every LLM Chat

AI Systems #ai-systems #llms
May 12

BLOG How LLMs Really Work

AI Systems #ai-systems #llms
Apr 13

NOTE Asking for Help is Good, But Quality Matters

AI Systems #career-growth #communication #debugging
Mar 30

NOTE If you are deduplicating documents, semantic similarity is probably the

AI Systems #ai
Mar 12

NOTE Yes, when the active node goes down, the passive (backup)

AI Systems #ai
Mar 08

NOTE Databases were not designed for the agentic space

AI Systems #ai
Mar 04

NOTE Wrote an essay on the BM25 algorithm that powers Elasticsearch,

AI Systems #ai
Feb 27

NOTE Turns out companies are reimagining the entire SDLC around agents

AI Systems #ai
Feb 21

NOTE TDD was ahead of its time

AI Systems #ai
Feb 19

NOTE Google just released TimesFM (a Time Series Foundation Model) -

AI Systems #ai

2025

(21)

Dec 07

BLOG Product Quantization

AI Systems #ai-systems #embeddings #information-retrieval
Dec 07

NOTE Wrote an essay on Product Quantization and how it helps

AI Systems #ai
Nov 27

NOTE Wrote an essay on what Q, K, and V matrices

AI Systems #ai
Nov 26

BLOG The Q, K, V Matrices

AI Systems #ai-systems #llms #system-design
Nov 22

BLOG How LLM Inference Works

AI Systems #ai-systems #llm-inference #llms
Nov 22

NOTE wrote an essay on how LLM inference works

AI Systems #ai
Aug 15

NOTE Spent the day going down the rabbit hole of LRU

AI Systems #ai
Aug 06

NOTE Running a distributed system sounds great, until one node starts

AI Systems #ai
Aug 05

NOTE Google built QUIC as they got tired of waiting for

AI Systems #ai
Jul 30

NOTE All databases, in fact, all TCP servers, are susceptible to

AI Systems #ai
Jul 02

NOTE Address matching is a super interesting problem and something I

AI Systems #ai
Jun 06

NOTE When people talk about UDP, we often hear the standard

AI Systems #ai
Jun 05

NOTE When building a highly available system, you generally focus on

AI Systems #ai
May 28

NOTE A few months ago, I was putting together an internal

AI Systems #ai
May 27

NOTE All frontend frameworks are converging

AI Systems #ai
May 21

NOTE ElasticSearch is a pretty popular search engine, but what happens

AI Systems #ai
May 20

NOTE One project I would highly recommend, if you haven't done

AI Systems #ai
May 19

NOTE I once saw a codebase at a pretty large company

AI Systems #ai
Apr 29

NOTE do one deep research a day

AI Systems #ai
Feb 28

NOTE DeepSeek open-sourced their high-performance distributed file system optimized for AI

AI Systems #ai
Feb 16

NOTE you don't need to watch my videos to become a better engineer.

AI Systems #ai-systems

2023

(3)

Jun 19

NOTE 3 papers to learn how ChatGPT works ⚡

AI Systems #chat-gpt
Mar 05

NOTE Spent this weekend reading about the internals of MP3 file

AI Systems #ai
Feb 24

VIDEO The Engineering Behind Large Language Models: Architecture, Training, and Decoding

AI Systems #large-language-models #chatgpt #rlhf

2022

(1)

Dec 18

VIDEO Improving Search Relevance with Intent Identification and NLP: Inside Zomato's Architecture

AI Systems #search-systems #natural-language-processing #elasticsearch