karnstack
All courses

How LLM Inference Works

9 lessons · ~3.75 hours

0/9 done3h 44m left

  • What happens in one model callFree
  • Tokens: what you pay forFree
  • Inside the forward passFree
  • Prefill and decode (Locked)
  • The KV cache (Locked)
  • Module quiz (Locked)
  • Batching many users (Locked)
  • Paged memory and prompt caching (Locked)
  • Making decode faster (Locked)
  • Module quiz (Locked)
  • Designing for cost and latency (Locked)
  • Module quiz (Locked)
Unlock every courseSee plans
  • Support
9293622
  1. Courses
  3. How LLM Inference Works
  5. Designing for cost and latency
Sign in