L
LLM Large
Sign In
Learning library

LLM Engineering & Core Transformers

Deep dive into Transformer Attention, KV Caching, LoRA/QLoRA Parameter-Efficient Fine-Tuning, Quantization, and Speculative Decoding.

All TracksAdvanced AI Interview LabJAX for ML EngineeringJevAgentic AI & Multi-Agent SystemsRAG UniversityLLM Engineering & Core TransformersDistributed Big Data EngineeringDistributed & AI System Design

LLM Engineering & Core Transformers

Deep dive into Transformer Attention, KV Caching, LoRA/QLoRA Parameter-Efficient Fine-Tuning, Quantization, and Speculative Decoding.

Transformer Internals & Inference Acceleration

The mathematical engine: Self-Attention, Key-Value cache optimization, and quantization schemes.

1 Lesson
Advanced35 min

Scaled Dot-Product & Multi-Head Self-Attention

Self-attention allows every token in an input sequence to dynamically route information from every other token in the sequence based on relevance, breaking the sequential bottlenecks of RNNs.