The advanced interview lab is ready. Loading the rest of the question bank…

AI engineering interview practice

AI engineering questions & answers

Practice advanced RAG, large-scale LLM serving, GPT-style model design, reasoning and production trade-offs. Answer first, then compare your assumptions with the worked explanation.

Open the worked lessons

Total Questions

32 Q&As

Core Sections

8 sections

Target Seniority

Staff / Principal

Practice format

Explain & compare

Showing 32 questions across 8 sections

Advanced RAG: retrieval, reranking and grounded answers

4 questions with worked answers

4 Qs

#10000

Staff Engineering Architecture AnswerMarkdown / High-Precision Format

Inspect the final context rather than the candidate list alone. More candidates may increase duplication, introduce conflicting versions or push supporting evidence out of the token budget. Compare reranker order, context placement and claim-level support on the same questions. Run an oracle-context experiment and keep the model and prompt fixed while changing retrieval. Report answer quality and latency together; a higher retrieval score does not establish end-to-end improvement.

#10001

#10002

#10003

Enterprise RAG: permissions, freshness and index migrations

4 questions with worked answers

4 Qs

#10010

#10011

#10012

#10013

LLM serving: KV cache, GQA and memory budgets

4 questions with worked answers

4 Qs

#10020

#10021

#10022

#10023

LLM serving: prefill, decode and speculative sampling

4 questions with worked answers

4 Qs

#10030

#10031

#10032

#10033

Design a GPT-style decoder and training pipeline

4 questions with worked answers

4 Qs

#10040

#10041

#10042

#10043

Post-training, preference learning and release evaluation

4 questions with worked answers

4 Qs

#10050

#10051

#10052

#10053

Frontier systems: MoE, reasoning budgets and tool execution

4 questions with worked answers

4 Qs

#10060

#10061

#10062

#10063

Design a large-scale GPT-style chat service

4 questions with worked answers

4 Qs

#10070

#10071

#10072

#10073

AI engineering questions & answers

Advanced RAG: retrieval, reranking and grounded answers

Retrieval metrics improved but answers got worse. What would you inspect?

When would you choose hybrid retrieval over a vector-only index?

How do you choose between RAG and a long-context prompt?

What should a RAG assistant do when its evidence is insufficient?

Enterprise RAG: permissions, freshness and index migrations

Why is filtering unauthorized results after generation unsafe?

How would you delete a document from a live RAG system?

How do you migrate from one embedding model to another safely?

What belongs in the cache key for an enterprise RAG answer?

LLM serving: KV cache, GQA and memory budgets

Why can a 4-bit model still run out of memory on long prompts?

How much KV memory does the worked 32k sequence need?

How do GQA, PagedAttention and FlashAttention differ?

What would you measure before increasing the maximum context length?

LLM serving: prefill, decode and speculative sampling

When is disaggregating prefill and decode a bad idea?

Does speculative decoding always preserve the target model distribution?

A prefix-cache hit rate is high, but answers still stream slowly. Why?

Why is GPU utilization alone a poor autoscaling signal for chat?

Design a GPT-style decoder and training pipeline

How do you explain GPT-style training without inventing GPT-4 or newer internals?

What goes wrong if a causal mask or training-label shift is incorrect?

How would you estimate optimizer memory for the example 7B model?

Which states are required for a faithful training restart?

Post-training, preference learning and release evaluation

When would you choose SFT rather than preference optimization?

How can an LLM judge mislead a release decision?

A release improves from 88% to 90% on 200 tasks. Would you ship it?

How do you prevent preference learning from rewarding confident misinformation?

Frontier systems: MoE, reasoning budgets and tool execution

Does a model with fewer activated parameters necessarily need fewer GPUs?

How would you decide which requests receive a larger reasoning budget?

What is the difference between a reasoning trace and a verification result?

How do you make a tool retry safe after a network timeout?

Design a large-scale GPT-style chat service

How many inference replicas does the worked example require?

What should happen when a user reconnects during a streamed response?

Why can request-per-second capacity hide an overloaded LLM service?

What is a realistic multi-region failover promise for an LLM chat service?