Generative AI interview questions and answers

What is generative AI?

Generative AI is a class of models that create new content, such as text, code, images or audio, by learning patterns from large training data. A large language model is the text version: given some input, it produces likely next text.

What is a large language model (LLM)?

An LLM is a neural network, usually a transformer, trained on a very large amount of text to predict the next token. That one skill, done at scale, lets it answer questions, summarise, translate and write code.

What is a token?

A token is the unit of text a model reads and writes: often a word piece rather than a whole word. In English one token is roughly three to four characters on average. Limits and API costs are counted in tokens, not words.

What is a context window?

The context window is the maximum number of tokens the model can consider at once, counting the prompt, any retrieved documents, the conversation so far and the answer. Anything outside it is invisible to the model.

What does temperature control?

Temperature scales how random token selection is. Low values (close to 0) make the output focused and repeatable, which suits extraction and code. Higher values make it more varied, which suits brainstorming but increases mistakes.

What is top-p (nucleus) sampling?

Top-p keeps only the smallest set of next tokens whose probabilities add up to p, for example 0.9, and samples from that set. It cuts off unlikely tokens while still allowing variety. Usually you tune either temperature or top-p, not both.

What is hallucination and why does it happen?

Hallucination is fluent output that is false or unsupported. The model is trained to produce plausible text, not to check facts, so when it lacks the information it can still generate a confident answer. Grounding with retrieved sources and asking it to say "I don't know" reduce it.

What is the difference between pretraining, instruction tuning and fine-tuning?

Pretraining teaches general language by predicting the next token on huge unlabeled text. Instruction tuning trains on prompt-and-response pairs so the model follows requests. Fine-tuning further adapts a model to one task, domain or style with your own examples.

What is a transformer?

A transformer is a neural network architecture built on attention instead of recurrence. It processes all tokens of a sequence in parallel and lets each token weigh every other token, which made training very large language models practical.

How does self-attention work?

Each token is turned into a query, a key and a value vector. A token's query is compared with every key to get attention weights, and its output is the weighted sum of the values. This lets a word like "it" draw meaning from the noun it refers to.

Why do transformers need positional information?

Attention by itself treats the input as an unordered set, so "dog bites man" and "man bites dog" would look the same. Positional encodings or embeddings add order information to each token so the model knows where it sits in the sequence.

What is the difference between encoder, decoder and encoder-decoder models?

Encoder-only models read the whole input in both directions and suit classification and embeddings. Decoder-only models generate text left to right and power most chat LLMs. Encoder-decoder models read an input and then generate an output, which suits translation and summarisation.

What is an embedding?

An embedding is a list of numbers (a vector) that represents the meaning of a piece of text. Texts with similar meaning get vectors that are close together, which is what semantic search and RAG retrieval rely on.

How do you measure similarity between two embeddings?

Cosine similarity is the usual choice: it compares the angle between two vectors and ignores their length, giving a value from -1 to 1. If the vectors are already normalised, the dot product gives the same ranking and is cheaper.

import numpy as np

def cosine(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

How does tokenization like BPE work?

Byte-pair encoding starts from single characters or bytes and repeatedly merges the most frequent adjacent pair into a new token. Common words end up as one token and rare words split into pieces, so the model never meets a truly unknown word.

Why is LLM text generation called autoregressive?

The model generates one token at a time, and each new token is appended to the input before predicting the next one. Output length therefore drives latency, and an early wrong token can steer the rest of the answer.

What is the difference between zero-shot and few-shot prompting?

Zero-shot gives only the instruction. Few-shot adds a handful of worked examples of input and expected output, which shows the format and edge cases. Few-shot usually improves consistency at the cost of longer prompts.

What is chain-of-thought prompting?

Chain-of-thought asks the model to work through intermediate steps before giving the final answer. It tends to help on multi-step reasoning such as maths or planning. In production you often keep the reasoning internal and show only the final answer.

What is a system prompt?

A system prompt sets the model's role, rules and output format for the whole conversation, separately from user messages. Put stable instructions there, such as tone, allowed topics and the JSON schema, and keep the user turn for the actual request.

How do you get reliable structured output such as JSON?

Give an exact schema and an example, use the API's structured-output or JSON mode when it exists, set a low temperature, and always validate the result in code. On a validation error, retry with the error message included.

What is prompt injection and how do you defend against it?

Prompt injection is text in user input or retrieved content that tries to override your instructions, such as "ignore previous rules". Defences: treat retrieved text as data, separate it clearly from instructions, give tools least privilege, and require confirmation before risky actions.

How can you reduce hallucinations in an LLM application?

Ground answers in retrieved sources and ask for citations, tell the model to say it does not know when the context lacks the answer, lower the temperature, narrow the task, and check outputs with validation rules or an evaluation step.

What is RAG?

Retrieval-augmented generation fetches relevant documents for a question and puts them in the prompt, so the model answers from that context instead of from memory alone. It keeps answers current and lets you cite sources without retraining the model.

When would you choose RAG over fine-tuning?

Use RAG when the knowledge changes often, must be cited, or is too large to bake into weights, such as company documents. Use fine-tuning to change behaviour, format or style. Many systems combine both: fine-tune for format, RAG for facts.

Walk through a basic RAG pipeline.

Offline: split documents into chunks, create an embedding for each chunk and store them in a vector index. Online: embed the question, retrieve the top matching chunks, optionally rerank them, then send the question plus those chunks to the LLM with an instruction to answer only from them.

How do you choose a chunk size?

Chunks must be small enough to be specific and large enough to keep meaning. A few hundred tokens with some overlap is a common start; split on headings or paragraphs rather than fixed characters when you can, and tune it with retrieval tests.

def chunk(text, size=800, overlap=100):
    step = size - overlap
    return [text[i:i + size] for i in range(0, len(text), step)]

What is a vector database?

A vector database stores embeddings and finds the nearest ones to a query vector quickly. It usually also stores the original text and metadata, so you can filter results, for example by document type or date, before or during the search.

What is approximate nearest neighbour search, for example HNSW?

Exact search compares the query with every vector, which is slow at scale. Approximate nearest neighbour indexes such as HNSW, a layered graph of neighbours, find almost the closest vectors much faster, trading a little recall for large speed gains.

What is hybrid search?

Hybrid search combines keyword search such as BM25 with vector search and merges the two rankings. Keywords catch exact terms like product codes and names that embeddings can blur; vectors catch paraphrases. Together they usually retrieve better than either alone.

What does a reranker do?

A reranker, often a cross-encoder, scores each retrieved chunk against the question more carefully than the first search did. You retrieve a wider set, say 30 chunks, rerank them, and keep the best few for the prompt, which improves precision.

How do you evaluate a RAG system?

Evaluate retrieval and generation separately. For retrieval, check whether the right chunks appear in the top results (context recall and precision). For answers, check faithfulness to the retrieved context and relevance to the question, using a labelled test set and, carefully, an LLM judge.

What are LoRA and parameter-efficient fine-tuning?

Parameter-efficient fine-tuning trains only a small number of extra parameters and freezes the base model. LoRA adds small low-rank matrices to chosen layers and learns only those, which cuts memory and lets you keep several task adapters for one base model.

What is QLoRA?

QLoRA loads the frozen base model in 4-bit quantized form and trains LoRA adapters on top. It makes fine-tuning large models possible on a single GPU with limited memory, with quality close to standard LoRA in many tasks.

What is RLHF?

Reinforcement learning from human feedback first collects human rankings of model answers, trains a reward model on them, then optimises the LLM to produce answers the reward model scores highly. It is one way models are made more helpful and safer.

How is DPO different from RLHF?

Direct Preference Optimization trains the model directly on pairs of preferred and rejected answers, without a separate reward model or a reinforcement learning loop. It is simpler and more stable to run, which is why many teams use it for alignment.

What is quantization and what is the trade-off?

Quantization stores model weights with fewer bits, for example 8-bit or 4-bit instead of 16-bit. The model needs less memory and often runs faster or on cheaper hardware, at the cost of some accuracy, which you must measure on your task.

What is an AI agent?

An AI agent is an LLM that works toward a goal in a loop: it decides the next step, calls tools such as search, code execution or APIs, reads the results and continues until the task is done. A plain chatbot answers once; an agent acts over several steps.

What is the ReAct pattern?

ReAct, short for reasoning and acting, alternates thought, action and observation: the model reasons about what to do, calls a tool, reads the tool result, and repeats. Grounding each step in real observations reduces made-up answers.

How does tool calling (function calling) work?

You describe each tool with a name, a description and a JSON schema for its inputs. The model replies with a structured request to call a tool; your code runs it and sends the result back; the model then continues. The model never runs code itself.

tool = {
    "name": "get_weather",
    "description": "Current weather for a city",
    "input_schema": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"],
    },
}

What is the Model Context Protocol (MCP)?

MCP is an open standard, introduced by Anthropic in November 2024, for connecting AI applications to tools and data sources. A tool provider builds one MCP server, and any MCP-compatible client can use it, instead of writing a custom integration for every app.

When would you use multiple agents instead of one?

Split work across agents when tasks are clearly separate, can run in parallel, or need different tools and instructions, for example a researcher and a reviewer. More agents add cost, latency and coordination failures, so start with one and split only when it helps.

How do you keep an agent safe and under control?

Give tools the least privilege they need, cap the number of steps and the budget, validate tool inputs, log every action, and require human approval for irreversible steps such as payments, deletions or sending messages.

How do you reduce LLM latency?

Stream tokens so users see output immediately, keep prompts and outputs short, use a smaller model for simple steps, cache repeated work, and run independent calls in parallel. Output length usually matters more than input length for total time.

How do you reduce LLM cost?

Route easy requests to smaller models, trim prompts, cache answers to repeated questions, use prompt caching for a long shared prefix, batch offline jobs, and cap output length. Measure cost per successful task, not just cost per call.

What is prompt caching?

Prompt caching lets a provider reuse the processed state of a prompt prefix that repeats across requests, such as a long system prompt or document. Repeated calls become cheaper and faster, so stable content should go first and per-request content last.

How do you evaluate LLM output quality?

Build a test set of real inputs with expected answers or grading rules, run it on every prompt or model change, and track the scores. Combine exact checks, human review of samples, and an LLM judge whose grading you have checked against humans.

What would you monitor in a production LLM application?

Latency, token usage and cost, error and retry rates, tool-call failures, user feedback, and quality signals such as validation failures or answers flagged as unsupported. Log prompts and outputs, with personal data removed, so you can debug and re-run them.