Daily Tech Answers

Question 1

What is a large language model?

Accepted Answer

A large language model is a neural network trained to predict the next token in a sequence of text, using the transformer architecture. Trained on very large text corpora, it learns statistical structure that generalises well enough to answer questions, write code and follow instructions. It has no database of facts and no lookup step: everything it produces is generated token by token from learned parameters, which is why it can be fluent and wrong at the same time.

Question 2

What is inference, and why does it cost what it does?

Accepted Answer

Inference is running a trained model to produce output. Its cost is driven by tokens and by two structurally different phases: prefill, where the whole prompt is processed in parallel and the work is compute-bound, and decode, where output tokens are generated one at a time and the work is bound by memory bandwidth. That asymmetry is why providers price input and output tokens differently, why output is usually several times dearer, and why long prompts are cheaper than long answers.

Question 3

What is retrieval-augmented generation?

Accepted Answer

Retrieval-augmented generation puts a search step in front of the model: the system retrieves passages relevant to the user's question from a corpus it controls, places them in the prompt, and asks the model to answer from that material. It was introduced in 2020 as a way to combine parametric knowledge with an external, updatable store. In practice most RAG failures are retrieval failures — the model answers faithfully from passages that were the wrong passages.

Question 4

What is an AI agent?

Accepted Answer

An AI agent is a system in which a model decides which actions to take, executes them through tools, observes the results and repeats until a goal is met or a stopping condition fires. The defining property is delegated control flow: the sequence of steps is chosen at runtime by the model rather than fixed by a developer. That is also the source of its difficulty — errors compound across steps, so reliability falls as task length grows.

Question 5

Why do AI systems hallucinate?

Accepted Answer

Because a language model is trained to produce likely text, not verified text, and nothing in that objective distinguishes a supported claim from an unsupported one. When the training data is thin, contradictory or absent for a question, the most likely continuation is still a fluent, specific-sounding answer. Training procedures that reward confident, helpful responses can make this worse. Grounding, abstention and verification reduce hallucination; they do not remove the underlying incentive.

Question 6

Why do enterprise AI pilots fail?

Accepted Answer

Rarely because the model was inadequate. Pilots fail because success was never defined against a baseline, because the workflow around the model was left unchanged, because the data the system needed was not accessible under existing permissions, or because nobody owned the ongoing evaluation once the demo ended. A pilot that produces impressive outputs but no measured delta against current practice cannot be approved or rejected on evidence, so it stalls.

Question 7

What are the common security risks in AI applications?

Accepted Answer

The distinctive risks come from treating model input as trusted. Indirect prompt injection — instructions hidden in a web page, document or email the model reads — turns a helpful agent into a confused deputy acting with its own permissions. Around that sit data leakage through prompts and logs, over-broad tool and retrieval permissions, insecure handling of model output that reaches shells or databases, and supply-chain exposure through models, plugins and connectors.

Question 8

How do AI answer systems choose which sources to use?

Accepted Answer

Most consumer answer systems retrieve before they generate: a query is rewritten into several searches, a conventional ranking system returns candidate pages, some are fetched, and the model composes an answer from the passages that survive. Selection is therefore mostly a retrieval-and-ranking outcome, filtered by what the crawler is allowed to fetch and by how easily a passage can be extracted and attributed. Sampling means two identical questions can draw different sources.

Question 9

How stable are AI recommendations?

Accepted Answer

Less stable than most people assume. Because answers are sampled and, in many products, grounded in a live retrieval step, the same question asked repeatedly can return different named products, different orderings and different sources within a single session. Stability varies by question type: well-established categories with obvious incumbents are relatively consistent, while crowded or fast-moving categories vary substantially run to run. Any single screenshot is an anecdote.

Question 10

What is MCP, the Model Context Protocol?

Accepted Answer

The Model Context Protocol is an open specification for connecting AI applications to external tools, data and prompts. A host application runs clients that talk to servers, each exposing tools, resources and prompts through a defined JSON-RPC interface over stdio or HTTP. Its purpose is to replace bespoke per-integration glue with one interface, so a connector written once can be used by any compliant client. It standardises transport and discovery, not model behaviour.

Question 11

What is tool calling?

Accepted Answer

Tool calling is the mechanism by which a model asks the surrounding application to run a function. The developer supplies tool names, descriptions and JSON schemas; the model returns a structured call with arguments; the application executes it and returns the result for the next turn. The model never executes anything itself. Every real capability, and every real risk, comes from what the application is willing to run on its behalf.

Question 12

What is a vector database?

Accepted Answer

A vector database stores embeddings — numeric representations of text, images or audio — and answers nearest-neighbour queries over them, so you can retrieve items by meaning rather than by exact match. It does this with approximate indexes such as HNSW or IVF, trading a small amount of recall for large speed gains. It is a component of retrieval, not a synonym for it: most production systems combine vector search with keyword search and a reranking step.

A reference library, not a blog

Artificial intelligence