Business Analyst: understanding LLMs and agents without coding

A position that becomes central

Business analysts and functional consultants have always occupied the translation zone between a business need and a technical build. With AI systems that zone widens: the business states an expectation in plain language, and engineering answers with a system whose behaviour is not guaranteed in advance. Nobody naturally holds that boundary. That is exactly where the role gains value — provided you acquire enough technical grounding not to be a spectator.

This guide describes that grounding: what to understand about LLMs, RAG and agents in order to specify properly, how to frame a use case, how to write testable requirements against a probabilistic system, and how to run an acceptance phase that actually means something.

The useful technical grounding — without coding

This is not about becoming an engineer. It is about understanding enough of the mechanics to ask the right questions and spot an unkeepable promise. Four notions cover the vast majority of enterprise projects.

• An LLM produces plausible text from a context: it does not query your database and does not reason over business rules it has never seen.
• RAG means retrieving relevant documents and handing them to the model: answer quality depends first on retrieval, not on the model.
• An agent chains actions — calling a tool, reading a system, writing somewhere — with some autonomous decision at each step.
• Evaluation means measuring answer quality against a known set of cases: without it there is no way to know whether the system works.

A simple self-check: when an answer is wrong, can you tell apart a retrieval problem, a source-data problem, a phrasing problem and a model problem? Those four causes call for four different fixes.

Understanding RAG from the business side

Most enterprise AI projects are RAG projects: answering from an internal document corpus. Seen from the business side, the difficulty is almost never the model. It is the corpus.

• Document scope: which documents are in, which are excluded, who decides.
• Access rights: two different users must not see the same answers when their permissions differ.
• Freshness: how often the corpus is refreshed, and what happens to outdated versions.
• Duplicates and competing versions: the leading cause of contradictory answers.
• Non-textual documents: tables, scans, slide decks, which the system often cannot read.

These are business questions, not technical ones. An analyst who frames the corpus properly does more for final quality than any model change.

Understanding agents without writing code

An agent is a system that decides on a sequence of actions to reach a goal. Where a conventional process follows a defined path, an agent chooses its path. That difference has direct consequences for specification, accountability and control.

• Which actions is the agent allowed to perform, and which are explicitly forbidden?
• Which actions are reversible and which are not? The latter require human approval.
• What happens when the agent fails mid-sequence: resume, roll back, alert?
• What trace remains of what was done, for audit or dispute?
• What volume and cost limits apply per run?

These are exactly the questions an analyst asks about any workflow. What is new is that the path is no longer known in advance: you specify the boundaries rather than the sequence.

Framing a use case that holds up

Many projects fail before the first line of code, because the use case was never stated in an evaluable way. Solid framing comes down to six points, all expressible by a business profile.

• The precise task: who does what today, how many times a week, in how long.
• The expected output: what a good answer looks like, with three real examples.
• Known edge cases: situations where a human hesitates too.
• The cost of an error: annoying, expensive or unacceptable — this drives the whole control setup.
• Available data: does it exist, is it accessible, is it current, who authorises it.
• The stop criterion: what result makes you continue or drop the project.

Writing requirements for a non-deterministic system

A classic requirement describes guaranteed behaviour: in this situation, the system does exactly this. On an AI system that phrasing is untenable for the output itself. It remains perfectly valid for everything around the output — and that is where specification should concentrate.

• What stays deterministic: access rights, traceability, output format, allowed actions, logging.
• What becomes statistical: answer relevance, measured against a reference set of cases.
• What must be absolutely guaranteed: no irreversible action without approval, respect for permissions, source citation.
• What must be designed explicitly: behaviour under doubt, missing information or out-of-scope questions.

A useful formulation: "the system must abstain rather than answer when no relevant source is found" is a testable requirement. "The system must answer correctly" is not.

Building the reference case set

This is the most concrete and most underrated contribution of a business analyst on an AI project. The case set is what makes it possible to say whether the system is improving or regressing. Nobody is better placed to build it: it requires fine business knowledge, not technical skills.

1. 1

   Collect real cases

   Pull historical requests — tickets, emails, files — rather than inventing examples. Fifty to a hundred cases is enough to start.

2. 2

   Write the expected answer

   For each case, the answer a good professional would give, with the source that justifies it.

3. 3

   Cover the hard cases

   Deliberately include ambiguous questions, cases with no answer, out-of-scope questions and known traps.

4. 4

   Define what counts as failure

   Wrong answer, answer without a source, out-of-scope question treated as valid: state the severity of each type.

5. 5

   Get business sign-off

   The case set commits the organisation. Domain owners should review and accept it, not just the project team.

6. 6

   Maintain it

   Add the errors encountered in production. A frozen case set loses its value within months.

Running an acceptance phase that means something

Traditional acceptance validates expected behaviour case by case with a binary verdict. On an AI system that format gives false assurance: ten passing tests say nothing about the eleventh. Acceptance has to change form without losing its decision function.

• Measure a rate across the whole case set rather than validating isolated tests.
• Set thresholds before the run, with the business, and separate tolerable errors from blocking ones.
• Test out-of-scope cases as much as nominal ones: a correct refusal is an expected result.
• Check access rights with several different user profiles.
• Replay the same case set on every change: it is the only protection against silent regressions.

A regression on an AI system does not show up as a technical error. It shows up as slightly worse answers that nobody notices for weeks. Systematic replay of the case set is the only defence.

Data, permissions and compliance: the business share

On an AI project, data questions are not administrative preliminaries: they determine feasibility. A business analyst handles them naturally, and skipping them is a frequent cause of late-stage cancellation.

• Which personal data may enter the system, and on what legal basis.
• Which data is confidential enough that it must never leave certain boundaries.
• How existing access rights translate into the system's answers.
• How long exchanges are retained, and who can read them.
• What must be documented for an internal audit or a sector regulation.

Adoption: the real success factor

A correct system nobody uses is a failure. Adoption of an AI tool runs into two opposite and equally damaging reactions: rejection — "it gets things wrong, I'd rather do it myself" — and blind acceptance — "it said so, therefore it's true". The support work is about installing clear-eyed use between the two.

• Explain the real scope in training, including what the tool cannot do.
• Show typical errors during the demo, not only successes.
• Give a clear instruction about what must always be verified.
• Provide a simple reporting channel and visibly act on it.
• Track real usage: frequency, drop-off, edits made to outputs.

A concrete case: a badly specified request

Initial request received by the project team: "automate customer complaint handling with AI". As stated it is unusable — it says nothing about which part of the handling, on what basis, or what counts as a good result.

Framing work turns it into three distinct items, only one of which is genuinely addressable in the short term:

• Classify the complaint by reason and urgency: repetitive task, historical data available, error cheap and correctable. Addressable.
• Draft a reply from contractual terms: feasible with source citation and systematic human approval, since a wrong reply commits the company.
• Decide on a goodwill gesture: a committing decision with variable business rules and human judgement. Out of scope — at best the system prepares the decision inputs.

Framing then specifies: 200 historical complaints as the reference set, a 90% correct-classification threshold on main reasons, every drafted reply approved before sending, full traceability of who approved what. That level of precision requires no development skill — only rigorous analysis applied to a new kind of object.

The stance that makes the difference

The risk for an analyst is to stay a messenger: relaying to the business what engineering says, and to engineering what the business wants. On an AI project that position adds little, because both sides are discussing an object each of them only half understands.

• Ask to see real system outputs, including bad ones, before signing off a specification.
• Refuse an unmeasurable objective and propose an evaluable rewording.
• Own the reference case set as a business deliverable, not as a testing chore.
• Make the cost of an error visible: it drives the entire control setup.
• Say no to a use case whose data does not exist — often the most useful contribution available.

This grounding does not make you an AI engineer, and it is not meant to. It makes you the person able to turn a vague intention into a system that can be evaluated, controlled and adopted — a capability very few organisations currently have.

• Use-case framing
• AI literacy
• Adoption