Difficulties in AI projects are rarely purely technical. They usually come from decisions taken separately by functions that do not share the same definition of the problem, the same constraints or the same horizon.
This guide describes the real decisions a team faces when building an AI product, the most frequent frictions, and the practices that reduce them. It does not propose an ideal organisation: effective teams look very different from one another.
The functions involved and what each brings
An AI team is not a stack of specialities. Each function carries a constraint the others cannot see from where they stand.
- Product: arbitrates value, scope and sequencing.
- AI engineering: makes the system feasible, reliable and operable.
- Data: determines what the system can know and how far it can go.
- Design / UX: decides what the user understands, controls and can verify.
- Domain experts: hold the definition of what is correct in the real context.
- Security and compliance: set the non-negotiable limits.
This grouping only serves to describe the interfaces between disciplines. Depending on the organisation, Data Scientist, ML Engineer and AI Engineer carry distinct responsibilities — methodology and evaluation on one side, industrialisation and operations on another, the design of model-based systems on a third. Placing them in one column here does not mean they are interchangeable, nor that one of these roles is enough on its own.
In a small company one person covers several of these. The question is not headcount, but whether every constraint has someone carrying it.
The decisions to make, and who carries them
A misassigned decision costs more than an imperfect one. The list below covers the structuring decisions of an AI product and how they are made in practice.
- The problem addressed — carried by Product and domain experts, with engineering saying what is realistic.
- The definition of a good result — carried by domain and Product, formalised with data and engineering.
- System scope — carried by Product, constrained by available data and permissions.
- Technical approach — carried by engineering, arbitrated with Product on cost and time.
- Evaluation protocol — carried by data and engineering, fed with real cases by the domain.
- Level of user control — carried by Design, constrained by risk and engineering.
- Guardrails and action limits — carried by security and engineering, arbitrated with Product.
- Timing of the production release — carried by Product and engineering, with domain agreement.
- Stopping a use case — carried by Product, based on what the domain observes.
Three decisions no one makes alone
A few decisions concentrate most of the disagreements in an AI team. They almost always have a named owner, but none of them can be informed from a single discipline: each one holds information the others do not have.
Is quality good enough to launch?
There is no absolute answer: it depends on the use case, on who is affected and on what happens when the system is wrong. No universal quality threshold exists, and an evaluation figure never settles the question on its own.
- Product: the expected value, the intended scope, who is actually affected, and what a limited or delayed launch costs.
- Data / ML / AI Engineering: evaluation results, observed failure cases, stability, technical constraints, what is observable in production and under which conditions the system can be operated.
- UX / Research: what users understand from the output, how much trust they place in it, whether they can spot and correct an error, and how they actually behave when shown a suggestion.
- Domain expert / business: what quality is genuinely acceptable in context, which exceptions matter, and the operational consequences of an undetected error.
The decision is rarely “launch or not”: it is about for whom, on which scope, and with what safety net.
Automate or assist?
That a task is technically automatable does not mean automating it is desirable. The trade-off is built from a few concrete elements, and it can differ from one use case to another inside the same product.
- The consequence of an error, and whether it can be reversed.
- Whether someone can verify the result with reasonable effort.
- How frequent exceptions are, and how much judgement they require.
- Responsibility: who answers for the outcome once the action is executed.
- The real value of automation compared with well-designed assistance.
Here too the answer is distributed: the business side says what an error causes, engineering says what the system does reliably, UX says whether verification is realistic for the user, and Product decides in view of the intended use.
What should happen when the system is uncertain?
A probabilistic system will meet cases it handles poorly. What it does in those moments is a product decision in its own right, not an implementation detail. A confidence score can help, but it does not solve the question: it only moves the threshold.
- Ask for additional information before proposing anything.
- Surface the uncertainty, provided it is genuinely interpretable by the user.
- Offer a result but require explicit validation.
- Fall back to a deterministic path or a known rule.
- Escalate to someone competent on that case.
- Decline to act, and say so clearly.
- Let the user take over at any point.
Product decides what the product promises in those cases, engineering and data say what is detectable and at what cost, UX designs what the user sees and can do, and the business side says which situations tolerate no automatic action at all.
An owner does not decide alone
Naming an owner prevents collective indecision; it does not mean that person holds the information needed. A healthy team separates four things: who carries and answers for the decision, which expertise contributes to it, what information is still missing, and whether a validation is required before it is applied.
This is not about writing a universal responsibility matrix: the split changes with team size and how critical the product is. What must stay constant is that each of those four questions has an explicit answer.
Recurring frictions
The same tensions recur across teams. Naming them lets people treat them as normal topics rather than personal conflicts.
Quality versus deadline
A system can be sufficient for a narrow first use and insufficient for a broad rollout. The friction disappears when the team explicitly separates the two levels of requirement instead of debating quality in the abstract.
Uncertainty versus date commitments
Exploratory work cannot be planned like known development. Teams that cope split the subject into questions to settle, with a date for the answer rather than a date for the final outcome.
Available data versus needed data
Many features fail not on the model but on data that is missing, poorly labelled or not authorised for that use. This belongs in framing, not in mid-project discovery.
Demo versus production
A convincing prototype says almost nothing about production behaviour. A healthy team separates what was shown from what was measured, and does not treat a demo as evidence.
Orphan evaluation
When nobody owns evaluation, it happens late, by the least well-placed person, on cases that are too easy. This is probably the most expensive and the least visible friction.
Without an evaluation owner
The test set is assembled at the end by engineering, from cases already handled. Results look good and teach nothing.
With a named owner
The domain provides hard cases from framing onwards, engineering instruments them, and the team follows a measure that moves when the system changes.
What matters is not who owns evaluation, but that someone explicitly does.
Turning a disagreement into an informed decision
Take the most common friction: Product wants to launch, evaluation still shows serious failure cases. As long as the discussion pits two opinions against each other, it goes nowhere. It moves once it is reframed as a hypothesis to test. The sequence below is a teaching scenario, not a procedure.
- 1
Hypothesis
step 1The remaining errors concentrate on an identifiable type of case, rather than spreading across the whole use case.
- 2
Risk
step 2If those cases affect a sensitive population or an action that is hard to undo, being rare is not enough to make them acceptable.
- 3
Missing information
step 3Their real frequency in use is unknown, and above all whether users can spot the error before acting on it.
- 4
Test
step 4A targeted failure analysis by type of case, user observation on those precise situations, and the instrumentation needed to count them once live.
- 5
Decision
step 5Depending on what the test shows: launch on a restricted scope, change the product so the error becomes visible, keep human validation on those cases, or postpone.
The right outcome is not known in advance and does not generalise: what transfers is the move from clashing opinions to a missing piece of information the team decides to go and get.
What concretely improves collaboration
- 1
Write down the definition of a good result
at framingOne sentence, validated by the domain, stating what the system must produce and what it does not have to do. It anchors every later discussion.
- 2
Assemble a set of real cases
at framingA set of field cases large enough to cover the difficult situations, the exceptions and the relevant out-of-scope cases is worth more than a long requirements document.
- 3
Separate demo from measurement
ongoingShowing a result is useful for alignment; deciding requires a measure on cases nobody picked for the occasion.
- 4
Make constraints visible
ongoingLatency, cost, permissions, data freshness: exposing them early avoids product decisions being invalidated later.
- 5
Name one owner per decision
per topicNot a committee: one person who decides after listening, and explains the criterion used.
- 6
Review failures together
periodicallyA short review with the domain on a handful of real failures steers the work better than a dashboard.
Teaching case: deciding whether to launch an AI feature
A fictional scenario, built to show how a team decides. It describes no real product and deliberately contains no figures.
A feature analyses business information and proposes an action to the user. On common situations, the team considers quality satisfactory. On ambiguous or incomplete cases, it varies far more. The question facing the team is not “is it ready?” but “what do we launch, for whom, and under which conditions?”.
- Product: identifies the use cases where the feature genuinely helps, and accepts a narrower initial scope rather than covering every situation.
- Data / ML / AI Engineering: shows where quality degrades, says what can be detected automatically, what is expensive to monitor, and what will have to be observed once live.
- UX / Research: designs how the suggestion is presented, checks that users can push back on it, and flags wording that invites excessive trust.
- Domain expert / business: states which actions must never run without review, and which errors can be undone without consequence.
From there the team can decide what is automated, what stays a recommendation, and what requires human validation. Ambiguous cases may, depending on context, trigger a request for information, a fallback to the usual process, or no suggestion at all.
The outcome need not be binary. A launch restricted to a few teams, automation limited to the best-covered cases, human validation kept on the rest, extra instrumentation before extending, or a new evaluation on hard cases before widening: all are reasonable decisions. What matters is not the choice made here, but that each discipline contributed the information it held and that one person decided with an explicit criterion.
The posture to adopt in each role
Understanding other people's constraints does not mean doing their job. The useful posture is knowing what limits the other functions, so your own requests are actionable.
- Product: express a need as a decision to improve, not a feature to ship.
- Engineering: state constraints in terms of product impact, not only technical terms.
- Data: say early what the data does not allow, and under what conditions that would change.
- Design: translate system uncertainty into interface elements people can read.
- Domain: provide real cases rather than general rules.
- Security: separate non-negotiable limits from recommendations, to avoid blanket blocking.
This collaboration skill is rarely written on a profile, yet it is highly visible in interviews: it shows in how someone tells the story of a decision, its constraints and its criterion.
Diagnosing an AI team
Six questions to ask inside an existing team or in an interview.
- Who decides what counts as a good result?
- Who owns the test set, and who feeds it with difficult cases?
- How does a technical constraint travel up to a product decision?
- Who arbitrates when quality, latency and cost conflict?
- When do domain experts see the system, and what can they still influence?
- How are security and data questions handled: upfront or at the end?
Key points
- Decisions in an AI team constrain one another: few can be taken in isolation.
- Evaluation is a shared decision, not a technical task delegated away.
- Recurring frictions are predictable; explicit rules dissolve most of them.
- Collaboration quality often matters more than individual excellence.
- Being able to state a decision and its criterion is a cross-functional skill.
- Each function benefits from understanding the others' constraints, without doing their job.
- AI engineering
- AI product
- Evaluation