From Vague AI Request to Defined Scope: Scoping an AI Project

It is a scenario we encounter almost weekly in AI consultancy. A board member or manager reads a promising article about Generative AI, gathers a team, and says: "We need to do something with ChatGPT for our internal documents." Subsequently, this assignment is thrown over the fence to the IT department or an innovation team.

The result? Endless "proof of concepts" that lead nowhere, frustrated developers, derailed budgets, and ultimately the conclusion that "AI is just not ready for our company yet." In most cases, however, the problem lies not with the technology, but with a fundamental lack of scoping.

In this article, we explain exactly how to turn a vague AI request into a tightly scoped, realistic, and actionable project. We cover formulating the right acceptance criteria, establishing constraints, and we provide a directly usable AI Project Scoping Canvas.

Why is Scoping AI Different from Traditional IT?

Before we dive into the methodology, it is crucial to understand why you cannot scope an AI project in the same way as building a regular web application or rolling out a new CRM system.

Traditional IT is deterministic. If you click 'Order' in a webshop, you expect the same result 100 out of 100 times. The rules are programmed by humans (if X, then Y). With deterministic systems, you can therefore write binary acceptance criteria: a feature works, or it doesn't.

AI, and specifically Large Language Models (LLMs) or machine learning systems, are probabilistic. They work based on probabilities. The model generates an output that is statistically most logical. This means that 100% error-free performance is an impossible goal. You can always encounter unexpected output, variations in phrasing, or even hallucinations (where the model confidently presents factually incorrect information). By the way, learn more about preventing LLM hallucinations on our educational domain.

Additionally, with AI, the data is part of the logic. With traditional software, bad data at most results in an empty field or an incorrect report. With AI, bad or incomplete training or context data leads to a fundamentally failing system. You cannot, therefore, scope an AI project without scoping the underlying data.

Step 1: Clarifying the Real Problem (Problem Framing)

The first step in scoping is to ignore the technology. "We want an AI chatbot" is a solution, not a problem. The question is: what business or operational problem are we trying to solve?

Use the 'Five Whys' technique to get to the core:

Why do you want an AI chatbot? Because customer service is overloaded.
Why are they overloaded? Because we are growing, but we cannot find additional staff.
Why does it take the current employees so much time? Because they receive 200 calls and emails per day about the status of returns.
Why do customers call about that? Because our track & trace page for returns is not up to date.
Why is it not up to date? Because the warehouse system does not synchronize properly with the website.

Suddenly, the problem is no longer a lack of AI, but a synchronization issue in the backend. If, after this analysis, you still decide on an AI application (for example, automatically summarizing long, complex legal files to prepare lawyers), you now have a crystal-clear goal in mind: Realizing time savings in the preparation of legal files.

Step 2: Formulating Success Criteria and Acceptance Criteria

Now that the problem is clear, we must define when the project is successful. In doing so, we make a strict distinction between Success Criteria (business perspective) and Acceptance Criteria (technical and functional perspective).

Success Criteria (The Business Case)

Success criteria are about the value the project delivers. For example:

The average handling time per email decreases by 30%.
40 hours per week are saved on manual data entry.
Customer satisfaction (CSAT) around email support remains the same or increases compared to the current measurement (benchmark: 7.8).

Tip: Make sure you know the current situation (the baseline measurement), otherwise you cannot demonstrate success. Want to make this financial? Read our article on calculating the ROI of AI projects.

Acceptance Criteria (Technical & Functional)

This is where probabilistic systems get tricky. You cannot write: "The model must never make a mistake." You have to think about acceptable error margins and speeds.

Important Assumption for AI Acceptance Criteria

Accept that an AI system will rarely perform better than the 'ground truth' or a human expert with limited time. Aim for the system to be 'good enough' to achieve the business value, not for it to be perfect. After all, humans make mistakes too.

Examples: Good vs. Bad

Domain	Bad Acceptance Criterion (✗)	Good Acceptance Criterion (✓)
Accuracy	The system must not make any errors when classifying incoming documents.	The system correctly classifies the document in at least 85% of cases. For a confidence score below 70%, the system routes the document to a human employee (Human-in-the-Loop).
Speed / Latency	The AI must respond immediately.	The Time-To-First-Token (TTFT) is on average less than 1.5 seconds, and the total generation time per response (up to a maximum of 500 words) does not exceed 5 seconds in 95% of cases.
Scope	The chatbot must be able to answer all customer questions about their products.	In this phase, the chatbot specifically supports three flows: Forgot password, Order status (API integration), and Consult return policy. All other intents result in a hand-over.
Safety & Guardrails	The AI must not say weird things to customers.	When entering prompts related to politics, violence, or competition, or during prompt injection attempts, the system returns a predefined refusal message. This is tested weekly with a red-team test set of 100 injection prompts.

Step 3: Defining the Scope (In/Out)

Scope creep (the project gradually expanding unnoticed) is the silent killer of AI initiatives. During a project, developers often discover new capabilities of a model, or the business gets new ideas ("Oh, can it do this too? Let's add that!").

It is essential to strictly define what is Out of Scope. Sometimes the Out of Scope list is more important than the In Scope list.

In Scope: Retrieving customer information from the internal CRM via a RAG (Retrieval-Augmented Generation) architecture. Supported languages: Dutch and English.
Out of Scope: Modifying (writing) data in the CRM by the model. Real-time speech recognition. Support for French or German queries. Integration with the HR system.

Step 4: Documenting Assumptions and Risks

At the start of an AI project, you do not know everything yet. The quality of your data is often a black box until you start working with it. Therefore, document your assumptions and identify project risks immediately.

Common assumptions:

We assume that the required API access to the SaaS application of vendor X can be arranged within 2 weeks.
We assume that the current data (e.g., manuals) is accurate, up-to-date, and available in a digitally readable format (no scanned, skewed PDFs without OCR).
We assume that the costs for tokens (LLM usage) will remain within the initial budget of €500 per month at the expected volume of 10,000 requests.

Step 5: Data and Access Requirements (Governance)

Without data, there is no usable AI. Before a single line of code is written or a single prompt is engineered, it must be clear what data is required and how it will be accessed.

Ask yourself the following questions:

Location: Where is the data currently stored? (SharePoint, AWS S3, SQL database, external vendor?)
Format: Is the data structured (tables) or unstructured (text, audio, video)?
Privacy & Security: Does the data contain Personally Identifiable Information (PII)? Does it need to be anonymized before being sent to an LLM? This is a critical point under the GDPR. You can read more about this in our article on AI governance in SMEs.
Access Rights: If the AI performs a search in SharePoint on behalf of employee X, is the AI also allowed to see HR documents that employee X normally cannot access? Setting up Role-Based Access Control (RBAC) in combination with AI is often the most complex part of the project.

Step 6: A Realistic Phasing (From Pilot to Production)

A successful AI project is rarely built and launched from start to finish in one go (the 'Big Bang' approach). With AI, you learn along the way how the model responds to your specific business data. We therefore always advise a phased approach. By defining this in your project scope, you directly manage the expectations of the management.

Phase 1: Data Discovery & Feasibility (1-2 weeks) - Is it actually possible? Is the data good enough? Go/No-Go moment.
Phase 2: Proof of Concept (PoC) (2-4 weeks) - A quick, technical validation. We build the system with a static export of a small portion of the data to test the quality of the LLM output. No complex integrations.
Phase 3: The Pilot (Minimum Viable Product) (4-6 weeks) - A test with a closed group of end users in a controlled environment, integrated into the real systems. For the exact details of this phase, read our guide on the 30-day AI Pilot.
Phase 4: To Production and Scale - Rollout to the entire organization, including monitoring (LLMOps) and continuous development. How to make this transition is covered in from pilot to production.

The Concrete AI Project Scoping Canvas

To help you actually document all these elements, we have developed the LLMnet AI Project Scoping Canvas. This is a pragmatic format that you can copy directly to your own word processor or wiki system (such as Confluence or Notion).

Ensure this document is filled out during a kick-off meeting attended by the technical lead, the data owner, and the business sponsor.

📋 AI Project Scoping Canvas

1. PROJECT INFORMATION Project Name: Client (Sponsor): Technical Lead: Start Date: Target Delivery Date Pilot: 2. THE PROBLEM (Problem Statement) - What is the current problem or inefficiency? (Describe the pain point, not an AI solution) - Current process description (in 3 sentences): - Current (measured) time spent or costs associated with this problem: 3. SUCCESS CRITERIA (Business Value) - Goal 1: (E.g., 20% reduction in time spent on X) - Goal 2: (E.g., X dollars/euros savings per quarter) - How is this measured? (Which metric/dashboard do we use for this) 4. ACCEPTANCE CRITERIA (Technical & Functional) - Quality/Accuracy: (E.g., "In at least 80% of test cases, the system provides the correct answer.") - Performance/Speed: (E.g., "Response within 3 seconds.") - Fallback/Error-handling: (What should happen if the model does not know the answer?) 5. SCOPE DEFINITION IN SCOPE: - - - OUT OF SCOPE (Explicitly excluded): - - - 6. DATA AND INTEGRATION REQUIREMENTS - Sources: (Specify which systems: e.g., AFAS, Exact, local fileshare) - Access: (Who needs to provide API keys or open firewalls?) - Privacy: Does the data contain personal data? [ ] Yes [ ] No - Anonymization required? [ ] Yes [ ] No 7. KEY ASSUMPTIONS AND RISKS - Assumption 1: - Assumption 2: - Risk 1: (E.g., LLM API limits are exceeded) -> Mitigation: ... - Risk 2: (E.g., Employee adoption is disappointing) -> Mitigation: ... 8. PHASING & GO/NO-GO MOMENTS - Date Feasibility Check (Go/No-go): - Date Proof of Concept presentation: - Date start MVP/Pilot with end users: 9. APPROVAL [ ] Business sponsor approval [ ] Technical lead approval [ ] Data owner / Security officer approval

Conclusion

The success of an AI project is not determined by choosing the latest model with the most parameters, or by writing the most ingenious prompts. Success, just like in construction, is determined in the drawing room.

By taking the time to rigorously scope your AI project – by identifying the real problem, applying probabilistic acceptance criteria, and strictly defining data requirements – you prevent your project from getting stranded in the notorious "pilot purgatory" phase. Use the canvas above for your next AI initiative, and you will find that discussions about expectations and results run significantly smoother.