How to Build an AI MVP: Architecture, Models, Evaluation and Cost

To build an AI MVP that people keep using, start from one painful workflow, not from a model. Decide what the AI is allowed to do inside that workflow, keep the business rules in code, ground every answer in your own data, validate every action the model proposes and measure quality on real examples before launch. The model itself is usually the easiest part to change later. S&P Global Market Intelligence found that 42% of companies abandoned most of their AI initiatives before production, a finding covered in our startup statistics.

This AI MVP development guide follows the order we use when founders ask us to build an AI MVP with them:

1. Pick one user, one workflow and one measurable outcome.
2. Collect real examples of the work and turn them into an evaluation set.
3. Design the architecture: workflow state, retrieval, validated tool calls and human review.
4. Choose models by tier: plain code first, small models next, frontier models only where they are needed.
5. Set a cost budget per task and a latency target per screen or call.
6. Ship a fixed-scope first release, usually planned as a 4-week build, then improve it from real failures.

The sections below cover how to build an AI MVP step by step, with what we learned shipping AI products such as Skalix AI, and a short list of cases where AI is the wrong tool for your MVP.

The weakest AI MVPs are a text box in front of a general model. Users have to know what to ask, results vary from one attempt to the next, and any competitor with the same API access can rebuild the product quickly. A wrapper can be a useful experiment, but it is rarely a business.

The stronger pattern embeds AI inside a job someone already does and already finds painful: qualifying inbound leads, reading supplier quotes, drafting a weekly post from a voice note, turning a room photo into a styled design. The user does not write prompts. They do their work, and the product handles several of the steps for them.

Three questions help you pick the right workflow:

• Is the work frequent and repetitive? Daily or weekly tasks give you enough real examples to evaluate against and enough usage to learn from.
• Is the input messy? Models earn their place on unstructured input: speech, free text, documents, images. If the input is already a clean form, plain code may be enough.
• Can a wrong answer be caught? Pick a workflow where errors are visible and reversible, or where a person reviews the output before anything irreversible happens.

Then write the outcome down in numbers you can observe: tasks completed without a person, minutes saved per task, leads qualified per week. That outcome, not model accuracy in the abstract, is what the MVP has to prove. If you are still deciding which slice to build first, our piece on MVP scope covers how to cut features without losing the result.

AI MVPs that hold up in production tend to share the same four layers. None of them is exotic, and all of them run comfortably on a small, conventional stack, such as TypeScript or Python services on top of PostgreSQL. When you build an AI MVP, plan these layers before you tune a single prompt.

1. Workflow and state, owned by code

The workflow is a state machine: the steps a task moves through, the transitions allowed between them and who owns each step. The model can suggest the next step or fill in a field, but code decides whether the transition is allowed. This one rule removes a large class of failures, because the model cannot skip a qualification step, invent a refund or book a meeting into a slot the calendar does not have.

Store state in your database, not in the conversation history. A task should survive a timeout, a crash or a handoff to a person and carry on from where it stopped.

2. Retrieval: the model sees only what it needs

Retrieval supplies the facts: product details, policies, past conversations, documents. Each step gets the few passages it needs rather than everything you have, and the product records which sources it used so answers can be checked. The next section covers this layer in detail.

3. Tool calls with schema validation

When the model wants to act, such as updating a CRM record, sending an email or creating a booking, it returns a structured tool call. Validate every call against a schema, using a library such as Zod or Pydantic, before anything happens. If the call fails validation, send the error back to the model for one corrected attempt, then fall back to a person. Never parse free text with regular expressions to decide an action.

Give each tool the narrowest permission it needs, make writes idempotent so a retry cannot double-book, and log every call with its inputs and result.

4. Human review and fallback

Decide up front which actions a person must approve: anything that moves money, makes a commitment to a customer, sends a bulk message or deletes data. Build the review queue as part of the MVP, with the context a reviewer needs to approve in seconds. When confidence is low or a dependency is down, the product should fall back to a person or a simpler path, not fail silently.

When a workflow chains several model-driven steps, the same rules apply with more force. Our guide to agentic AI workflows covers multi-step agents, tracing and approval design in more depth.

Retrieval-augmented generation (RAG) is the standard way to make a model answer from your data, and it is easy to over-engineer. For a first release you rarely need a separate vector database service. PostgreSQL with the pgvector extension stores embeddings next to your users, accounts and permissions, so one query can filter by tenant and access rights before ranking by similarity. Our notes on PostgreSQL for startups explain why one well-designed database goes a long way at this stage.

What matters more than the choice of store:

• Clean source data. Remove duplicates and outdated versions. A model will confidently quote a policy you retired last year if it is still in the index.
• Sensible chunks with metadata. Split documents along their natural structure and tag each chunk with its source, date, owner and access level.
• Hybrid search. Combine keyword and semantic search. Product codes, names and numbers often match better by keyword than by meaning.
• Citations. Return the source with every answer, and check automatically that factual claims come from retrieved content.
• Live facts from tools, not the index. Prices, stock and availability change. Fetch them through a tool call at answer time instead of embedding them.

If your users can only see some records, enforce that in the retrieval query itself. Asking the model to ignore documents a user should not see is not access control.

Routing every request to the largest frontier model is the most common cost and latency mistake in early AI products. Most steps in a workflow do not need it. We pick models per step, using three tiers:

Small open-weight models are worth a look when data must stay inside your own infrastructure, or when a narrow task runs at high volume. Hosted frontier models are usually the right starting point for the hardest step, because they show you what good looks like before you optimize for cost.

Put a thin interface between your code and any model provider. Models are updated and repriced often, and your evaluation set, covered next, is what tells you whether moving a step to a cheaper or newer model is safe.

An AI MVP is ready when it passes an agreed bar on real examples, not when the demo goes well. Build the evaluation set in the first week, from the examples you collected while choosing the workflow:

• Typical cases that represent everyday work, in roughly the mix users will send.
• Edge cases: missing information, contradictory input, unusual formats, other languages.
• Cases that must be refused or escalated, such as out-of-scope requests and attempts to override the system's instructions.

Score each output with automatic checks wherever possible (valid schema, required fields present, no invented prices, sources cited) and with rubric grading where judgment is needed, calibrated against a person's grades on a sample. Run the whole set on every prompt, model or retrieval change. A fix for one case should never quietly break five others.

Our guide to LLM evaluation walks through golden sets, graders and release gates step by step.

Two numbers should exist before the first line of code: a cost budget per completed task and a latency target per surface. A voice agent has to respond within a natural pause in conversation. A chat answer can stream. A nightly document job can take minutes. Different targets lead to different architectures, so set them before you build an AI MVP, not after the first invoice.

Ways to stay inside both budgets:

• Route by tier, as above, so the expensive model only sees the hard steps.
• Trim context. Send the passages a step needs, not the whole history. Long prompts cost more and often reason worse.
• Stream output so users see the first words quickly, and keep slow lookups out of the critical path.
• Cache repeated retrievals and stable answers, and batch background work that does not need to be instant.
• Cap and alert. Set per-user and per-account limits and alert on cost per task, so a runaway loop or an abusive user cannot surprise you.

Model usage is only one line in the budget. For the engineering side, our breakdown of the cost to build an AI MVP covers typical ranges for design, build and running costs.

At Arcforth, a first AI release is usually planned as a fixed-scope 4-week build, and our fastest first release reached users in 3 weeks. Before the build, many founders start with a Product Pilot or a free clickable prototype, which we deliver within 48 hours of agreeing the scope, to test the workflow with users before any model work.

A typical build looks like this:

After launch, the work shifts to reviewing failures, handoffs and cost per task every week, adding real failures to the evaluation set and releasing improvements in weekly cycles. That loop matters more than anything in the first build, which is why our MVP development process treats launch as the start of learning rather than the end of the project.

Some of the best AI product decisions are to leave AI out. Skip it, or delay it, when:

• A rule can decide. If the logic fits in a clear decision table, code is cheaper, faster and exact.
• Errors are unacceptable and cannot be reviewed. If a wrong output causes serious harm and no person can check it in time, the workflow is not ready for a model.
• You have no real examples. Without examples of the work you cannot evaluate, and you will not know whether the AI helps.
• The input is already structured. Forms, dropdowns and clean records rarely need a model to interpret them.
• The value question is still open. If you do not yet know whether users want the workflow at all, test that with a prototype first. AI adds cost and uncertainty to a question a clickable flow can answer.
• Latency or cost cannot fit the budget. If the task must respond instantly at a very low cost per call, a model may not fit yet.

Plenty of good AI products start as a plain software MVP that captures the workflow and the data, then add AI to the steps where it clearly earns its place. That is still a sound way to build an AI MVP: the AI arrives in the second release, on top of real usage data.

Skalix AI is an AI-native revenue platform with voice agents, web chat, multi-channel follow-up, CRM sync and handoff to a human rep. It went from build to launch in 4 weeks, and it follows the architecture above closely: conversations run through defined steps, facts come from business systems through tool calls, CRM updates are validated before they are written and a person takes over when a conversation needs one. The lesson we carry into every project is that reliability came from constraining the model, not from a cleverer prompt.

Livespace.ai, an AI interior design marketplace, taught a different lesson. Designers productize their style and homeowners see it applied in their own room, so every output has to stay faithful to two things at once: the designer's style and the real space in the photo. Getting that right on everyday phone photos, not curated samples, was the real work. The product reached 2.8K users within three months.

In both cases the AI was one component of a product whose workflow, data and review paths were designed first.

Arcforth is one product squad covering strategy, UX, engineering and QA. Through our AI MVP development service, we help founders and industry experts build an AI MVP around the workflow they know best, using the architecture, evaluation and budgets described here.

If you want to see the workflow working before you commit to a build, start with a Product Pilot ($1,499 once). It covers one high-value workflow, a market and competitor scan, core flows, a clickable prototype, build-ready requirements and an MVP roadmap; the engineering build is scoped and agreed separately afterward. If your goal is automating calls, chat and follow-up inside your own operations, our AI automation agency page shows how that work is shaped.