Why Every AI Project Should Start with a Two-Week Proof of Concept

The most expensive AI projects we hear about are not the ones that failed loudly. They are the ones that were specified for six months, built for six more, and then quietly discovered that the model could not actually do the thing reliably enough to matter — a discovery that could have been made in week two for a fraction of the cost. AI projects have a property traditional software does not: the core risk is feasibility, not engineering. Nobody wonders whether a checkout page is possible. Whether an LLM can classify your messy supplier invoices at 95% accuracy, in your language mix, on your real data? That is a genuine open question, and no vendor deck can answer it.

This is why we push every AI engagement — including our own internal products — through the same gate: a scoped proof of concept, roughly two weeks, before anyone commits real budget. Our PoCs start around US$3,500. That is not a discount teaser; it is deliberately priced as the cheapest insurance policy in the project, and sometimes its correct outcome is "don't build this." We have told prospects exactly that, and we consider those engagements successes.

Why AI needs a PoC when normal software doesn't

Three structural reasons:

• Demos lie. Every AI capability looks magical on curated examples — that is what demos are for. Production data is where accents, typos, edge-case formats, and ambiguous inputs live. The gap between demo performance and production performance is the single most underestimated number in this industry, and you can only measure it on your data.
• Accuracy is a business variable, not a technical one. A model that is 85% accurate is a triumph for drafting marketing copy and a catastrophe for invoice booking. Until you know the achievable accuracy and the cost of each error, you cannot compute ROI — you can only feel enthusiasm.
• Unit economics can kill a working feature. We run our own AI products and pay our own model bills, so we have internalized this one: a feature that works but costs US$0.40 per use inside a US$10/month subscription is not a feature, it is a leak. Token cost per unit of value must be measured before you scale, not after. (We covered the operational side in controlling LLM costs in production.)

A two-week PoC is not a smaller version of the project. It is a targeted experiment to kill the project's biggest unknown while it is still cheap to be wrong.

How to design a PoC that actually proves something

The failure mode of PoCs is being a toy: a chat UI over sample data that impresses everyone and proves nothing. A real PoC has four properties.

1. One question, written down

Not "explore AI for customer service" but: "Can we draft correct responses to at least 70% of tier-1 support emails, in Chinese and English, such that an agent approves them with minor edits?" If you cannot phrase the question with a number in it, you are not ready to spend money answering it.

2. Real, ugly data

The PoC must run on a representative slice of production data — the misspelled queries, the photographed-at-an-angle documents, the customers who write three topics in one email. Fifty to two hundred real examples with known correct answers beats ten thousand synthetic ones. Assembling this evaluation set is often half the PoC's work and, frankly, half its value: the eval set outlives the PoC and becomes the yardstick for every future model swap and prompt change.

3. Thresholds agreed before results exist

Decide in advance what number means go, what means kill, and what means iterate. This sounds bureaucratic; it is actually psychological armor. Once a demo exists, sunk-cost gravity and stakeholder excitement will bend any ambiguous result toward "green light." Pre-committed thresholds are how you keep the experiment honest.

4. Skip everything that isn't the risk

No SSO, no admin panel, no design system, no Kubernetes. A script, a spreadsheet of results, and the thinnest possible interface for stakeholders to poke at. Every hour spent on production plumbing during a PoC is an hour not spent on the only question that matters. The plumbing is week 5's problem, if week 5 happens.

What two weeks actually looks like

1. Days 1–2: Lock the question and thresholds, collect and label the evaluation data.
2. Days 3–7: Build the thin pipeline — prompting, retrieval if needed, a first pass with a strong model to find the ceiling.
3. Days 8–10: Iterate against the eval set; test a cheaper model tier to sketch the cost curve.
4. Days 11–14: Run the full evaluation, write up accuracy, failure patterns, cost per unit, and a build/kill/iterate recommendation with an honest estimate for the production phase.

The deliverable is not the prototype. It is the evidence: a short report a non-technical decision-maker can act on, plus the eval set, plus working code that a production build can grow from. If someone offers you a PoC whose deliverable is a demo video, you are buying theater.

When to kill, and when to skip the PoC entirely

Kill signals we take seriously: accuracy plateaus well below threshold after real iteration; the errors are confidently wrong in ways humans won't catch; cost per unit exceeds the value per unit with no tiering path; or — the most common one — the PoC reveals the actual bottleneck is data quality or process, not intelligence. A kill verdict at US$3,500 with evidence is not a failure. It is the entire point. It also frequently redirects the budget somewhere better: twice we have seen "we need an AI agent" turn out to mean "we need a search box and cleaner data."

Honesty also requires the reverse list. You can skip a PoC when the pattern is genuinely commoditized and the failure cost is low: transcription, translation drafts, meeting summaries, boilerplate generation with human review. Buy an off-the-shelf tool for a month and see if the team uses it — that is the PoC, and it costs a subscription fee. Custom PoCs earn their keep where your data, your language mix, your accuracy bar, or your integration surface is unusual.

The takeaway: in AI, conviction is cheap and evidence is not much more expensive. Two weeks and a few thousand dollars buys you a measured accuracy number on your own data, a cost-per-unit figure, and a defensible decision — before the six-figure commitment, not after. If you have a candidate use case and want it pressure-tested this way, that is exactly what the PoC tier of our services exists for.