Begin with the work, not the demo
A convincing demonstration answers a narrow question: can this system produce an impressive response under these conditions? A pilot needs to answer a harder one: can a team use it to improve a real workflow without creating unacceptable costs or risks? Those are different tests. Start by describing a task people already do, the people it serves, and the failure that makes it worth improving. “Use AI in customer service” is too broad. “Help a reviewer draft answers to opening-hours enquiries using one approved reference” is a bounded starting point.
Write down the baseline
Observe the existing process before changing it. Record what a completed task looks like, how a reviewer decides whether it is acceptable, and where delays or corrections occur. Do not reduce success to response speed. A quick answer that needs substantial correction can create more work than a slower, reliable process. Include the time spent checking sources, fixing errors and escalating uncertainty. For an early educational exercise, a small, deliberately varied set of synthetic enquiries is more useful than uploading an unexamined folder of real customer messages.
Make ownership visible
Name the person responsible for the workflow and the person authorised to stop the test. Identify who can approve data use, who reviews outputs and who receives reports of failures. One person may hold several roles in a small team, but the responsibilities should still be explicit. A vendor is not a substitute for an internal decision-maker. Before involving students or external contributors, agree supervision, confidentiality, attribution and access. A friendly invitation does not establish permission to use institutional information.
Design the uncomfortable test cases
Include questions the reference cannot answer, contradictory requests and requests outside the agreed scope. Suppose a synthetic library reference gives opening hours but says nothing about laptop lending. Ask whether a visitor can borrow a laptop on a closed day. The useful behaviour is to state the supplied hours, acknowledge the missing lending information and avoid inventing an offer. An answer that sounds helpful but makes up lending terms should fail. Keep the prompt, reference, output and reviewer explanation together so another person can understand the judgement.
Agree stop conditions before results arrive
Write down which failures require pausing the test. Examples for discussion include disclosure of restricted information, an unsupported eligibility claim or an action performed without approval. These are starting points, not universal thresholds. The right conditions depend on the setting and must be approved by the accountable owner. Keep the initial exercise read-only: it can draft, but it cannot send, purchase, decide eligibility or update a system of record. If a human cannot inspect and reverse a proposed action, the scope is probably too large for a first test.
Publish a decision, not just a success story
Close the pilot with a short note: the workflow, baseline, permitted inputs, method, failures, review costs and decision. “Do not proceed” can be an excellent outcome if it prevents an unsuitable deployment. If you continue, state what must change and who owns the next review. Share only material cleared for publication. The value of a community is not a collection of impressive demos; it is the ability to learn from decisions that other teams can inspect and challenge.
Try this next
Write a one-page brief with six headings: task, current process, owner, permitted data, acceptance evidence and stop conditions. Ask someone outside your discipline to identify the least clear sentence. Revise that sentence before selecting a tool. This is an original discussion guide for the founding concept, not a validated evaluation protocol or legal advice.
Sources & editorial context
Original discussion analysis, not a reported interview or programme result. Synthetic examples are illustrative. Related reading provides context, not endorsement.
BPOAI — The Team Behind BPOAI.ai Joins NVIDIA Inception (9 October 2026) ↗
POEA Job Finder — Andy Schachtel Keeps People in the Loop (9 October 2026) ↗