AI strategy01
Before you buy another AI tool, map one day of work.
A practical way to find a useful AI starting point: follow one task, identify the friction, and define a result worth measuring.
The journal / AI strategy / Note 02
AI strategy · Field note 02An AI pilot needs an exit, not just a launch.
It is easy to start a pilot. The harder part is deciding what evidence would justify continuing it. Work backwards from that decision.
Write the final decision first
Before anyone signs up for a tool, finish this sentence: “At the end of this trial, we will decide whether to…” Be specific. Perhaps the decision is whether to use an assistant for a particular internal summary, with a named reviewer, for a defined team.
This stops a pilot from quietly becoming an open-ended exploration. Exploration has a place, but it should be described that way. If a trial is meant to inform an operational decision, write down the question it is supposed to answer.
Keep a fair comparison
Use examples that resemble the real task, and include some awkward ones. A tidy sample can hide the very difficulties that matter: an incomplete note, conflicting instructions, or an exception that requires judgment. Use information approved for the tool and purpose.
Keep the current method as a reference. Compare the finished outputs against the same expectations. If one version receives extensive human editing and the other does not, record that difference. The comparison should account for the work needed to produce an acceptable result.
Name the owner and the reviewer
The person running the experiment and the person approving the output may have different jobs. Make both roles explicit. Someone should maintain the trial log; someone should be able to say that an output is unsuitable even when it looks polished.
A short log is enough to begin: the task, the input version, the output, the review result, the correction needed, and the final decision. Keep it in an approved place. The purpose is to make learning visible, rather than rely on the most memorable demonstration.
Agree on the stop conditions
Discuss what would pause the trial. Examples could include a repeated factual error, unexpectedly high review time, or a task that cannot be tested without information the chosen system is not approved to handle. These are proposed design criteria, not universal thresholds.
Also decide how you will return to the existing workflow. Keep source material and responsibilities clear enough that stopping does not strand unfinished work. Reversibility makes it easier to try an idea without treating every experiment as a commitment.
Finish with a short decision note
The final note should explain the original question, what was tested, what improved, what failed, and what the team recommends. Include the limitations. A narrow pilot cannot establish that the same approach will work everywhere.
Choose to continue, change, or stop, and describe the conditions attached to that choice. A pilot earns its keep by producing a grounded decision. A successful launch event is a much weaker result.
One thing to take away
Define the decision, comparison, reviewer and stop conditions before the trial begins.
Further reading
NIST — AI Risk Management Framework ↗
For a broader framework on considering and managing AI risk across an organization.
Pasqualino Mazza
Entrepreneur, global AI consultant, and content creator. Writing about AI, business, and leadership from a Toronto perspective.