Your pilot is done. Now decide what to do next: scale up, adjust, or stop, and document learnings for the future.
The first step is moving the pilot into production. The product needs to go beyond provisional testing to implementing a real, maintained system that people actually use.
Consider sign-offs, infrastructure, handover and training.
What changes or improvements will you make before scaling?
Once in production, scaling can take three forms.
Use the same use case and office, but address more cases. For instance, extend coverage from sample data to the full set, or expand it from in-office use for validation to on-the-ground use for primary prediction.
Watch forNew edge cases the pilot didn’t see. Plan a review at 3 months.
Apply the same use case horizontally across more offices or locations. For instance, roll it out to additional districts or ministries.
Watch forLocal context differences, training requirements, onboarding.
Apply the same or a slightly adapted approach to a different use case. For instance, fraud detection methods for tax could be adapted to electricity theft.
Watch forTreating every case the same and copy-pasting the method. Each new domain needs to review the problem-definition step and adapt accordingly, using the steps in this playbook.
What does the first phase of scaled deployment look like? Name the offices, the users, the timeline, and who owns it.
Document what happened, including what did not work. Honest reporting builds credibility, improves your next project, and helps other governments facing the same challenges.
Hold conversations to discuss outcomes and lessons learned with your team. Maintain a repository of AI projects so teams can easily find and access existing AI projects they can build on while avoiding repetitive work.
See how one team moved from problem to production and beyond.
Zambia Evidence Lab & National Statistics Agency
The national statistics agency needed a way to improve their national statistics through better classification of open-ended survey responses.
The survey data was readily available, but occupation and industry classifications often suffered due to the overwhelming number of possible codes, poor and unstructured response quality, and lack of time and context.
An AI classification system was identified as a valuable approach to address this problem.
An initial classification system was built and evaluated for accuracy, which showed promising results. It was refined and tested with various models, but not to perfection.
The Zambia Evidence Lab scheduled a deployment with the statistics agency, involving a training with numerous public servants for hands-on testing and use of the tool. Some edge cases were identified, along with a post-hoc analysis of failure points which resulted in iterations for improving the prompt, testing different models, adjusting parameters, and improving input survey data.
The first step was deploying it for use within the agency offices. Next was to deepen the tool's reach by implementing it on the ground during survey collection, involving change management with training for enumerators, a phased rollout, and randomised treatment groups to evaluate the real impact of the tool.