The 90-day pilot plan ends in a decision meeting with three allowed outcomes: scale, fix, or kill. Plenty has been written about getting to that meeting. Almost nothing covers the week after it, which is strange, because the week after is where most credit unions quietly lose the thread.
We keep seeing the same story. A pilot clears its criteria, everyone congratulates the owner, and the automation drifts into production without much ceremony. Six months later a second candidate surfaces and the institution starts over from zero: new baseline debate, new compliance questions, another round of arguing about who owns the thing. The first win produced an automation. Whether it produced any ability to repeat the win is a separate question, and at most shops the honest answer is no.
Closing that gap takes a handful of moves in the 60 days after the decision meeting. None of them need budget approval, which is exactly why they get skipped. Work that costs nothing tends to carry no deadline.
Give the scale decision an end date
A pilot that clears its kill criteria has earned production status. It has not earned the right to wander into production without a plan.
Scaling is a short, boring project, thirty days in most cases. The parallel run ends on a date rather than a feeling: staff stop doing the work the old way on a specific day, and the manual procedure gets archived, not deleted, because you want the fallback documented and clearly labeled as the fallback. Teams that keep running both paths indefinitely are paying for the automation twice. Some do it for a year before anyone adds up the cost.
Review changes shape at the same time. During the pilot you checked every output, then loosened by category as agreement held. Production review is a sampling regime, and a sampling regime only works written down: what percentage gets checked each week, what threshold triggers escalation, whose name is on the checking. Leave it verbal and it decays into no sampling by the second quarter. We have watched that happen at institutions that did everything else right.
Monitoring needs an owner and a place to live. The four numbers from the parallel run still apply (volume, override rate, error rate, cycle time), and where they land matters less than that somebody reads them on a schedule. A vendor dashboard wired into a monthly ops review is ideal. A spreadsheet updated every Friday is fine. A vague intention to watch how things go is neither.
Then the examiner file gets its closing entries: decision memo, cutover date, sampling plan, monitoring cadence. The folder you opened on day 15 of the pilot now tells a complete story, from candidate selection through the controls that survived into production. On exam day that folder does more for you than any policy statement your board has ratified.
Fix decisions follow the same logic with a shorter fuse, one 30-day cycle against the same criteria, scheduled before anyone leaves the room. Kills produce a one-page postmortem and hand their baseline method and compliance file to the next candidate. Treat a clean kill as proof your criteria have teeth; examiners read it that way too.
Write the playbook before it evaporates
Everything your team learned in those 90 days currently lives in the pilot owner’s head and a scatter of meeting notes. Give it a month and half of it is gone. Give it a staff departure and all of it is.
So somebody writes the playbook, one versioned document, four sections. Baselining first: the measurement method from the pilot, items per week, minutes per item, error and rework rates, and where the historical data came from, so future candidates get measured the same way and results stay comparable. Compliance second: the third-party risk review, the one-page data map, the proportionate model risk memo, and the examiner file structure, each turned into a template with the blanks marked. Third, the review-and-loosen rule, meaning the thresholds you used for moving categories from full review to spot checks. Last, the decision meeting itself: who attends, which numbers appear, the three allowed outcomes, and the standing rule that a second fix cycle means the pilot already failed and nobody wants to say so.
Keep it under ten pages. One test tells you whether it is any good: could a manager who watched the first pilot from a distance run the second one without booking time with the person who ran the first?
Among the credit unions we have seen make automation stick, the common trait was never budget or vendor choice. It was that the second pilot cost about half what the first one did, because nobody had to reinvent the baseline argument or renegotiate what a decision meeting looks like. The machinery already existed. That is what the playbook buys.
Somebody has to own the portfolio
The pilot had a named owner with authority to stop it and an obligation to report numbers. The program needs the same thing at a slightly higher altitude: one person accountable for every automation, live and proposed.
In a $250M to $5B credit union this is a slice of an existing role, not a hire. It usually belongs in operations rather than IT, because the underlying decisions are about workflows and staffing before they are about systems. The program owner runs the intake queue, keeps the playbook current, chairs the quarterly reviews, and holds the relationship map across compliance, IT, and the vendors.
Two warnings from the field. Committees are fine for oversight and fatal for ownership; the pilot plan rule applies double here, because a program owned by a committee is owned by nobody. And resist the pull toward the most enthusiastic volunteer. Enthusiasm helps, but the job is mostly measurement, documentation, and declining weak candidates, so give it to the person who kept the override log current without being reminded. Every organization has one. It is rarely the person who talks the most about AI.
Workflow number two: adjacent beats ambitious
Momentum after a win invites a familiar mistake, the ambitious second act: a member-facing chatbot, a lending model, something the board can see. Hold off for one more cycle.
Go back to the shortlist the Three-Signals Test produced and pick the candidate closest to your first win. Same department, same document types, same vendor platform, or same reviewer pool, any of those overlaps will do. The staff already trust the review pattern. The compliance memo needs a paragraph of changes instead of a rewrite. The integration scars are fresh enough to be useful.
Tooling is the real fork in the road. Extending the incumbent vendor into a second workflow is faster and keeps monitoring in one place, but it deepens dependence on a platform whose weaknesses you have only seen at pilot scale. Bringing in a second vendor gives you a comparison point and some negotiating posture, at the price of another integration and another risk review. We have seen both go well and both go badly. What matters is choosing on purpose and writing down why, because your successors inherit the precedent either way. Our vendor map walks through the tradeoff.
Member-facing work belongs in the plan eventually. It goes better as project four than project two, after your review machinery and examiner file have survived a few quarters of contact with reality.
The maintenance nobody wants to schedule
Guardrails are boring by design, which is why they get skipped, and why programs die in year two. Four habits carry most of the weight.
A quarterly production review: thirty minutes per live automation, covering the monitoring numbers against baseline, override log themes, vendor changes, and drift in the underlying process. Drift deserves the most attention, because the workflow you automated in March is rarely identical to the workflow running in November, and nobody tells the model.
A live intake process, because once one automation works, requests arrive from everywhere. Route them through the same three-signals screen and the same shortlist discipline instead of letting the loudest department jump the queue. A program that automates whatever gets requested turns into a pile of half-maintained scripts within two years. We have seen the pile.
A budget line with a name. Scattered pilot spending eventually needs a home in the plan, and we made the full argument in the 2027 budget category piece: a named category with an owner survives budget season, while orphan line items get harvested.
A standing exam posture, meaning the examiner file stays current as a habit rather than a scramble. Call it ten minutes per quarter per automation. Cheap insurance against an expensive afternoon.
What month twelve looks like
It helps to set expectations honestly, because a year of good work produces a modest-looking picture: three or four automations in production with numbers a director can recite, one kill documented without drama, a playbook on version three, and an intake list longer than the build capacity, screened and ranked. The ops leader answers the board’s inevitable AI question with figures instead of adjectives. That picture will not impress a conference audience. It will, however, put you ahead of most of the industry, and the evidence in our case study review keeps pointing the same way: what separated the institutions that made automation stick was the repeatable machinery behind the deployments rather than any single deployment.
More screening and scoping material lives in the back-office automation pillar.
If your pilot just cleared its decision meeting and you want a second set of eyes on the scale-up plan, or on the playbook draft, Advisor Labs runs a 45-minute back-office AI audit: book a conversation.
Get one substantive analysis like this every Tuesday: subscribe to the AiForCU newsletter.