How to Review and Scale IRIS After the First 90 Days
DBR77 Team7 min read
On this page
After 90 days, review IRIS on six things: whether people use it on every shift, whether the data is trusted, how fast issues close, whether the target loss moved, how people treat AI recommendations, and what the floor thinks of it. Then make one of three decisions: scale, fix first, or stop. Before you scale, turn the pilot setup into a template (asset names, stop reasons, roles, rules and approval gates) so each new line starts from what already works.
Ninety days is long enough to see habits form and short enough to fix what is wrong before it spreads.
Why the 90-day point matters
A pilot that works on one line is not yet a system for the plant. The World Economic Forum's 2023 paper on its Global Lighthouse Network notes that even these leading sites, which no longer suffer from pilot purgatory, are working to overcome a "scaling slump" as they try to extend their impact across whole production networks.
The 90-day review is the Check step in PDCA. The Lean Enterprise Institute describes it as evaluating the results in terms of performance, followed by Act: standardize and stabilize the change, or begin the cycle again. Without the check, the plant copies its mistakes to the next line along with what worked.
- Day and late shifts
- 96%
- Night shift
- 71%
- Day shift
- 12 min
- Night shift
- 31 min
After 90 days, line 2 records 96% of stops with a reason on day and late shifts.
The review: six areas and the evidence
Bring data, most of it straight from the system.
| Area | Evidence to bring | Healthy sign | Warning sign |
|---|---|---|---|
| Use | Share of orders, stops and work orders recorded, by shift | Similar on all shifts | Night shift far below day shift |
| Data quality | Share of downtime with a known reason, counts against real output | "Unknown" small and stable | Reasons added every week |
| Loop speed | Time from signal to response, from task to closure | Falling, then stable | Many tasks closed late or as "no action" |
| Target loss | The one loss the pilot was chosen for, against the baseline | Moved in the right direction | Flat, or moved for unrelated reasons |
| AI recommendations | Accept, change and reject rates, override reasons | Rejections have reasons | Everything accepted without review, or ignored |
| People | Short interviews with operators, setters, shift leads | "It saves me time" | "It is extra work on top of the old forms" |
The people row counts as evidence: a system that the night shift sees as extra work will fade on the next line too.
Reviewing the AI on its own
AI needs its own review, because a model that worked in month one can drift. NIST's AI Risk Management Framework says AI systems should be tested before their deployment and regularly while in operation, and notes that they may need more frequent maintenance because of data, model or concept drift.
Look at four things:
- Accept rate by type. A type of recommendation that is rejected most of the time needs a new rule or should be switched off.
- Override reasons. Read a sample. They often show missing data, such as a changeover the model did not know about.
- Blind acceptance. If one person approves everything in seconds, the approval gate is not doing its job.
- Gates. Decide per type of recommendation whether to keep approval as it is, tighten it, or allow more suggestions to go straight to a task list. Keep a person in the loop for anything that touches safety or product release.
Scale, fix or stop
Write the decision down, with the evidence behind it.
- Scale when use is even across shifts, data quality is stable, the target loss moved, and AI recommendations are reviewed rather than rubber-stamped.
- Fix first when results are good on some shifts and weak on others, or when data quality is still moving. Fix it on the pilot line, then review again in 30 days.
- Stop or rethink when the target loss did not move and the floor sees the system as extra work. Find out why before spending more.
Illustrative example: after 90 days, line 2 records 96% of stops with a reason on day and late shifts, but only 71% at night, where there is no super user. Average response to a stop is 12 minutes on day shift and 31 at night. AI suggestions for work order priority are accepted about 60% of the time, and most rejections cite planned changeovers. The decision: fix first. The plant names a night super user, adds the changeover plan to the data the model reads, reviews again in 30 days, then copies the setup to lines 1 and 4.
How to copy a working setup to more lines and plants
Scale by copying a template from the pilot line.
- Write the template: asset naming, stop reason list, roles, task rules, escalation ladders, approval gates and a short training pack.
- List what may differ locally, for example shift patterns or line-specific checks. Everything else stays the same.
- Name one template owner. Changes go through that person, so line 6 does not invent its own reason codes.
- Scale inside the plant first. Same people, same culture, shorter feedback.
- Then add a second plant, with one person from the first site on the go-live team.
Larger changes to layout or capacity can be tested first; How to Turn a Digital Twin Into a Repeatable Decision Process shows how. For the machine data side of the same idea, see Turning IoT into a Repeatable Operating System.
How this works in DBR77 IRIS
DBR77 IRIS gives the review its evidence without a separate project. IRIS KPI compares performance across shifts, lines, plants and time periods, with drill-down to machine level. IRIS Gemba tracks open and closed issues by tier and area. Data & AI keeps each recommendation with the action a person took on it and the outcome of those that were applied, so accept and reject rates come from the record.
Scaling is module by module and site by site. Modules can be added at any time, and data and configurations carry over, so the pilot setup is the starting point for the next line. Changes can be tested first in DBR77 DT. How to Roll Out IRIS Module by Module Without Disrupting the Plant describes how the first waves are planned.
FAQ
Is 90 days long enough?
For a first decision, yes. It covers enough shift cycles to see habits form. Some losses take longer to move, so keep reviewing monthly.
Who should be in the review?
The plant director, the project manager, one shift lead per shift, maintenance and quality leads, and IT. Keep it to two hours.
What if results are mixed?
That is the most common case. Choose "fix first", name what must change, and set a date for the next review.
When should a second plant start?
When the template has been copied to at least one more line in the first plant without major changes.
Conclusion
At 90 days, bring the evidence for use, data quality, loop speed, the target loss, the AI and the people, and write down one decision: scale, fix first, or stop. Before you scale, turn what worked into a template with one owner and copy it to another line in the same plant. Start a second plant only when that copy needed no major changes.
Sources
- World Economic Forum, Manufacturing Lighthouses and the Path to Impact: Adopting AI at Speed and Scale, 2023
- Lean Enterprise Institute, PDCA
- NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), 2023