Training Is Not Adoption: Build a Simple Change Scorecard

The new process has launched. Most people attended training. The slides were sent, the recording is available and the project plan shows the activity as complete.

But staff are still using the old spreadsheet.

This is the gap between delivery and adoption. Training is important, but it is only one input. A change has taken hold when people can perform the new task, use it in real work and achieve the intended outcome without creating harmful workarounds.

A simple change scorecard makes that visible.

Why this matters

Projects are often measured by outputs: system configured, communications sent, training delivered and go-live completed. Those outputs show that the project did work. They do not prove that the work changed the service.

Output-focused goals can create a false sense of accomplishment. Good measurement connects a baseline, user behaviour, service outcomes and action on early trouble signals.

A change can look successful from the project room while struggling in daily operations. High training attendance may sit beside low usage. Strong initial login numbers may hide abandoned tasks. Faster processing for one group may create extra work for another.

The solution is not a huge dashboard. It is a small set of measures that connect support, behaviour, outcomes and barriers.

Use the ADOPT scorecard

ADOPT is a plain-language framework for measuring whether workplace change is taking hold:

• Audience reached: Did the people who need to change receive the right support?
• Demonstrated ability: Can they complete the new task correctly?
• Observable behaviour: Are they using the new process in real work?
• Performance outcome: Is the change improving the service or workflow?
• Trouble signals: Where are people struggling, avoiding the change or creating workarounds?

The five parts form a useful chain. People need to be reached, become able, change their behaviour, improve an outcome and receive help where trouble appears.

ADOPT is not an official government or professional-body framework. It is a practical way to organise measures drawn from established performance, service and benefits-management principles.

Step 1: define the behaviour before choosing metrics

Start with a sentence that describes what people will do differently.

Weak: "Staff will adopt the new portal."

Stronger: "From launch, staff will submit non-urgent equipment requests through the portal using the required location, priority and required-by fields."

The stronger statement identifies the group, action, channel and quality standard. It gives the team something observable to measure.

Also name the intended benefit. Perhaps the portal should reduce missing information, shorten triage time or give requestors clearer status updates. If the behaviour cannot reasonably lead to the benefit, revisit the change logic.

Step 2: establish a baseline

A baseline is the starting point used for comparison. Before rollout, measure the old process where possible. How many requests arrive through each channel? How often is information missing? How long does triage take? How many follow-up contacts occur?

The baseline does not need to be perfect, but its limitations should be stated. If reliable data does not exist, use a short sampling period, a manual count or an agreed estimate while improving measurement.

Do not rewrite the baseline after results arrive. That turns measurement into storytelling.

Step 3: choose one or two measures for each ADOPT area

Audience reached

Training attendance can sit here, but it should show whether the right people were reached, not merely the total number of attendees. Consider affected people briefed, attendance by role or work pattern, access to job aids and people who joined after launch.

Demonstrated ability

Ask whether people can perform the changed task, not whether they watched the explanation. A realistic practice task, a short checklist, evidence of correct performance and time taken during practice are often more useful than a quiz about policy wording.

Observable behaviour

Behaviour is the centre of adoption. Use operational evidence showing whether the new way is being used: the percentage of eligible work completed through the new process, use of required fields, continued use of the old channel and repeat use after the first attempt.

Measure behaviour at a team or process level unless individual detail is genuinely necessary and appropriately governed.

Performance outcome

An adopted process can still be a poor process. Check whether it delivers the intended benefit through completion rate, cycle time, avoidable rework, service-user effort, staff effort or delivery of the intended project benefit.

Numbers explain what is happening; conversations and observation can help explain why.

Trouble signals

Trouble measures are early warnings. Look for help-desk contacts, abandoned tasks, manual workarounds, duplicate records, exception requests, complaints or uneven results across groups.

A rise in support contacts is not automatically failure. It may be expected during early adoption. The useful question is whether the signal is reducing, stable or revealing a design problem.

Step 4: make the scorecard decision-ready

For every measure, record:

• Measure
• Baseline
• Current result
• Direction of travel
• Target or acceptable range
• Owner
• Next action
• Review date

A number without interpretation invites debate. A decision-ready entry might say: "Portal usage is 56%, below the 75% first-month range; two teams account for most email requests; local coaching and a form change are due Friday." That supports action.

Use trends rather than single points. Segment results where it helps the team understand different experiences, but protect privacy and avoid unnecessary monitoring of individuals.

Step 5: agree the review rhythm

Early after launch, review weekly or fortnightly. As the change stabilises, move to a monthly or normal operational cadence.

Keep the discussion short:

1. What changed since the last review?
2. Which group or step is struggling?
3. What evidence explains the pattern?
4. What action will we take?
5. When will we check whether it worked?

The scorecard belongs with the operational owner, not only the project manager. The project team may establish it, but the business area must be able to maintain it after handover.

A realistic workplace example

A fictional health-service operations team launches a portal for non-clinical equipment requests. The project reports that 94% of staff completed training. Leaders initially treat this as strong adoption.

The ADOPT scorecard shows a different picture:

• Audience reached: 94% of affected staff completed the briefing, but evening-shift attendance was lower.
• Demonstrated ability: 78% completed a sample request correctly; most errors involved the priority field.
• Observable behaviour: only 56% of eligible requests used the portal; email remained common in two teams.
• Performance outcome: correctly submitted requests moved faster, but incomplete submissions created follow-up work.
• Trouble signals: support questions clustered around priority definitions and supervisor approval.

The team does not schedule another organisation-wide presentation. It adds plain-language examples beside the priority field, runs short practice sessions for the two teams still using email and asks supervisors to reinforce the correct channel.

Two weeks later, the team reviews portal use, incomplete submissions, follow-up contacts and staff feedback. It keeps the useful changes, adjusts the guidance and continues monitoring until the new behaviour is stable.

Training was part of the solution. The scorecard showed where training alone was not enough.

Common mistakes and risks

• Counting attendance as adoption. Attendance proves exposure, not ability, behaviour or benefit.
• Starting without a baseline. Without the old result, a new number has little context.
• Choosing only easy measures. Completion certificates may miss whether work changed.
• Tracking too many indicators. A crowded dashboard hides the signals that should trigger action.
• Reporting only the average. Role, location, accessibility need or work pattern may reveal a group being left behind.
• Using measures to punish people. Adoption data should uncover barriers and improve support.
• Ignoring privacy and proportionality. Collect only what is needed and avoid unnecessary individual surveillance.
• Stopping at go-live. Behaviour and benefits often need monitoring after the project team steps back.

Try this this week

Choose one change that is live or approaching go-live. In 30 minutes:

1. Name one behaviour that should be different.
2. Write the current baseline, even if it is an estimate that must be improved.
3. Choose one measure for ability, behaviour, outcome and trouble.
4. Assign an owner and review date.
5. Decide in advance what result would trigger support, redesign or escalation.

Keep the first scorecard to one page. If a measure does not help someone decide what to do, remove or replace it.

Key takeaway

Training tells you who had an opportunity to learn. Adoption tells you whether people can work differently and whether the change produces a worthwhile result.

Measure the chain from reach to ability, behaviour, outcomes and trouble signals. Then use the evidence to improve the change, not merely report it.

For more practical guidance on project delivery, change management, responsible workplace AI and continuous improvement, connect with Raf on LinkedIn and read the full article on rafsanga.com once published.

AI tools assisted with research and drafting. The final article was reviewed and approved by the author.

Further reading

• Australian Government Digital Performance Standard
• Australian Government: Criterion 9 - Monitor your service
• Atlassian Team Playbook: Goals, Signals and Measures
• GOV.UK Service Manual: Measuring service success
• Project Management Institute: Benefits Realization Management Framework