Part 2 - Are Your Improvements Working? How to Know for Sure – defining success for your initiatives
This is part 2 of our guide in defining success for your improvement program.
In part 1 – we covered ‘defining success for your program’. That post defined the key aspects used to define the success of your overall program and ensure you know, as business transformation leads, continuous improvement leads, or Chief Transformation Officers, if your efforts are paying off.
In part 2, we want to focus again on defining success but this time at the micro level – for individual initiatives in your improvement program. How do you define success for each initiative ?
The short answer is to define:
- Impact KPIs
- Process KPIs
- Risk KPIs
Read on for more details!
Impact KPIs – what are you trying to achieve ?
The overall measure of success for your initiative needs to be defined by an ‘impact’ KPI. This KPI should be directly linked to the overall program goals (e.g., reduce cost, improve quality, increase customer satisfaction) and tell you:
- What the expected business benefits of the initiative are
- If your initiative delivered the expected benefits – or not…
Knowing the expected benefits is useful not just to ‘keep score’ but also to know how much effort can/should be devoted to the initiative – would additional resources or investments be justified ?
If the impact KPI is not linked to the program’s goals, then you and any key sponsors would be right in questioning whether this was actually a priority right now and if the initiative should potentially be deferred for now whilst you concentrate on the program’s objectives.
Process KPIs – early indicators of success…
Whilst the impact KPI target gives you a great view of what you’re trying to achieve – it is, by its nature, an ‘output’ KPI – with a defined benefit directly linked to the program’s improvement goals. The impact KPI will tell you if your initiative worked and delivered the expected benefits but if it didn’t work it won’t tell you the reason why…
Additionally, as an ‘output’ KPI, the impact KPI will not tell you if the initiative is not working until after the work has been completed. ‘Output’ KPIs are also known as ‘lagging’ KPIs for this reason. A good analogy is when driving a car and looking in the rear mirror – its great to see where you’ve been but its not going to help going forwards. So with an ‘impact’ KPI, not only do you not know why the initiative failed but you don’t know until after the event – wonderful…
To help ‘drive’ you forward, you need process KPIs – also known as ‘leading’ or ‘input’ KPIs.
Process KPIs are the critical KPIs usually associated with the physical activities that deliver the result. These process KPIs are the ‘inputs’ to delivering an outcome. If the process KPIs improve – then they will generate an improvement in the output – all other things being equal. Tracking the process KPIs is like looking forwards when driving – now you’re getting real time feedback on if you’re driving in the right direction!
Lets run through an example to make this a little more concrete and less abstract. Lets assume, you have a printing machine and the initiative goal is to increase the daily output of the machine from 10,000 units/day to 12,000 units/day – a worthwhile 20% jump. You’ve agreed with your team that the overall measure of success is the increase in units/day production but after implementing your initiative you’ve not seen an increase and you don’t know why not ☹.
You need the process KPIs to understand exactly why not.
High level process KPIs might be Availability, Productivity and Quality and that would immediately give you some more insights as to what is and is not working. If your initiative is actually targeting to improve line changeovers then the process KPI might be to track the line changeover times. Further detailed process KPIs might be to track changeover times by crew or by product changes (eg from product A to product B). Now you would really understand what is happening – what’s working and what’s not working.
Risk KPIs – what could possibly go wrong ?
Whilst impact and process KPIs tell you if you’re successful or not in reaching your business objectives – they don’t tell the whole story.
If you’re implementing something that incurs some business risk (or even safety risk) – then you need to keep track of appropriate ‘risk’ KPIs (also known as ‘adverse’ KPIs).
These are the KPIs to ensure that you’re not adversely impacting someone or something as a result of implementing your initiative. Ignoring the risk KPIs can lead you into a position where you think you’ve successful implemented an initiative and delivered a successful outcome and seen the lift in your output KPI – but have failed to realize that you’ve actually badly impacted the business in some way and actually taken a step backwards…
For example, for a cost cutting initiative you might want to track output quality, or on-going customer satisfaction – so that you don’t unknowingly impact quality or customer satisfaction during implementation.
You need to ensure you have an accurate baseline for your risk KPI before you start implementation (and know the ongoing variance usually seen).
During implementation, if you see an adverse change in the risk KPI you need to quickly take appropriate action by first understanding if the change is linked to the implementation and, if so, what’s causing the change and what can be done to correct this.
Ultimately, you may have to wind back the implementation and rethink the approach.
If you do think there is business risk in your implementation – you should also consider conducting trials before a full roll out so that you’re constraining the risk to selected areas (for a limited time).
KPI tracking and reporting – technology
Bringing these KPIs together – you now have at least two or more KPIs to track including:
- An impact KPI
- One or more process KPIs
- Potentially a risk KPI (if you’re initiative is likely to incur some business risk)
During a baseline period, throughout implementation and for sometime after this (more on this next section…) you want to actively track and report these KPIs.
This is where technology should be your friend – your business improvement processes and systems should be doing the heavy lifting for you in reporting these KPIs.
All the key stakeholders involved in this initiative need to see the KPIs online whenever they wish – and they need to instantly see:
- each KPI has a well understood baseline
- that the KPIs are being tracked on a regular basis (e.g., weekly) and
- are heading in the right direction.
Any initiative progress reviews should be using this data as context for understanding progress and any required actions.
Relying on the initiative owner to emailing KPI spreadsheets or updates each week can work but it will be painful and inevitably fall over as being too hard to do. Instead, just make sure you have an online system able to keep track of your initiative KPIs – its simpler and everyone will be looking at the same data.
Long term sustainability – tracking KPIs
As mentioned earlier, its important to keep tracking KPIs for some time after implementing the initiative. If the initiative relies on any process or behaviour change then you’ll need to track the KPIs for some time after completing the initiative.
There is no solid rule on how long you need to continue tracking but the objective is to continue tracking until you are confident that the results are sustainable.
A major issue when implementing any initiative that involves process or behaviour change is that people naturally revert back to the ‘old way’ of doing things. There might be multiple reasons for this:
- teams never received adequate briefing/training on the ‘new way’
- the ‘old way’ is easier
- new starters aren’t trained on the ‘new way’
- no-one is checking if the new process is being followed (more on this later)
This natural reversion to the ‘old way’ of doing something is why these types of initiatives are harder to implement. Procurement initiatives or engineered/capex initiatives (e.g., updated equipment) are easier to sustain performance uplift and carry less risk of reverting and losing the initial performance uplift.
So keep tracking the KPIs until you’re confident that the results are actually being sustained.
Changing behaviours
If the KPIs are dropping away, for whatever reason, then the initiative owner needs to take appropriate action.
There could be many underlying reasons for the change in KPIs and the initiative owner will need to investigate to fully understand the root cause reason(s) and take appropriate actions.
Potential reasons might include:
- Something new has come up; the initiative worked and delivered but now a new issue has occurred
- Not everyone is following the new way of completing the process
- Not everyone was trained
Whilst investigating the reasons for the performance drop-off and the corrective actions needed might extend the time taken to complete the initiative, and include many unplanned steps that were never identified in the original initiative plan – these are the critical actions needed to deliver success.
Longer term, you can/should implement a system of process confirmations to ensure that the ‘new way’ of completing the process is actually being followed.
Process confirmations are a fairly structured review of the way the work is being done. They not only identify if the process is not being followed as expected but provide a great coaching opportunity with the frontline. The reviewer, the person completing the process observation, should ensure the operator is aware ahead of time of the process confirmation and the process confirmation will be done.
The process confirmation identifies if an operator understands the new process (and knows where the documentation is), observes the operator completing the process, confirms if the operator is following the steps defined in the documentation and, importantly, talks through the observation and findings in a discussion between the reviewer and the operator.
Learning should be two way and also a good opportunity for the reviewer to understand additional opportunities for improving the process.
Process confirmations are a whole subject in itself – and the subject of a longer post also available to you online.
Conclusion
Defining measures of success (impact, process and risk KPIs) will ensure that you know what success looks like and will help drive initiative implementation towards a successfully outcome.
The KPIs indicate if the initiative is working and will give you the direction to drive towards success but remember that they need to be supported by the initiative owner, sponsor and other key stakeholders physically changing whatever needs to be changed.
In summary the KPIs are to be used as context in your regular process reviews defining if the initiative is trending in the right way but if they’re not trending in the right direction you will need to agree on what are the appropriate actions needed and actually complete those actions to deliver the expected, sustained performance uplift.