Part 1 - Are Your Improvements Working? How to Know for Sure – defining success for your program
Imagine setting out on a cross-country road trip without a map or GPS. You might start with enthusiasm, but soon, uncertainty creeps in. Are you headed in the right direction? Are you making good time, or are you just driving in circles? Delivering improvement can make you feel similarly unsure of whether you are actually heading in the right direction.
For business transformation leads, continuous improvement leads, and Chief Transformation Officers, this uncertainty can hit close to home. You’re tasked with steering an entire organization toward a better future, but how do you know if your efforts are paying off?
The truth is, what gets measured gets managed. But identifying the right metrics and keeping track of meaningful progress can feel daunting. This guide is written in two parts and together are your map—your guide to ensuring the program and every initiative in the program is on track and delivering the value you’ve promised.
- Part 1: Defining success for your program
- Part 2: Defining success for each initiative
Part 1 is focused on the defining the right measures of success for your overall improvement program.
Defining overall success in Improvement
Let’s begin with the foundation: What does overall success look like?
The definition will vary widely by organization and also whether you’re undertaking a one-off large transformational improvement program or instead a continuous improvement program:
- In a transformation, success might simply be defined as “higher profits” or “happier customers.” The objectives should relate directly to the organization’s strategic imperatives set by the senior leadership team. The transformation goals are likely to be one-off goals set for the lifetime of the transformation e.g., 12-18 months.
- In continuous improvement, success might be a little more nuanced in that not only will there be an organizational improvement goal (e.g., higher profits or happier customers) but also additional people/process goals around the process of continuous improvement itself (e.g., workforce engagement, number of initiatives running, adoption rate) enabling the organization to understand if the continuous improvement process itself has been adopted.
Success in improvement (either a transformation or continuous improvement) begins with clarity. It’s about aligning your improvement program objectives with your organization’s broader goals. The overall measure of success should fit with your organization’s strategic objectives – otherwise you need to question yourself as to why doing it.
The overall measure of success should succinctly define the expected business performance uplift and be the overall justification for the program’s business case. In other words, the defined uplift will justify any costs, investments and resources needed to deliver the program.
If an organization simply states the goal as “improve operational efficiency.” —it doesn’t convey what that actually means? Shorter production cycles? Lower defect rates? Reduced waste? Without clarity, the organization will struggle to align their effort and risk the improvement program stalling.
Here’s how you can do it:
- Set SMART Goals: Specific, Measurable, Achievable, Relevant, Time-bound. A vague goal like “improve productivity” becomes actionable when rephrased as “reduce cycle time by 15% within six months.”
- Differentiate Short-Term Wins from Long-Term Impact: Short-term wins, like cutting costs, should build momentum, but they shouldn’t come at the expense of long-term sustainability.
- Ensure Cross-Functional Alignment: Success doesn’t happen in silos. Every department must understand how their efforts contribute to the bigger picture.
Defining an overall measure of success is step one.
Defining additional supporting measures of success
Whilst the overall measure of success is critical to measure the success of the program and to communicate progress to all stakeholders – its not enough on its own to track the performance delivery of your improvement program.
Key issues of the overall measure of success are:
- It’s probably insufficient on its own without support from other sub-measures of success and it’s an output lagging KPI which doesn’t give you instant feedback on whether you’re doing the right things that will lead to success
- It doesn’t tell inform you anything about potentially adverse impacts
- It doesn’t reflect the people side of the transformation
You need more measures of success! And you also need some framework for understanding the relevance or importance of these additional measures and why/when you need to use them.
Lets run though each of these in detail.
Sub-measures of success
As mentioned, the overall measure usually needs to be supplemented with additional sub-measures for two reasons:
- It may not give enough detail or granularity on the uplift expected
- It’s an output lagging KPI and won’t give you immediate feedback on the success of your program (and if you’re doing the right things to deliver success)
Firstly, the overall measure usually needs additional supporting measures.
As an example, a transformation I supported in the resource sector used a target unit cost ($/tonne) as the overall measure of success. This made perfect sense in the resource sector where an organization’s overall financial performance and health is heavily dependant on where they sit on the cost curve.
The unit cost measure was itself broken down into two sub-measures
- a planned total output goal (tonnes)
- a planned total annual cost ($)
Each of these measures were themselves broken down into more detail – by site and function. This allowed the improvement program to know if they were on track and also, critically, where the program was not on track (i.e., site and/or function).
Secondly, the overall measure of success is a high-level measure aligned with the organization’s strategic goals. By its very nature, its an ‘output’ (or ‘lagging’) KPI or measure used to measure overall success and justify the program’s costs. It takes time for the results to come in before they can be consolidated into the final ‘output’ measure. It doesn’t tell you if you’re not on track until after the event.
For example for a large transformation, it might be months before your see a change in the output results. You will need time to define the program, mobilise resources, collect improvement ideas, define priority initiatives, plan implementation and then complete those priority initiatives.
You need to supplement your ‘output’ measures with some ‘input’ or ‘leading’ measures – which will tell you ahead of time if you’re on track for delivering success. You can react to these input measures in ‘real time’ and make program changes needed immediately.
You might consider:
- Program engagement (number of employees, communications)
- Number and value of improvement ideas collected by area
- Time taken to complete implementation planning
- Implementation milestones on track vs missed
You’re looking for leading indicators to measure success way before you’re expecting the final output results to be delivered and these indicators should tell you as early as possible exactly where you might be seeing issues and what to do about them.
Adverse impact measures (risk)
Whilst the overall measure and sub-measures give you good direction if your program is successful and heading in the right direction, they give no indication of any major program risks.
For example if you’re undertaking a cost reduction program, are you still able to output your product or service at the same quality ? Are your customers still satisfied with the product or service ?
For an improvement program, whether Continuous Improvement or a large Transformation, you need to ensure you’re not ‘adversely’ impacting your business. Otherwise, you run the risk of successfully reducing cost or increasing output as per your objectives but you’ve also incurred some adverse impacts. Some of these impacts might be subtle and it could be a while before they’re recognized.
You need to establish up front what risk measures you will track and ensure you have a good baseline in place before commencing.
People measures
The measures discussed so far really relate to business performance (either success or risk) and don’t factor in the people side.
Some improvement programs will naturally impact the people side more then others (think cost reduction).
As an example, the success of a merger between two financial services companies that I managed some years ago was measured overall by the $ and customers retained (before vs after merger). However, we also wanted to ensure that the people side of both companies was not severely impacted and that the merged entity could keep operating on a ‘business-as-usual’ basis through the immediate ‘Day 1’ months of the merger. Therefore, we tracked people attrition rate during the planning and eventual merger and also tracked if all key executives were retained and reported on both these measures throughout the merger.
These example people measures are ‘hard’, quantitative measures but you might also want to consider softer people measures such as ‘employee engagement’ and ‘employee satisfaction’ through surveys.
Reviewing measures of success
Once you have your key measures of success defined, you can use them as context to help drive your improvement program.
They should be used as the basis of your regular progress reviews (along with progress to plan/gantt information) and allow you to structure your reviews:
- Overall measure of success: Are we hitting the expected overall program goal ? Are we moving quickly enough ? (your overall measure should be timebound)
- Sub-measures of success: Are there areas of concern ? What are the actions needed ?
- Adverse/risk measures: Have we adversely impacted the business in any way ? What do we need to do to correct this ?
- People measures: Is everyone still behind the improvement program ? Will we be able to sustain our business long term ? Are actions needed ?
Note that the measures are simply an input into the progress reviews. The key output from the reviews are the actions needed to maintain progress and correct for any deviations from the expected plan.
Communicating progress
Now you fully understand the complete set of measures used to define success in an improvement program, it should be much clearer what measures should be communicated and when.
The overall measure of success is the key measure to be continuously communicated – both the baseline and target in the early days to ensure everyone understands the program’s objectives and purpose and later when the program starts to deliver results to see how the actual performance is doing against target to ensure the full uplift is achieved.
Referring to the earlier resources example given earlier, the continual tracking and reporting on progress towards the unit cost objective helped simplify and focus all the communications of what was a complex transformation involving hundreds of initiatives across the entire organization and ultimately helped to deliver success.
The full set of measures being used to measure success for your program, should ideally be available online, instantly to everyone involved in the program. This ensures a ‘single source of truth’ and full transparency on progress and will help you hit the program’s objectives.
Conclusion
This post covers in detail the key measures of success that are needed for any improvement program (major transformation or continuous improvement).
As discussed there are varying measures used:
- Overall measure of success that aligns with the organization’s strategic goals
- Sub-measures with more detail and adding a mix of leading KPIs to the output overall measure
- Adverse/risk KPIs to track and report potential business risks
- People KPIs to ensure long term sustainability of your business
They’re all needed in some form. They all have a purpose and the key is to use them correctly in the right way at the right time.
Read part 2 to understand how to define measures of success at the initiative level.