Testimonials

Tracking the Impact of Improvement Activity

Closing the circle in the ROI (Return on Investment) improvement cycle

As with most things in Business, improvement activity is usually considered to be part of a ROI cycle. The organisation spends money training and supporting change agents (Belts in the Lean Six Sigma model) and then uses them to undertake improvement activity. This activity is expected to create value for shareholders, customers and employees; but whereas it is quite straightforward to track the impact of spending on the P&L account, it is not so straightforward to track the results of the improvement activity, and these often get neglected.

Without results tracking it is not possible to close the circle in the ROI improvement cycle, however, and the organisation may therefore feel that it has not invested its hard earned cash wisely in Lean Six Sigma training and support. This would indeed be a shame if in fact there was a return.

What makes it even more difficult is that improvement activity is also expected to impact on customer satisfaction and organisational culture, and these are notoriously difficult to track.

This article will lay out some guidance on how to go about tracking the impact of improvement activity to help close this cycle.

Tracking Financial Impacts

We should start with financial impacts. Lean Six Sigma projects are usually focussed on process improvements and progress is usually measured in terms of process metrics such as scrap, rework and lead-time reductions. Projects may also focus on improved OTD (On Time Delivery) metrics. How can these best be tracked?

Scrap reduction projects usually result in labour and material savings, as illustrated below:

Scrap savings = Scrap Rate1 * Quantity Shipped * (Labour Cost + Material Cost)

1 (as a proportion of quantity shipped)

Rework reduction projects usually result in labour savings as illustrated below:

Rework Cost = Rework Rate1 * Quantity Shipped * Labour Cost

There may also be material savings associated with rework reductions as repaired parts may well need parts replacement.

Lead-time is interesting as it is quite difficult to directly equate lead-time reduction to cost savings. In fact it is more likely to impact the Balance Sheet via inventory reductions. These are usually accounted for as one off impacts at the weighted average cost of capital for the Business.

Inventory reductions may also result in direct cost savings as well, however, as excess inventory typically consumes expense money to store, count and generally manage it. This can cost as much as 20% of the inventory value over a typical year.

Lead-time can be impacted by Lean initiatives tackling the constraint or Six Sigma projects increasing 'right first time' rates (and consequentially reducing lead-times). The impact of such improvements can be estimated as below:

Overall Cycle time = (%RFT * RFT cycle time) + (%Reworked*Rework cycle time)

Poor OTD performance may result in liquidated damage charges which directly impact the P&L account, they may also lead to excess inventories as a lack of delivery confidence tends to lead to a 'just in case' mentally within an organisation.

Tracking these benefits over time is also important, and although only standard project tracking techniques are required. It is surprising how many organisations new to structured improvement activity fail to do this adequately. It is important to track the benefits against plan and to ensure that they are fully integrated into the organisation's budgetary process.

A typical benefit tracking sheet is illustrated below:

LSS Benefit Tracking

Please click to ENLARGE

LSS Benefit Tracking

Tracking Customer Impacts

Customer behavioural impacts can often be tracked by means of spending patterns. It is sometime possible to track increases in revenue or market share as a measure of customer impact but it is usually difficult, if not impossible, to control for extraneous market factors such as macroeconomic change or competitor activity. For this reason most Lean Six Sigma Belts avoid these as means of judging customer impact.

Projects aimed at reducing customer complaints can easily be tracked by means of counts, proportions or values, but it is sometimes useful to track the impact on customer attitudes or opinions, and this is probably best tracked by means of surveys. These have the advantage of direct comparison of replicated questionnaires before and after an improvement initiative, and if sufficient customers are surveyed then hypothesis testing tools can be utilised to check for statistical significance.

In the example below customers have been asked to rate the quality of service in a call centre using a ten point Likert scale before and after a Lean Six Sigma project. Only 50 customers were sampled in each case, and although only a one point increase in the median score was observed it is still enough to prove statistically significant (p = 0.036) using a one tailed Mann-Whitney Test:

Individual Value Plot of Before, After

Tracking Cultural Impacts

If you think tracking customer impact is difficult then tracking cultural impact in an objective way is near to impossible. Businesses usually resort, therefore, to a mixture of employee survey responses, maturity assessments and Business KPI's linked to employee behaviour.

Many organisations run regular employee satisfaction surveys and questions relating to attitudes to change, improvement and customer centricity can be used to track change over time in a similar way to how customer attitudes are monitored and tested for.

Maturity assessment is usually carried by means of audit or questionnaire. They seek to place an organisation on a graduated scale designed to show where an organisation is on its journey to excellence. Some are quite complex and all embracing (such as the EFQM Excellence Model) others are more focussed on improvement, such as the one below:

Tracking Cultural Impacts

Please click to ENLARGE

Tracking KPI's linked to employee behaviour is relatively straightforward inasmuch as improving trends and ability to hit targets can be objectively assessed, whether these metrics actually measure culture is another matter. KPI's utilised for this are often:

- Project completion rates

- Project cycle times

- Benefits per project

- Belt or participant rates

These are often combined with process or business performance metrics, which should indicate that the culture is supportive of improvement.

It can be seen above that although some effort is needed in tracking the impact of improvement activity in an organisation such efforts are well worth making. On a project by project scale it is important to know that projects have been worthwhile and had some positive impact or shareholder, customer or employee value. From a Deployment perspective it is perhaps even more important, because the Leader or Leaders actually making the investment decisions need to be able to judge that s/he has made a wise investment.

As stated in the introduction it is easy to count the costs - and believe me organisations always do. It is not so easy to count the benefits, however, and a failure to do so can easily lead to an organisation falsely concluding that an investment in structured improvement is not a wise one.

In this article I have sought to impress the importance of tracking the impact of improvement activity within an organisation so as to help close the ROI cycle and give the Leadership some reassurance that it is investing its money wisely in the necessary training and support of improvement agents (Belts in the Lean Six Sigma model). I have also sought to give some guidance about how to go about tracking these benefits from a financial, customer and cultural perspective.

These techniques are taught to our Champions and Belts in our public courses advertised on this website.

Author Name - Mike Titchen (MBB, SigmaPro)

Mike is a highly experienced MBB and has worked in both manufacturing and financial services industries. Mike joined SigmaPro as a Master Black Belt and lead trainer in 2006. His consulting and training experience has involved the development and delivery of training materials for Six Sigma programmes for several high profile clients, and Mike has helped clients deliver improvement savings totaling over £5M.

Before he joined SigmaPro, he has implemented and has been responsible for Six Sigma with Norwich Union, Textron and General Electric. Mike has an MBA and a BA (Arts), and away from work he enjoys golf, tennis, snooker and listening to music.