Capability Levels .... - Omniprove
Main menu:
Capability Levels ....
Introduction Download
The ability to understand the performance of processes provides organisations with a key advantage in delivering stakeholder needs and customer satisfaction. This advantage can be further enhanced through not only understanding the performance, but being able to predict the outcome of the process as early as possible. With better prediction comes more reliable estimating and earlier corrective action if the process performance is not delivering what is expected. The degree to which a process is understood, managed and can be predicted is often
expressed in terms of process capability levels.
expressed in terms of process capability levels.The TickITplus scheme has adopted five process capability levels which have been taken from those defined within ISO 15504:2. This standard actually identifies six capability levels but the TickITplus scheme excludes level 0, Incomplete, although in practice this would be the case if processes failed to achieve even the Foundation level in TickITplus.
Whilst an assessment can be undertaken at any level, process improvement must progress systematically through the levels as the levels build upon each other. For example, if an organisation has identified that a particular process is currently at the Performed (Foundation) level and wishes to improve it to the Established (Silver) level it must first progress through, but not necessarily be assessed at, the Managed (Bronze) level. These models not only identify process capability but in a way identify organisational maturity – it’s often a cultural journey for the organisation.
The first level in ISO 15504:2 represents a process that isn't actually managing to achieve its purpose, which means that it is not causing the desired change of state, producing adequate output or even working within defined constraints. This level isn't formally defined in TickITplus although in practice, if a process fails to achieve the Foundation level it could be said to be at level 0.
When a process is at level 1, Performed (or Foundation level in TickITplus , it is capable of achieving its purpose in terms of effecting a desired change of state, creating required outputs and operating within defined constraints. However, how it is performed in terms of incurring costs, meeting schedules, effects on staff stress, level of issues and problems, etc. is very unpredictable and when it does work is very often unrepeatable. Organisations operating with processes at level 1 often involve heroes called in to 'get the job done' or last minute late night work with the infamous pizzas being brought in. The organisational desire is to primarily achieve the schedule while attempting to satisfy the requirements but often at the expense of costs, i.e. increasing resources, overtime, etc. to achieve the goal. The main component missing in these circumstances is the ability to fully manage the processes.
Processes operating at level 2, Managed (or Bronze level in TickITplus , are being actively managed which means their objectives are clear, they have been planned, are being monitored and, if necessary, adjusted or re-planned. They have appropriate non-human and human resources assigned with allocated roles and responsibilities and the stakeholders have been identified and their interaction with the processes defined. Additionally, the outputs (or work products) from the processes are now clearly specified, defined and controlled.
The objective for processes can vary depending on the nature of the process, although typically a process objective is specified in terms of an organisational policy or policies. For example, the security management process objectives could involve policies that require the protection of organisational assets at all times through defined and implemented controls. The objectives for the project management process could also be expressed through policies for ensuring that all projects have an allocated project manager responsible for ensuring that projects are planned, monitored and controlled throughout the life of the project and that resources are made available to ensure the successful delivery of the project. As the process matures through the capability levels, specific measurable objectives can also be added. Note that this doesn't preclude allocating specific measurable objectives to a process at any time, but at the lower levels (see above) the predictability of achieving these measurable objectives will be significantly lower and therefore it's probably less important to identify them. Furthermore, in some cases, it can actually be counterproductive to specify measurable objectives where the reason for them is not fully understood by the organisation.
Planning a process involves four key aspects and one highly desirable aspect. In essence if something is planned it means that a W3H principle has been followed. The first 'W' - What, must be known in order to undertake any sensible planning, i.e. what is it that is required. Planning then involves scheduling activities which is the second 'W' - When, and also the allocation of resources, especially human resources and this is the third 'W' - Who, which gives us the W3. The 'H' is the How and this is usually covered by identification of the approach to be taken which can be defined in procedures or specified as part of the planning for the activity (or process). The highly desirable element is another W, the Why, and this can greatly help people understand all the other elements of planning work.
Interestingly, there is no specific need to have defined procedures to support the activities at level 2 as the driver is typically through the process objectives, e.g. the policies, although, in reality, most organisations do see a need for defined procedures. Organisations running with level 2 processes will have strong polices which drive practice, but not necessarily consistent practice. The aim at level 2 is that the processes are managed, not that the whole organisation manages them in the same way. For example, given a good policy for project management as mentioned above, a project manager is free to implement the objective in whichever way is felt necessary. It could be said that a plan is a dynamic or living procedure. An approach to an activity can be described in a plan when the W3H elements are dynamic or specific or can be described in a procedure when they are static or general. For example, if configuration management (CM) is always undertake in the same way (How), by the same group of people (Who), at regular defined intervals (When) and to achieve the same aim (What) then it could just be simply defined in a procedure, e.g. the CM team supporting regular maintenance updates to a standard product. On the other hand, if CM needs to be undertaken specifically for a particular project using specific project tools and methods (How), involving specific named people (Who), at project defined periods (When) to achieve the desired project activities (What) then it really needs to be planned. Often however, the elements of W3H are mixed between procedures and planning.
While this planning approach should achieve the process purpose and should be repeatable in each particular case, there are a number of issues and potential problems, some of these being:
- Problems detected and fixed in one implementation of a process will often only benefit a narrow part of the organisation.
- Problems will be slower to detect as they will be dependent on the approach taken by individual staff implementing the processes.
- Moving staff around the organisation will be more difficult as they may have to learn different approaches in practice.
- The organisation can't gather significant data on how the processes are operating and even if they could, the measures would not be comparable as they would have resulted from different approaches.
- Some areas of the organisation may be operating very effectively while other areas could be struggling, yet there will be no common ground to understand the variances.
- Improvement of the processes will be based on, and benefit only, the individuals or small groups of people who perform the processes.
- Staff movements or changes can have a direct influence on the ability of the processes to achieve their purpose and consequently there is an increased risk of reverting back to level 1 processes.
So, for an organisation operating processes at capability level 2 there is usually a good level of individual repeatability and, even to a certain degree, individual predictability, but there are also risks of failure and missed opportunities. For an organisation that has progressed to operating level 3 processes the problems discussed above will, in the main, have been addressed. Organisations with level 3 processes have identified the 'best' processes throughout the organisation and will have formally defined these as the 'preferred' way of working. It is important to note that even in organisations with level 3 processes it may not always be possible do everything in the same way, but they will have a standard way of working along with clearly defined and approved rules for varying from the standard approach when necessary.
Given the ability to vary or, as it is referred to, tailor, the standard approach it could be said that an organisation at level 2 is simply using a standard approach where every process is actually tailored to individual needs. However, this is not the meaning or intent at level 3. Level 3 processes are fully defined and tailoring is ideally kept to a minimum and, when tailoring is necessary, the implications are reviewed and formally approved. It's
worth noting that tailoring does not mean taking something out or not doing something, it means doing it in a different or more appropriate manner. For example, it's not unheard of to see a system that allows 'small' projects to tailor-out the need to do some processes, such as risk management for example. But the management of risks is just as important to any size project or indeed for that matter any type of work. What might actually be meant is that the very formal approach to the way risk is managed on large complex projects may not necessarily be needed on small projects (although it might just be easier to do everything the same way in an organisation anyway). The management of risks should always be undertake, as should all other processes, but that the approach, method, tools, techniques and other operational factors need to be appropriately considered in how it is performed.
worth noting that tailoring does not mean taking something out or not doing something, it means doing it in a different or more appropriate manner. For example, it's not unheard of to see a system that allows 'small' projects to tailor-out the need to do some processes, such as risk management for example. But the management of risks is just as important to any size project or indeed for that matter any type of work. What might actually be meant is that the very formal approach to the way risk is managed on large complex projects may not necessarily be needed on small projects (although it might just be easier to do everything the same way in an organisation anyway). The management of risks should always be undertake, as should all other processes, but that the approach, method, tools, techniques and other operational factors need to be appropriately considered in how it is performed.In many cases it is typical to see organisations state that the tailoring of project work is achieved through the project plans but in practice this often means that the project manager has simply defined the way they want to work and often there is no formal analysis or understanding of the consequences or any formal approval by process owners to proceed in the stated way. But why is this important?
In one respect, tailoring everything every time doesn't actually demonstrate that there is a standard way of working and therefore the organisational process won't actually be operating at level 3 anyway, but more importantly, the organisation would never be able to move to the higher process capability levels. Indeed, this may not actually be a desire of the organisation but nevertheless many of the potential problems listed above would still exist.
Notwithstanding the above, an organisation that does demonstrate processes operating at level 3 will be able to:
- Detect process problems more quickly, because more uses of the process exist and the improvements will benefit more of the organisation
- In some cases prevent problems from happening in areas where the particular process problem hasn't been experienced yet.
- Move staff around more easily as the processes will typically be operated consistently, albeit with a few carefully considered and approved variances.
- Gather process related performance data from consistently operated processes.
- Distinguish where processes are ok but not being implemented correctly against those cases were the processes are not actually working effectively even though they are being consistently implemented.
- Call upon a wider group of organisational staff to discuss and improve the processes that the organisation operates.
- Significantly reduce the risk of process failure as the dependency on staff operating the processes is lower, i.e. be less dependent on particular staff who would now the good approach irrespective of the defined processes anyway.
So, given these benefits, why would organisations want to go any higher with process capability? The most significant facet of an organisation that operates level 3 processes is that, apart from consistency in implementation, they have the ability to understand what has happened and what is happening as a result of operating the processes – a sort of rear mirror view. The next step is to understand what might happen in the future, and operating level 3 processes provides the foundations for achieving this.
Organisations operating at capability level 3 have standard processes along with controlled tailoring of those processes and considerable process measures on how the processes are performing. It's often said that it isn’t possible to undertake measure on projects because every project is delivering different products. However, for organisations operating capability level 3 processes the measures are taken from the processes not the product, and, as discussed, the processes are consistently performed and the impact on the measures from the variances (tailoring) is fully understood.
However, having lots of measures from different projects, albeit using the same processes, still has one problem. While the process may be the same, the product (or service) that it is being delivered by the project will vary in size and therefore to be able to compare process measures it must also be possible to compare product (or 
service) size - this is known as normalising the measurement data. This isn't easy. For example, years ago, code size was often measured in Lines of Code (LOC) but there were problems. If an 'if-then-else' construct is placed on three separate lines is that three lines of code, do we count comments and blank lines, etc.? There was, and still is, a large effort placed on function point counting but this too has its problems and difficulties, not least of which was the effort and experience necessary to accurately calculate function points - it is a skill. These days, organisations have tried requirement counting, Use Case counting, story point counting etc., but in practice it doesn't really matter what is used as long as it is consistent. Having said that, sometimes it does matter what is used when productivity levels are being used to measure the efficiency of different organisations in supplier selection cases but this is not a primary element of TickITplus - but TickITplus can still help in these situations.
service) size - this is known as normalising the measurement data. This isn't easy. For example, years ago, code size was often measured in Lines of Code (LOC) but there were problems. If an 'if-then-else' construct is placed on three separate lines is that three lines of code, do we count comments and blank lines, etc.? There was, and still is, a large effort placed on function point counting but this too has its problems and difficulties, not least of which was the effort and experience necessary to accurately calculate function points - it is a skill. These days, organisations have tried requirement counting, Use Case counting, story point counting etc., but in practice it doesn't really matter what is used as long as it is consistent. Having said that, sometimes it does matter what is used when productivity levels are being used to measure the efficiency of different organisations in supplier selection cases but this is not a primary element of TickITplus - but TickITplus can still help in these situations.Estimating the project, product, development or service size is a key Base Practice in TickITplus and should have been fully considered and implemented at Level 1 (Foundation Level) for the reasons discussed above (see Base Practice 2 in the Maintenance Management and in the Requirements Analysis processes in the TickITplus Base Process Library)
So, given that an organisation operating capability level 3 processes with some form of sizing (or normalisation) factor, it should now be possible to use this measurement data in a statistical manner to predict the performance of processes. Using statistical techniques on all processes would not be practical or appropriate as the cost of implementing such techniques would be prohibitive and, actually, unnecessary anyway. What is important however, is to apply the statistical techniques to processes, or actually sub-processes, that are key, critical or are giving concern to the organisation. These usually can be identified from the business plan, i.e. where the organisation wants to go next, develop into or improve.
By having consistent comparable data it is possible to build process models based on the historical performance of the processes. For example, it would be possible to build process performance charts on how 'good' the testing process was in detecting defects, or how 'good' the estimating method was in predicting schedule accuracy or even predicting how many risks would turn into issues and hence how much additional work is required to address these issues.
These charts are often based on statistical process control (SPC) techniques involving mean and range. Given a set of measurement data from a consistent and normalised data source (i.e. the level 3 processes) it is possible
to establish the mean and range of the process performance. From this, it is possible to understand the variance and to equate this with normal distributions through standard deviations. The world, including organisations and their processes, usually operate in a normally distributed manner - the world varies! Given a normal distribution on the performance of processes it is then possible to plot new uses of the processes against the models and detect where processes are not performing normally. This can be done early in the use of a process and therefore corrective action can be taken earlier and also specific 'abnormal' values can be detected and analysed for their cause.
to establish the mean and range of the process performance. From this, it is possible to understand the variance and to equate this with normal distributions through standard deviations. The world, including organisations and their processes, usually operate in a normally distributed manner - the world varies! Given a normal distribution on the performance of processes it is then possible to plot new uses of the processes against the models and detect where processes are not performing normally. This can be done early in the use of a process and therefore corrective action can be taken earlier and also specific 'abnormal' values can be detected and analysed for their cause.So, apart from being able to predict when processes are not performing within expected normal variances the data also provides opportunities for process improvements by identifying particular uses of the process that don't fit the normal parameters. Data points that don't fit within the normal pattern can be either special causes of variation or outliers. The difference is that an outlier is actually a data point that shouldn't be included in the data comparison, maybe because it came from a process that was tailored in an uncontrolled or unapproved manner. A special cause of variation is a data point that came from using the standard process (with approved tailoring) but which didn't exhibit the normal characteristics. This might be that it was indicating a process that was performing better than expected in which case it would be worth understanding further what had happened. It might also be that it came from a problematic process in which case it would be worth establishing the causes in order to make improvements.
It's worth mentioning at this point that it would be very unusual to try to statistically control an entire process as there would tend to be too many factors influencing the process. More likely it would involve statistically controlling key components of the process that are considered influential on the success of the overall process, but for the sake of simplicity reference to processes has been used in this discussion. Also, there are rules other than simply exceeding the limits, but again it's not necessary to go into detail here provides the foundations for achieving this.
At level 4, organisations are working on understanding and removing special causes of variation in the process, but this doesn't particularly address the range of variation in using the process. In building the process performance charts, historical data from the use of the processes are collected together and the mean and range established. The range is often expressed in standard deviations, with three standard deviations being the most common value to set on the upper and lower limits; the limits being the points at which the data points become special causes of variation or outliers.
Having a process in control is a good thing as it provides reliable predictive data but having a wide range doesn't contribute well to estimating the outcomes of the process. If, for example, the testing process had achieved level 4 and the defects detected in testing consistently ranged from 5% to 40% it might be desirable to reduce this range to nearer the process performance mean whilst also, maybe, increasing the process performance mean at the same time. By understanding the variances in the data points, known as common causes of variation, this will be possible.
The results from this improvement journey for the business can be huge especially when the processes that are to be taken to this level are those that have been identified as having a significant impact on business performance. Statistically controlling every process, or just any old process, is neither practical nor beneficial. The processes, or as mentioned earlier the sub processes, that should be considered for this level of control are often those that have already been seen as contributing to business performance. For example, if the business is suffering from a higher than desirable level of live bugs then maybe it’s one of the verification processes that needs targeting. If the business is involved in state-of-the-art development where the risks are high then potentially the risk management process could be optimised to ensure the minimum number of subsequent issues. Similarly, if the engineering processes include numerous review activities such as requirement reviews, design reviews, code reviews etc. and these are seen as not working effectively or even efficiently then the review processes could be selected for the higher levels of control.
You can download this article in PDF format here.
Home | Services | Resources | Applications | Shop | About | News | Contact | Associates | General Site Map