Analysis of Risk Mitigation and Risk Response Scenarios

Let’s assume that we have a residential construction project. The project might be delayed due to permitting. If the permits are delayed, our project may be delayed and cost more money. We have  several of options, we could accept the risk and cross our fingers and hope it doesn’t happen. We could choose a mitigation strategy: hire a consultant to assist with the permitting process, but this will cost money. Finally, we could execute a response plan if the risk occurs, which will also have costs associated with it. In any case, this risk can affect a number of tasks and resources and our best course of action before we decide anything is to compare the different scenarios.

Now you have three scenarios:

·         Accept

·         Mitigate

·         Response.

We can perform Monte Carlo schedule risk analysis on each scenario. These scenarios must include any other risks and uncertainties that can impact the project as all of them will contribute to the statistical distribution for project cost and duration. The results of the simulations can be presented in different forms, but we prefer using multiple cumulative probability plots (S-curve) (Figure 12.2). Using on results of analysis, you can estimate cost of each scenario based on certain confidence level (80% in our case) and then choose the best a course of action. In this case you use an indicator “project cost” and select risk mitigation scenario. Also S curves give a very good idea what would be the riskiest scenario: than wider the curve than more risky scenario is.

Scenario analysis helps to identify the effectiveness of the different options. Since mitigation and response efforts affect multiple tasks and resources, we need to calculate the entire schedule with and without mitigation and response efforts and compare different indicators including cost and duration. Project risk analysis software may offer some features that facilitate this comparison. For example, we can save the scenarios as baselines with mitigation plans and without mitigation plans and compare their costs.

We can also rank risks using scenario analysis as each risk in each scenario will have a risk score. It is also possible to isolate the impact of a risk on the project by running two simulations, one with the risk open and the other with the risk closed or disabled. The difference between the two would show the actual cost and delays caused by the risk. Repeat this process with all of the risks and we get a ranking of risks based on the calculated cost or delay caused by each risk. While this method does appear to provide more precision, it also is more time consuming. If you do require a cost of risk, it may be the effort. However, if you are simply looking to rank your risks, the method of risk ranking based on correlation described in Chapter 9 is more than adequate.

How many iterations are required

Some people believe that if they increase number of iterations in Monte Carlo simulation, they will have much more accurate results. But remember that the  accuracy of results depends on the accuracy of input data or in our case, how we define statistical distributions for task cost, duration and other parameters. Let’s assume that your estimate low and high durations of task in software development project. Even you if you perform this task few times, you cannot tell if duration is between 4 and 6 days, or between 3.5 and 6.5 days. In this case, the accuracy of estimation is greater than 10%. Such accuracy is different for different projects and different tasks. Sometimes it may be 1-5% of you maintain records and do repeatable project. However sometimes it is very significant number. If everybody would estimate project duration very accurately we would not need any analysis and you would not need to read this book. 

Standard deviation of project duration vs. number of iteration.

Mean of project duration vs. number of iteration.

Now let’s take a look how much accuracy additional Monte Carlo simulations would add. When we calculate perform Monte Carlo simulation we calculate mean and standard deviation of project duration. There are the results for standard deviation and mean of project duration of very small real software development project (Figure 5.13 and 5.14).

As you can see here if number of iterations is small, there is a significant difference between results on current and previous iteration. However, after few hundred iterations, the difference would be reduced significantly. If fact the difference between standard deviation for 500 iterations calculation and 550 iterations calculation is only 0.4%. The difference between mean for 500 iteration case and 550 iteration case is even less 0.04%. Remember the accuracy of our input uncertainties around 10%? We found that it real world it does not make sense to do many iterations. In most schedules 300-500 iterations will be correct optimal number of iterations. There are two cases where you would need to do more iterations:

1.      You have very rare events, which you would like to capture in your schedule risk analysis. For example, earthquake with probability 0.01% per duration of the project. So your number iterations should be at least 10,000 iterations in this case.

2.      It is important for you to monitor results of “extreme percentiles”, for example for P1 or P99. If you input distributions with low probability on the tails, such as triangular, you may need to increase number of iterations to get more accurate results of simulation.

Modern schedule risk analysis software can perform Monte Carlo simulations quite fast. Results could be quite different for different schedules, hardware, and software packages, but we can give you some idea. It may take 30 seconds – 1 minute to run 2000 iterations for 5000 task schedule on average computer. Any problems with performance may occur with large integrated schedules with tenth of thousands of tasks if you attempt to do very many iterations.

Role of the PMO in Project Risk Management

In a perfect world, projects would be like an isolated island untouched by outside forces and constraints, but unfortunately this isn’t the case. Projects inhabit an environment that is characterized by many constraints and risks. In larger organizations, a specific project is just one of many that are being planned or executed at any one time, the project portfolio. These  projects share common risks and uncertainties due to constraints and objectives of the larger organization in which they exist. Within this environment, the Project Management Office (PMO) provides risk management governance over the project portfolio to ensure that risks are managed consistently and aligns with the organizations objectives.

Consistency is important to ensure that same processes and criteria are used when generating  information used in the decision making process. If there is no consistency, it is difficult to compare project performance in the portfolio and allows personal biases to affect the analysis.  PMO’s ensure consistent risk management practices first by developing and distributing guidelines and specifications, templates, and training as necessary. These guidelines outline the standard organization risk management processes for identification, assessment, planning, monitoring, and controlling risks.  The guidelines will often come accompanied by templates or sample workbooks that provide standardized documentation to support the risk management process, including planning documents, assessment matrixes, and sample reports.

Along the same lines, the PMO can support the processes by providing training and mentoring to project team members. In fact, training and mentoring is just as important as providing the risk management framework (processes etc.) as this will ensure team members understand their roles in the process and how to perform them successfully. Without the training and mentoring, guidelines and specifications will suffer the same fate as  most copies of Tolstoy’s War and Peace. It is often featured prominently on bookshelves, but on closer inspection it becomes clear that book has rarely if ever been opened. Training bridges the gap between theory and practice and ensures that all projects know how to perform the required steps in the risk management process.

Finally, PMOs provide a portfolio risk management capability where the assessments of the project risks are rolled up at the portfolio level. This rolling up allows companies to identify common risks across their projects that can be managed most effectively at the organization level. This centralization of the risk management the PMO is able to provide a portfolio risk reporting capability for various organization stakeholders.

To conclude in organizations with a portfolio of projects, PMOs can provide an important role in overseeing the implementation of standardized project portfolio risk management process that includes providing a process framework, mentoring and training, a portfolio reporting capability.

Qualitative Risk Analysis for IT and Software Development Projects

If you have been following the unravelling of Phoenix, the Canadian governments new payroll system, you are aware of at least one major IT or software development project that has suffered catastrophic failures.  It is not alone, if you have them time, Google “software project failures” and it will return, the IEEE’s Software Hall of Shame is perhaps my favorite. At this point, you will probably come to the same conclusion shared by many in industry that IT projects can be extremely risky and prone to uncontrollable overruns and failure to deliver planned capabilities. IT Projects experience more risk than most other types of projects, a phenomenon that suggests that IT project managers would be wise to manage and hopefully minimize project risks. Now that we have made the case that if you are the PM of a IT project you need to manage risk, here is a brief overview of how you can quickly put one in place.             

First, you will need a risk management plan. Risk management plans outline who, what, when, and how (why was covered earlier). The plan can be a simple document that outlines how risks will be managed through the course of the project. It includes:

-          how you will identify, assess, and control risks

-          when activities will take place,

-          how you will communicate your plans to stakeholders.

Second, from the risk plan, you can develop a risk register. A project risk register is central to capturing and maintaining the information required to identify, assess, respond, and control your risks. Risk registers are commonly spreadsheets set up to track and assess risks. Common information that is tracked in a risk register:

Name	Descriptive name of risk
Description	If … then statement
Risk ID	Unique ID
Date Identified	When the risk was first identified
Likelihood	Probability or chance that a risk will occur
Impacts	Impact to project schedule, cost, quality, etc.
Severity or Score	Calculate level of risk severity
Owner	Person responsible for risk
Mitigation or Response Plan	Description of how and when the risk will be managed

The next step is the Risk Assessment, in which potential risks are identified and then assessed for their potential impact on the project. To identify risks, you can use several processes, including expert opinion, historical data, risk workshops etc. Once risks are identified, your team can then assess them based on criteria outlined in the risk management plan. For qualitative risk assessment, you can set up a simple probability and impact matrix that outlines how your project will assess probability and impacts. Here is an example that could be used on a small project:

Assessing the risk requires a simple multiplication of the risk probability by the highest impact rating using the numeric values . So for example, if a risk had the following assessment of:

Probability	Cost Impact	Schedule Impact	Quality Impact
4	3	4	4

The risk score =  Probability x Highest Impacts Score = 16

Risk scores can be converted to a rating. Using color help highlight the risk severity rating.

Rating	Score
Insignificant	1-5
Minor	6- 10
Medium	11- 15
Serious -	15-19
Critical	20 - 25

With the assessment completed, the next step to develop your strategies to manage individual  risks based on the assessment. Commonly, these strategies are: Avoid, Transfer, Mitigate, or Accept. Regardless of the selected strategy, any actions associated with plans  must be documented and include the person responsible, when it will occur and any costs associated with the plans.

Finally, during project execution, you monitor and control risks. This includes regular status updates (reviews) of existing risks monitor any changes and ensure controls are working. During project execution, schedule periodic risk sessions to include any new risks. The results of the risk reviews should be included in the Risk Register.

Final thought, keep it simple. Like many things in life, the best results come not from perfection, but from consistency. Keeping your risk management simple and paralleling your development process, will help to ensure consistency and ultimately better results.

Visualizing Risk Adjusted Linear Schedules

For visualizing location based projects, a popular alternative to Gantt charts are time location charts. Location based projects are usually construction projects, such as pipelines, buildings, and roads. In addition to standard scheduling data (start/finish times, duration, resources, and costs) each activity has location data.

Time location or linear charts are a way of visualizing project schedules with linear locations on the horizontal axis, and dates on the vertical axis. Schedule activities are then plotted onto the chart according to the locations over which they occur and the start and finish data that the project schedule determines.  This method is extremely effective in visualizing spatially how actual work will be performed and maximizes resource efficiency.

While, the appearance of time location charts is much different than the more common Gantt charts, they still include the requisite data (start time, finish time, duration, and task relationships) to allow for schedule risk analysis. When a schedule risk analysis has been performed, it is possible to generate a location based chart that compares the original vs a risk adjusted schedule. If you are familiar with RiskyProject, this type of view would be the equivalent of the Results Gantt, which shows the results of the simulation compared to the original schedule.

Creating a risk adjusted location based schedule is quite simple. First, depending upon the software you are using to generate the location based chart, set up the Simulation Results view which the specified columns and order of columns. Second, run a simulation. Third, export the results to Excel. At this point you may have to add the location based data for each activity. Finally, import the Excel data into the software that you are using to visualize the location based schedule.

Risk Scores and Risk Prioritization

A very common topic of discussion with RiskyProject users is how risks are scored.  In most cases,  users are familiar with traditional qualitative risk scoring that is a simple probability multiplied by impact. The advantage of this type of scoring is that it is both easy to understand and communicate severity to stakeholders.  If you are used to see this type of scoring, RiskyProject ‘s quantitative scoring can appear difficult to comprehend and communicate and you might ask why we don’t use the standard scoring method.  To understand why, I’ll first discuss the shortcomings of the standard risk scoring method.

Risk scores are of second order importance when assessing project risks. Primarily, risk assessment is about prioritization and the score should be considered as a value that is used to prioritize how risks will be managed and traditional risk matrix type scoring is notoriously unreliable and can often lead to poor or “worse than random decisions”.  This is analogous to how high  total cholesterol has been used as a marker for heart disease.  Using this as a marker, health authorities developed nutritional guidelines designed to lower cholesterol in the general population that focused on a low fat diet that has resulted in a massive increase in diabetes and obesity. If this was a project, it would viewed in the same light as Napoleon’s invasion of Russia. With this in mind, we can see how important it is ensure what you are measuring is an accurate reflection of the state of your project.

To increase the accuracy and validity of the risk scores and ranking, RiskyProject calculates risk scores based on their measured impact on defined project parameters such as duration or costs.  Risk probability and impacts to cost and schedule assigned to project activities and resources. So, in addition to the estimates for probability and impacts, the calculation also takes into account estimates for task durations, costs, and resource allocation. In each iteration, when you run a simulation RiskyProject does two things to calculate risk scores. First, it measures the impact of each risk on each parameter is measured in absolute units (days, dollars, etc).  As each risk occurs probabilistically and can have a range of impacts, these impacts can range from 0 – x depending on the parameter measured. Second,  the total project cost, duration, finish time, work and success rates are calculated. From these two measurements, the correlation between each and parameter is calculated. This correlation accurately reflects the expected or probabilistic impact of each risk on each parameter. Using risk weighting, this ranking can be extended to rank risks based on their overall impact on a project.  Risk weighting assigns a relative importance of one project outcome over another. For example, if a project a delay in a project will incur substantial penalties or other losses, schedule impacts can be assigned a higher importance for scoring purposes and is included in the risk score when looking at rankings for all parameters.

Communicating the results of this type of analysis can be challenging if your team is expecting to see risks scored as “High” or “20”. A risk score of  32.5% doesn’t elicit the reaction of a risk ranking of severe, highlighted in red. Make your stakeholders aware that the focus of the assessment is not to generate scores, but to prioritize risk planning by accurately ranking of risks based on calculated impacts on project objectives. If that is not enough, it is possible to calculate the expected values of risks in dollars and days,  but that is a subject for another post.

The focus of risk assessments needs to be shifted from labels or scores to ranking and prioritization. This requires abandoning or minimizing the use of unreliable qualitative methods and the use of quantitative methods that incorporate risks, cost, and schedule data that provide a more valid basis for decision making.

Project Risk Identification

What could possibly happen to my project? If you are a project manager or member of a project team, this question has probably crossed you mind on many occasions and it is a key to project success. When you ask this question, you are identifying risks that could happen during your project, which is the first step in managing your project risks. This step is called “Risk Identification” just asking what could happen is a good way to start, there are several methodologies you can use to improve your ability to identify project risks.
Brainstorming is perhaps the most popular method for identifying risks. While imagination and “thinking outside the box” are encouraged, it should be guided so that it focuses on risks that could impact key project objectives or critical activities. The goal is to cast a wide a net as possible, nothing out of bounds at this stage.
Alternatively, you can use input from experts or SME (Subject Matter Experts). One such method is Delphi. Delphi is a much more regimented process in which a facilitator sends out questionnaires to participant who remains anonymous throughout the process. They are asked to return their answers to the questionnaire to the facilitator. The facilitator then captures the ideas and sends out new questionnaires to refine the items identified in the first round. This process is iterative and continues until a consensus is reached.
Interviewing techniques can also be used to identify risks. In this process, confidentiality is used to elicit risks from project team members that ordinarily might not be identified in group settings due to group think and psychological effects. The confidentiality provides anonymity that otherwise would constrain individuals from providing complete disclosure. These three techniques are perhaps the most popular and can be used in any scale of project, but there are additional methods that can be used to augment the risk identification process.
SWOT analysis (Strengths, Weaknesses, Threats, Opportunity) is a process in these elements are represented a diagram and the project team identify internal and external factors for each element. It is very useful not only for identifying risks, but also to support decision making in the presence of risk and uncertainty.
Cause and Effect analysis are extremely useful in identifying underlying situation that can cause a risk to occur. Often cause and effect diagrams will generate causal chains of events where conditions and actions can trigger or make more probable a series of related risks to occur.
Event Chain Diagrams is a visualization technique that identifies risk and causal relationships between them. The relationships are event chains and help project managers identify and manage the risks that can have most impact on project plans.
Risk templates or checklists are by-products of the risk identification process. Using past projects, project managers can assemble lists of common risks that can be applied to future projects. These lists or templates can provide the basis for risk identification at the start of a project: however, as each project brings its own unique set of objectives and constraints, lists and templates should only be used as a starting point as they only represent a portion of the risks.

For many projects even using vigorous risk you may still miss many risks during original risk identification process. It often happens for research and development project when you don’t have enough historical project information. These “known risks” or those you identified before project starts are “tip of the iceberg”. Many risks can be identified only after project starts as part of project control. Therefore is it important to repeat risk identification process on different phases of the project.