The Project Excellence Framework: Re-framing When to Apply Agile, Waterfall, and Change Management

Have you ever wondered, why do so many digital transformation projects fail? What is it about larger projects that present such a challenge? These questions are what led me to develop the Project Excellence framework.

The Challenge with Large Projects

Wrike reports a stark difference between the success rate of large projects (>$10 million) at only 10% vs. small projects (<$1 million) at 76%.

If Agile is a “better” project methodology than Waterfall, why aren’t digital transformation projects more successful? Forbes reports that only about 5-30% of digital transformation projects are successful defined as meeting or exceeding the original expectations of the project. This is based on studies from McKinsey, BCG, and Bain Consulting.

If you do a quick search on the internet, people have postulated many reasons on why digital transformation is challenging. I have not come across anything that satisfactorily explains the reasons, most just include a laundry list of 5-12 reasons which is what led me to develop the Project Excellence Framework.

Success cannot be achieved without some fundamentals in place, (1) a valid business case, (2) a committed project sponsor willing to fund the project, and (3) a competent project manager who is able to build a high-performing team. Even with these in place, there is clearly a need to improve success rates on larger projects – agile on its own is not sufficient, neither is waterfall or hybrid project methodologies.

What does the Project Excellence Framework provide?

When it comes to large projects, we need to leverage agile, water and change management but the key is understanding when and how to use these tools – this is what the Project Excellence framework provides.

The Project Excellence framework explains the relationship between four key variables. Once a problem has more than 3 variables, it becomes a complex problem to solve, and this is the reason I believe people have had difficulty finding what the solution is to increase the probability for project success. Agile methodologies are not sufficient. Adding Change Management alone is not sufficient. It is understanding where and when to apply these tools that matters. The four key variables of the Project Excellence framework are: Project scope (size), Level of Cross-Functionality, Relative Cost of Rework, and Project Phase (time).

The Project Excellence Framework:

-         The Excellence formula: Both quality and acceptance are equally important for a project to be successful. Excellence is not additive; it is a multiplication equation.

-         The Q (Quality) Graph: Evaluate risk prior to selecting project methodology. Project Size and the Relative Cost of Rework are the primary drivers of risk.

-         The A (Acceptance) Graph: The larger the project and the more diverse the project stakeholders are, the more important change management becomes.

-         The T (Time) Graph:  Leverage Agile methodologies for lower risk phases of the project and Waterfall methodologies for higher risk phases of the project

The Excellence Formula – the Recipe to Project Success

The excellence formula shown in the framework comes from Six Sigma and I have briefly mentioned in a previous article how it can be applied to project management. This article will expand the understanding of how the excellence formula helps to tie together some of the key variables to project success.

The main thing to remember is that excellence is a multiplication equation – it is not possible to reach a good project outcome with only a great idea or great project management tools (Quality). Human factors are just as important. Anything less than a 6 on the acceptance scale will automatically result in a failed project.

Acceptance – the “Soft Skills” of Project Management

In the Forbes article referenced earlier, the author, Dr. Corrie Block goes as far as to say “Stop promoting your technologists to their level of incompetence in the finance and soft skills areas needed to execute across a mosaic of stakeholder groups. Or at least train them when you do!”

This is why incorporating change management into larger projects is so critical. Click this link to read an earlier article I wrote on some helpful change management frameworks. Change management has historically been a separate discipline than project management with little overlap. Thankfully, this is starting to change with more companies recognizing the need to incorporate organizational change management into projects.

Quality – the “Hard Skills” of Project Management

There are many different tools that are useful in project management, such as Stakeholder Analysis, Decision Analysis, project planning/Gantt charts, risk analysis, etc. These “hard skills” make up the quality aspect of the excellence equation.

It has been interesting to watch project management evolve over the past 20 years. Projects are typically run using Waterfall methodology, Agile methodology, or a hybrid of the two.

Digital transformation efforts are often led by IT and IT projects are usually managed using agile methodologies. Could it be possible that incorporating some principles from waterfall into the execution phase of a large digital transformation project can increase the probability of success?

In the past decade or so, many companies have tried to convert as many projects as possible over to Agile for fear of being labeled as “old-school” or not keeping up with the times. This trend has lost some of its luster as companies realize implementing Agile project methodologies is not a silver bullet. Agile project methodologies have also had to evolved to provide more structure to accommodate larger projects.

Unfortunately, due the bias against Waterfall this past decade, certain segments of project managers have not been properly exposed to fundamental project management tools and have even been taught to think that Gantt charts are “bad” let alone understand how to manage the critical path in a project timeline.

Large digital transformation projects are like software projects in some ways and in other ways they are like large construction projects, so they are more challenging because it requires proficiency in both agile and waterfall methodologies.

The actual body of knowledge of project management is wider and better than before but we are less able to take advantage of it when there is a negative bias against waterfall. We should be trying to incorporate both agile and waterfall principles when applicable into projects.

The most important factor that drives the project methodology selection is risk. Let’s select 4 distinct types of projects to illustrate how to use this graph.

·        Type 1 projects are large traditional waterfall projects for example, a skyscraper or a new car

·        Type 2 projects are small traditional waterfall projects like a house renovation or a new hardware component for a smartphone

·        Type 3 projects are large agile projects such as a new software release

·        Type 4 projects are small agile projects like a simple app or a service improvement project

When high risk projects are run using predominantly Agile methodologies, the probability of failure increases. When low risk projects are run using predominantly Waterfall methodologies, the probability of unnecessary time and money spent on the project increases. Many projects will benefit from using both methodologies (hybrid approach) but at different phases of the projects. That is what we will explore next.

Time and Points of No Return

The question we need to ask ourselves is not SHOULD we use agile or waterfall but rather WHEN is it best to use agile or waterfall during the course of a project?

The Q graph needs to be used in conjunction with the T graph; the T stands for time and is a reminder that the right project management methodology may need to evolve over the course of the project. Every project goes through four steps: (1) Initiation, (2) Planning, (3) Execution, and (4) Closing. Larger projects may go through iterations of these four steps for each phase of the project. Agile projects iterate through steps 1-4 (in a typical sprint) or between steps 2-3 rapidly but they still must go through these four steps.

Here is an example of how different project methodologies can be used at different times in the project, even in traditional waterfall projects such as pharmaceutical development.

In many projects there is what I call points of no return. There may be multiple points of no return throughout the course of the project. Points of no return are extremely critical in a project and should go through a thorough decision analysis.

In the example pharmaceutical example above, one point of no return is when the drug company submits the drug to the FDA to start the approval process. They cannot go back and change the active pharmaceutical ingredient (API). Another point of no return might be once they begin construction of the full scale manufacturing facility and place orders for the main pieces of equipment which are often customized.

For example, if you are renovating your kitchen, the initial design phase is very agile, you can rearrange the layout of your kitchen as much as you’d like. But once the kitchen cabinets are ordered, that is a point of no return. There are some smaller changes where agile can still be applied, like changing the color of the cabinets if they are to be painted on-site, but the layout is fixed. The same goes with a large digital transformation project. Once the main vendor or software platforms are selected, that is a point of no return. The programming can still be done in an agile manner, but it will be all based on the platform chosen.

Practical Takeaways from the Project Excellence Framework

-         Leverage the power of AND when selecting project methodology - neither agile nor waterfall methodologies are superior to the other. The selection of the methodology should take risks into consideration which are driven by the size of the project and the cost of rework should the project fail.

-         If you are managing a large project, you or someone on your team needs to be familiar with change management principles.

-         Adopt an agile mindset, especially at the start of the project as this is the most critical phase. At the start of a project, all projects are inherently agile as risk is low and the feedback from agile iterations will increase the success rate.

-         Go through a thorough decision analysis before the points of no return in your project.

Contact Claritas Consulting & Coaching to discuss your project strategy needs or follow us on LinkedIn for more useful leadership and project management resources and frameworks. If you found this helpful, please share this article with others.