Managing A Complex Project, More An Art than A Science

Managing Complex Projects With The 4S Approach

(Illustration: Courtesy of WallPaperAccess)

“Any plan won’t survive its first encounter with reality. The reality will always be different. It will never be the plan.” — Jeff Bezos

What is it that really makes a project complex? The answer is not specifications, time, budget, or quality, but vagueness, uncertainty, and change. Whatever attention you give to framing, planning, budgeting, and staffing, the reality will always be different. Delivering a large-scale project complying with all these constraints and expectations is challenging, beyond what the books on methods or tools often state.

In this paper, I present what I learned from my experience managing product development and system integration projects.

Why managing complex projects is complex

In an ideal world, you would have clear, precise, and complete requirements. The client would hand you a final and approved document that describes the situation as-is, the destination they would like to attain, and even the road to reach the goal. It would also describe who will be the users of the resulting product, those who will enter data, those who will consume the insights that the product will generate, and all those who will be required to validate the process from start to finish. In the real world, the client is not one single person but many persons from different organizations, often located in different countries far from each other, with conflicting agendas, and different backgrounds. It is not a neat document that the client will give you but many documents whose purpose will not always be clear, just as their completeness and current status approval. The worse is that the context is ever changing. In the middle and even at the end of development, new change requests will continue to fly in from all sides of the client’s organization to the project manager.

The agile method to the rescue: necessary but not sufficient

The modern agile method brought a solution to the change request problem inherent to complex problems, but pushed to the extreme, it may bring a risk of chaos and a lack of visibility. While in the classical waterfall model, stakeholders outside of the project usually share a common appreciation on the progress made, this is not always the case in the agile method where only the project team members have the full picture. In a world governed by changes and uncertainties, the agile method is necessary but not sufficient.

In my decades of managing technological complex projects, I developed a process that integrates the agile method and the waterfall model within the 4S framework, commonly used by strategy consulting firms. Thanks to this integration, I had flexibility to cope with changes and uncertainties while keeping the project in line with the vision and objectives. This approach worked well for technology, innovation, and transformation projects where the levels of uncertainty, risks, ambiguity, and changes were high.

The agile method within the 4S framework: flexibility under control

The 4S framework was developed by strategy consulting firms to solve complex problems. It consists of four intertwined stages: problem-statement (1st first “s”), problem-structuring (the 2nd “s”), problem-solving (3rd “s”), and solution-selling (4th “s”). For a complete coverage of this general method, I recommend the book Cracked It! by Bernard Garrette, Corey Phelps, and Olivier Sibony. In the following sections, I will show how to use it for managing complex projects such as the development of a new product neither the client nor you have encountered before.

First “s” — Problem-statement

In the problem-statement stage you capture all stakeholders’ individual visions to develop a common one on business objectives, types of users, success criteria, and organizational changes. You also take into account the client’s constraints in terms of timing, budget, capabilities, and resources — both financial, technical, and human resources. At the end of this stage, you should have a full understanding of what is working, what is not, and how things could work better.

While it sounds straightforward, many of the projects actually fail at this stage because of one or more of the following reasons:

• In the majority of cases, the people who specify the needs and wishes are not the ones who will use the product. Make sure to have users from start to end of the project. Indeed, require to have both input users (those who will enter data) and output users (those who will consume data). Many of the problems encountered in customer relationship management and business intelligence systems cannot deliver output data due to the lack of the required input data.
• Managers, be they business or technical, hate writing or reading, even if it is very important to have a complete documentation about the requirements. A process I used for one of my most complicated project was to capture the requirements through two-day-long workshops. First day, start with paper boards to capture the requirements. Ask the users to show their needs and wishes in the form of screens and a flow between these screens. Take pictures of all the paper boards at the end of each meeting. Turn theses pictures into a visual prototype during the night. Second day, show the screens with buttons and animation on the morning of the next day, and repeat the same process. During the two days, the only high-tech tools used were the HD camera and a prototyping software. To complete the capture of all requirements, there was a need of three workshops in total with all the stakeholders being present at the same time.
• There is a tendency on paying high attention on what is in the scope and much less on what is out of scope of the project. This tendency is the main reason why clients and suppliers often do not agree on what is delivered versus what is expected. For the client, even not written features are mandatory. For the suppliers, what is not in the requirements is not needed. To avoid such situation at the end of the project, it is highly recommended that you clarify with the stakeholders to share with you what is out of the scope of the project as early as possible.
• Technical people tend to mix up business requirements, functional specifications, and technical specifications. Often technical documents are hard to read and follow by non technical people. Indeed, it is easy for a non specialist to be confused by the acronyms that are not common to each sector of the high-tech industry. Web development specialists have their jargon, and so do data scientists and enterprise applications developers.
• Sometimes clients want to see things flowing and running, and urge suppliers to start showing results and stop documentation as if the latter was not as important. For a new entrant, it is not rare to spend too much time reconstructing a global picture of the project and only solely then identify where and how he or she can contribute.

To quote psychologist John Dewey, “a problem well put is half solved.” This rule of thumb must be the first guide for a project manager when addressing a new project, be it the development of a new product, the integration of a new system, or the migration to a public cloud. Do not overlook the problem-statement stage if you want to avoid the risks of project failure.

As problem-statement is mainly about writing and if you want to learn or improve your skills to master this stage, the book The Pyramid Principle by Barbara Minto will help you do the job.

Second “s” — Problem-structuring

Now that the problem-statement helped you capture the as-is situation and to-be solution, you enter the problem-structuring stage where you have to:

Identify your project’s type among the following four:

• A derivative project, which is any incremental product change such as the release of a new version, or the development of add-ons or enhancements to a running system.
• A breakthrough project, which is the development of a completely new product, using breakthrough technologies such as as deep learning or blockchain.
• A platform project, which is between the two, with the objectives to leverage existing assets, intellectual properties, and knowledge and make a high leap with a new development.
• A research and development project, which is any initiative where the company is betting on the future, without very much experience of a similar project.

For more details about how to identify or classify projects, I recommend the book Business Marketing Strategy by V. Kasturi Rangan, Benson P. Shapiro, and Rowland T. Moriarty, JR.

Assess the complexity of the project among the following three:

• Assembly projects, which deal with products that perform a well-defined function within a larger system, or are independent, self-contained products that each performs a single function of its own, e.g., a mobile application, and which often involve small teams located in a single location.
• Systems projects, which involve large teams to create, implement, or replace large enterprise-wide applications such as an ERP or a CRM system.
• Array projects, which deal with a dispersed collection of systems, including hardware and software components.

For a complete coverage of project complexity, I recommend the book Reinventing Project Management by Aaron J. Shenhar and Dov Dvir.

Frame the to-be solution, processes, usage, and tools, and develop a migration strategy to move from the current solution to the new one without interrupting the services or disrupting the users until the new solution is fully operational. The to-be solution can be simply an incremental version of the current solution, a revamp, or a completely new solution. It is only at this stage, when you have evidence, that you can sign-in for timeline, milestones, and a budget. Before this stage, it is all about rules of thumbs and wishful thinking.

Identify the technology, or the combination of technologies, that is pertinent to the project at hand. For instance, if the project is to develop an AI application, then you should consider neural networks and deep learning if you have a lot of data, mathematical optimization if you do not have enough data but you have equations and constrains, business rules if you have domain knowledge, and probabilistic programming if you have uncertainty and noise in your data. It is also at this stage where you decide which features will be implemented in the front-end, server-side, application logic, and database layers, as well as the frameworks and tools that will be used for development, testing, deployment, and monitoring.

Select the best-fit approach for the next stage, problem-solving. If carefully conducted, the outputs of the problem-statement should help you select between the waterfall model or the agile method: if the requirements are clear and stable, then the waterfall model wins. If they are ambiguous and susceptible to change, then the agile method wins. For details on the differences and similarities between the two approaches, the Software Engineering Institute at Carnegie Mellon University is a rich resource.

Here again, the book The Pyramid Principle by Barbara Minto will help you do or improve the job.

Third “s” — Problem-solving

At this stage, you should have completed both the business requirements and the functional specifications. You also should have formed both the project team and the steering committee.

Whether the implementation is assigned to an external supplier through a request for proposals or to an internal team, a good approach is to combine the best of waterfall model and agile method in a series of short-term sprints of specifications, design, build, test, and run activities as follows:

Select the highest priority features the users want to deploy first.

Develop, test, deploy, and re-prioritize to take into account the changes that may have occurred since you started the previous sprint.

Revisit the complete plan according to the change requests and new priorities.

Repeat until the project is done.

For my projects to remain on track and my teams to stay focused, I always add the following three rules:

• Never stop the development of an already started feature; you would demotivate the team.
• If the project sponsor or the client comes with a new urgent feature, make sure to remove from the agenda one equivalent feature that is not started; a good rule to keep the timing and budget on track.
• Apply Microsoft’s rule of the three thirds: one third for writing features, one for implementing them, and one for testing them.

Combining both the agile method and the waterfall model enables you to control uncertainties and vagueness in the business requirements or the functional specifications, and at the same time document the project progress to keep information flowing within and beyond the project team. If you search for a big powerhouse who combined both, do not go further than Microsoft. For a full coverage of how they mastered software project management, I recommend the now classic book Microsoft Secret by Michael A. Cusumano and Richard W. Selby as well as the book The Art of Project Management by Scott Berkun.

Forth “s” — Solution-selling

In the solution-selling stage, you promote the project, explain its benefits, and train the users to get early endorsement and support. It is not because you, the project sponsor, the project team, and the steering committee are much excited about the visionary initiative and new breakthrough system you developed that everybody will be excited to the same extent. One of the biggest mistakes in transformation projects is to overlook this stage and postpone it to the end of the project.

Indeed, this stage should be carried out since the start of the problem-statement stage, and remain active until the end of the problem-solving stage. While the three previous stages require skills in analysis, synthesis, and organizational and technical skills, the solution-selling stage requires marketing and training skills. You have to make sure there is a training and marketing work-stream to help the users and managers fully capture the project’s outcomes, in particular those who will be impacted by the new system. Successful projects do this with a special attention to regular meetings and workshops, on-site and online training, continuous support and service.

Although in the previous stages you were merely talking to minds, here you should talk to hearts. A lot of books have been written about change management; my two favorites are Leading Change by John P. Kotter and Organizational Transformation by Bruce J. Avolio.

Final layout

Although I have presented the four stages as sequential, they actually overlap as shown in the figure below. The size of the stages is illustrative but usually the problem-solving stage takes longer than each of the others. The figure also shows how the waterfall model and the agile method merge to build a hybrid approach to manage the project context changes and at the same time stay aligned with the project vision.

One last thing to discuss is the tools without which we cannot conduct any of the aforementioned stage.

Tools

Project management requires tools for writing and communicating business requirements, technical specifications, meeting minutes, progress reports, and so on. My recommendation is to work with the simple rather than the sophisticated. When selecting a set of tools, favor product practicality over its features:

• All the stakeholders are not technology-native or love technology enough to allocate time learning or playing with it. Productivity tools are the most commonly used. If you expect the stakeholders will use a wiki to write their requirements or Slack to communicate, then you will probably not see a lot of writing. Busy business people prefer meetings and short memos.
• Senior-level people prefer information to come to them rather than for them to search for it. Email wisely used would do the job. Configuring or building a portal that requires sign-in dramatically reduces its number of regular users.
• Some tools will not align with the client’s licensing or security policies. It would be better to work with the tools the client has already widely adopted. For example, large clients standardize on some tools and you will have no choice but to use theirs no matter how much yours are superior.
• New tools may fade and disappear putting the future of the project at risk. Think of how many promising tools failed and with them all the projects that relied on the promises. The world of high-tech is best known for its new technologies that do not cross the chasm between early users and mainstream users.

Takeaway

Complex projects are characterized by high levels of uncertainty, ambiguity, and risks. They are hard to plan and deliver on specifications, time, quality, and budget. To harness their complexity, four skills are needed.

• First, conceptual skills, that is, the ability to see the project as whole. This includes recognizing how specifications, time, budget, and quality depend on each other, and how changes in any one affect all the others.
• Second, technical skills, that is, a proficiency in the technologies, methods, and processes needed for the project at hand. You do not conduct a web development project exactly as a data science project. The two require two different sets of technologies.
• Third, organizational skills, that is, an ability to work as a group member and to build cooperative effort among different people with different agendas, backgrounds, and cultures.
• Fourth, marketing and communication skills to sell the project beyond the project team and the steering committee, in particular to the users whose jobs will be impacted.

Are training, degrees, and certificates sufficient to make one a super complex project manager? They help but practice, practice, and practice is what makes the difference. If there is one thing you should remember from this article, it is that managing complex projects is more an art than a science. If you want to enhance your skills, I rather recommend you do not limit your readings to project management and technical matters. Indeed, reading about strategy, marketing, and organization is necessary.

Disclaimer

Although based on previous research, consulting, and implementation work, the views, thoughts, and opinions expressed in this article belong solely to the author, and not to the author’s current or previous clients or employers. The article was not reviewed nor endorsed by the individuals and companies mentioned. The author welcomes comments, please post them here or on LinkedIn.

About the author

Hassan Lâasri is a consultant in digital, data, and AI for business transformation. His areas of intervention include strategy implementation, project management, and market development. You can reach him on LinkedIn at linkedin.com/in/hassanlaasri