This article is part 1 of a 5-part series that introduces agile concepts. In this first part, I level-set with the history of project management and agile so you can see where we have come from and where we are going in the project management industry._
It would be wrong to say that project management and agile are two different and competing ideologies. Let’s unpack that statement with a look into the rearview mirror to see how project management and agile found their way into our corporate lexicon.
The “waterfall” method
Around 1896, Karol Adamiecki created what we refer to as the harmonogram/harmonograf. Around 1910, Henry Gantt popularized the chart, and we all know it today as the Gantt chart.
The reason you hear people refer to a Gantt chart as the “waterfall” method is because of the nature of the chart itself. The idea is to lay out all the work, milestones, and deliverables required for a project, broken out into what we call work breakdown structures. You define this work from beginning to end (top to bottom) and define when the work occurs from start to finish (left to right). What you end up with is a chart that looks like the following image, that sort of looks like a waterfall.
Some projects benefit greatly from this waterfall approach, such as construction, engineering, and highly regulated projects. Building a house is a common example because it would not be prudent to construct the roof before the frame, or build anything without first acquiring the land or getting approved engineering diagrams.
Contrary to popular belief, the waterfall method is not designed to be so set in stone that you cannot make changes. If along the way, there is new scope, you define it and place it into the work breakdown structure where appropriate. However, as you can imagine, managing a project that is more than a few months in length could cause a significant load to maintain the project, so you typically need a project manager or project scheduler to keep it up to date.
Project Management formality
In the late 80s and early 90s, the formal title of project manager became extremely popular in almost every industry. For example, when I was working in IT at a refinery just outside Boston, my formal title was IT Analyst. However, it was obvious that my role was focused on building plans, managing teams, reporting on status, and all the things a project manager might do. I took night courses at Boston University to receive my project management certification and was promoted with the title of Sr. Project Manager.
To provide standardized testing and create a common lexicon, the Project Management Institute (or PMI) released a book called A User’s Manual to the PMBOK Guide. Put simply; it laid out the core elements of project management, and standards of practice project managers should follow.
The PMBOK is a small book that covers all the primary topics and goes from there to highlight other areas of study. To this day, anyone with the title Project Manager will have likely read, has been trained in, or has knowledge of the PMBOK.
Project management offices (PMOs)
In the early 2000s, we saw an ever-increasing number of project managers in all areas of business. These project managers were great at the work they do, but their approach to how they track their projects and report on status rarely matched that of their peers. In other words, even with all these standards, there was no central clearinghouse for how a company manages, reports, and assigns resources consistently.
Executives saw this as a problem, and we saw the rise of Project Management Offices (or PMOs). Every organization treated a PMO differently, and sometimes there were many PMOs. For example, the CFO might have a PMO that tracks capital and opex project costs while IT might have a PMO that manages software development efforts. Further, if IT was managing a large scale project, there could be a separate PMO just for that effort. Facilities groups would have PMOs to prioritize and track their building ane maintenance efforts. In short, PMOs were everywhere.
Enterprise Project Management software
So what happens when project managers need standards, but there are too many PMOs with too many standards? You use software to consolidate and standardize. Prior to the 2000s, most software was installed on a computer. For example, project managers would use the popular Microsoft Project or Primavera software products to manage their Gantt charts and critical paths. However, rolling all those projects, resources (people), budgets, etc. was not easy because there was no back-end database to pull the details together.
In the early 2000s we saw the emergence of enterprise project management (or EPM) software, which today we refer to as portfolio and project management software (or PPM). EPM software promised that every project manager could publish their projects to a centralized server. Using a web-based front-end, PMOs could look at the resource needs across all the projects, create standardized reporting dashboards, and provide rolled-up status reports to executives (or what we project managers like to call sponsors).
Agile team flow
As the story goes, in 2001, a group of 17 software developers and engineers met at The Lodge at the Snowbird Ski Resort in Utah. Out of that meeting came a set of values, termed the Agile Manifesto, or as titled initially, the Manifesto for Agile Software Development.
As you can see in the following image, the manifesto has a divergent view from the traditional project management methods of the day, which were to apply a waterfall approach to delivering a project.
If you think about project management coming from the early 1900s to the early 2000s, there was this thought that every project required as much detail as humanly possible before any work could start. The waterfall method required you to build a charter with a scope statement. Software development lifecycles (SDLCs) required detailed technical and user requirements prior to development can begin, and so on. You get the idea; there was a lot of so-called red tape necessary to get a project moving.
Now, if you take a look at the agile values, we are saying don’t necessarily follow a detailed plan, we can embrace change and focus on minimal amounts of documentation.
Interlude: Build it, and they will come
I remember watching a Lewis Black comedy special. He did a bit where he was telling everyone how to get people to visit your unknown town or come back to a city that lost its luster. His theory was this: If you want people to visit you, throw lots of money at building something, anything, just spend money and build it. Once built, people will show up.
Perhaps the Gantt chart would never have become widely used if it weren’t for the fact it was highlighted as the project management tool to build the Hoover Dam.
To optimize waterfall plans, we need diagrams like PERT charts and techniques like network diagrams and critical path management. Those approaches got their time in the sun with their creation and use on the Polaris Submarine project and the 1968 Winter Olympics. And of course, more and more projects would shine a positive light on the importance of using these techniques.
As you can see, all these and other high profile projects helped shine a light on these best practices and tools, which resulted in high adoption rates across many industries.
Scrum and agile frameworks
Having a list of four agile values would not be enough to gain traction.
There are many so-called agile frameworks, and one of them is called Scrum. Hirotaka Takeuchi and Ikujiro Nonaka published an article titled The New New Product Development Game for the 1986 Harvard Review. That article, laced with rugby terms, was recognized for its visionary thinking. Later, that thought leadership would see further refinement by people like Ken Schwaber and Jeff Sutherland, who, along with others, referred to the framework as Scrum.
The general concept of Scrum is we know there is work to be done. Rather than detail everything, a product owner will list and prioritize what we know today in a backlog and document them as user stories (more on that in another article).
An agile team (Scrum team), works on what they can for a short burst of time, called a sprint. During that sprint, software developers meet every day for about 15 minutes in a standup (daily scrum) meeting. When the sprint is over, the output is a skippable increment (increment) that meets the definition of done as defined in the user story.
After the sprint’s completion, there is a retrospective (sprint retrospective) that defines what could be improved for the next sprint, which is part of a continuous improvement loop.
Finally, the whole process starts over again with another sprint and more shippable increments.
Agile’s “Hoover Dam”
The problem with software development up until the mid-2000s was the big monoliths they created. It was not long ago that any enterprise application required you to purchase a massive license for an equally massive system that required dozens or even hundreds of developers to implement.
In the “old” way of building enterprise applications, there was one database that was carefully designed and fine-tuned to meet the user’s requirements. Business rules were scattered between the database, the user interface, and middleware that made the application function. If any of those parts broke, you probably broke the whole product, and it could take teams of people to figure out what went wrong. That is why it took many years for you to see new and improved versions of a software product.
There was much friction that caused software developers to stop and wait rather than sit and develop. For example, if they needed a server that did not exist, there were procurement delays that could take weeks or months. When the new server was available, IT needed to install and secure the equipment into server rooms, which would cause further delays. Sometimes developers had to install specialized software onto those servers, which went back to procurement, causing more time delays. The point is, a developer may be waiting weeks or months before they could even start doing the important work of developing software.
In the mid-2000s, Amazon released AWS (or Amazon Web Services). AWS allowed (and allows) software developers to do what took months in mere minutes. Need a server? Type a few commands, and it is running.
Further, AWS brought with it services that the developer can use, all on-demand, and easy to scale. Need a database? Create it. Need to store gigabytes of files? Create a file store. Need to scale your application from 10s or people to hundreds of thousands? Increase the application’s throughput. I’m not saying there is work involved, but months of delays went down to days, minutes, and hours.
A new software source control system called Git gained popularity, allowing developers to manage their code more efficiently. Open source communities loved the use of Git and built scaffolding services that made it easier for developers to write code. We take it for granted that when we need a website, a developer can create one quickly, but they can only do so because of open source software provides much of the building blocks, so the developer does not need to build something from scratch.
Finally, in the mid-2000s, Apple introduced the iPhone, with the promise of a computer in your pocket, not just a phone with some extra features. Writing software for the iPhone required lightweight programming that meant you needed to lean on services like AWS to do the heavy lifting for your app. Writing apps on the iPhone also meant the phone may not always be connected to the internet, something most software applications always relied on, so it would take a different kind of effort to build iPhone apps.
The [then] radical approach with AWS was that it freed developers from building monolithic applications into small components that work together as a single product. One small team (an agile team or Scrum team) of developers could work on a specific feature, like say, the administration interface. Another small team could work on the user-facing aspects, and yet another small team could work on the marketing website. These are examples, but you can see the early start of small, agile teams working together on key components that can be delivered quickly without reliance on some “big bang” new release of the product.
So it was that the “Hoover Dam” for agile was a set of services and new hardware that would drive a sort of arms race for businesses to remain relevant and adopt technology at a pace as we have never seen. Of course, it meant breaking down the old method of building applications and rethinking team structures.
Agile is organizational design, not project management
Now, in 2020, many organizations have been considering what it means to “go agile.” Some of those organizations are still grappling with the software development part of agile, but others are moving full-speed ahead with implementing agile in other lines of business, like Marketing and HR, to name a few.
We need to acknowledge that the term _project management _is a catch-all for processes, best practices, and standards. For example, I can say the process of identifying the critical path in a Gantt chart is a project management best practice. In the same breath, I can also say the procedure for writing a good user story is considered a project management best practice.
Methodologies and frameworks are nothing more than a way to collect those project management processes and best practices and compile them into a formalized approach that trains individuals how work should get done in a repeatable manner.
Agile is a way of organizing teams and optimizing team flow, so work gets done faster and with few restrictions.
The term “going agile” means creating lean, fast-moving teams. Those teams have people that support them by removing roadblocks that prevent the flow of work. An organization may say “we are using Scrum to do agile,” but that organization is not really “using scrum” if they haven’t also modified their organizational structure. They must rethink the role each individual plays in that organization, and have a continuous improvement model in place to keep changing and remain competitive.
Glossary of agile terms
The good folks at Hubstaff have a nicely curated glossary of agile and project management terms your might want to check out.
Free training
Click the link below and I will present this material to your team in an interactive one-hour meeting. Click the link now to book the training:
https://calendly.com/billraymond/meet
Educational training package
An enterprise and educator’s license is available with a PowerPoint deck, training video, PDF, and the text of this article. Click the link below to learn more:
Watch the supplemental YouTube video
Acknowledgments and disclaimers
I have compiled the history of project management from research, with a heavy reliance on Wikipedia and similar sources. Photos are courtesy of unsplash.com and pixabay.com. The Scrum framework image was from scrum.org. All copyrights and trademarks are property of their respective owners, and I do not represent any third party organizations other than my own, which is Cambermast LLC. Original images were designed by me, Bill Raymond.