Batman vs Joker. Coca-Cola vs Pepsi. Michigan vs Ohio. Agile vs Waterfall. All equally important conflicts to the day-to-day lives of software engineers. How to pick just one to talk about is quite the dilemma. However, in being given the requirement to pick just one, the conflict that is most relevant to me is that of Agile and Waterfall organizational practices in software engineering. In a sense, both are outdated and misinformed strategies for catering work to the modern software engineer. However, of the two, agile creates the most egregious indignities upon engineers’ ability to self-manage and work equitably.
Firstly, to discuss Eric Wittman’s article “What Every Company Should Know About Agile Software Development,” he discusses the ways in which, rather optimistically, Agile software can overcome many of the monotonies of Waterfall methodology. Whitman would go as far as to say that Waterfall software, for all its emphasis on a secure and organized workflow, misses the core of a software engineers’ day-to-day workflow.
He argues for Agile software on three main bases. For one, he believes that Waterfall software tends to produce software which, rather than catering to what is most important, blanketly assumes everything to be equally necessary and causes confusion. He elaborates to say that, while Waterfall software delivers a bad but wider spanning product, Agile software creates good software in small iterations which can be later expanded upon. Further, he argues that Agile software approaches are more in-line with how human beings handle problem solving, going so far as to claim that Waterfall software, for all its attention to planning, neglects key facets of how humans problem-solve. Finally, he appeals to the fact that Agile software engineering practices tend to be preferred by programmers looking for work, and thus must be good.
An Agile-oriented organization named agile42 outlined in their article titled “Scrum Hit list — 10 good reasons for Scrum” why they feel that agile methodology, and more importantly, Scrum, is important to a healthy development environment. While one could easily replicate their article via paraphrasing into a more formal essay, to illuminate the points which seem most poignant; here is what they believe:
Agile allows for increases in changeability, giving the project its room to breathe as market demands and customer requests change. Transparency is built in, allowing everyone to ensure that work is getting done as effectively as possible. Scrum costs less and is more efficient, ensuring that the workflow is as optimized as possible.
However, these two articles in favor of Agile methodology do not stop to consider that, if something is too good to be true, it likely is. Both articles seem to be written from the point of view of a project manager rather than an on-the-ground software engineer. Therefore, to accept them at face value would be to do an injustice to alternative viewpoints.
In a blog post by Michael O. Church, he dives into how most of the “anti-Waterfall software” ideas expressed and addressed by agile software methods tend to cause more problems than they solve. While one should be hesitant to call Church “pro-Waterfall,” he is most definitely “anti-Agile”. He addresses in his blog post how Agile software, for one thing, is mostly popular amongst people who are new to the industry, unaware of their own abilities, and desiring increases in power and respect. While Waterfall software delegates tasks such that software engineers’ roles are no more than “grunt work” as he so elegantly claims, agile software allows engineers a better seat at the table.
However, there are several costs. For one thing, the over-transparency of Agile development tends to mean that engineers have less flexibility in their workflow. While Church applies a, what one could call, extreme analogy to Nazi Germany here, it might be more accurate to compare this to the transparency attempted to be instilled into truck drivers as of 2019. Truck drivers, of course, need to be awake at the wheel in order to drive. While, in the past, paper logs would be kept by drivers which they would send in to their employer to illustrate their sleep cycles, new laws attempted to impose a tracking device which would digitally record when drivers were driving in order to prevent them from driving for too long. While this seems positive, what it actually meant was that, in order to be more efficient with deliveries, truck drivers had to drive as early as this digital system would allow them, thus making it so that drivers would often need to drive while sleep deprived. This, to Church, is comparable to the role of a software engineer under Scrum.
Over-transparency and constant check-ins with a manager mean that the creativity and loose workflow of an engineer are restrained. Additionally, Church explains, the supposed weeding out of unproductivity in the workplace typically means that agile project managers will push aside engineers who cannot explain their work on a day-to-day basis and who might have some slow days and some fast days, as the managers will instead prefer a steady flow of increased development. This causes further harm when one considers that it creates something of a social hierarchy between different types of workflows, despite all intending to be allowed within the rules-as-written of Agile software. This hierarchy can exclude those for whom being disrespected in an office setting may be a deterrent based on past experiences, specifically pertaining to marginalized groups. One of the most apt descriptions Church employs is describing agile software as “glorified ‘crunch time’” (Church par. 57), analogizing Agile software to being an emergency overworking scheme.
Alesia Krush’s article “Why software developers (quite honestly) hate Agile” goes into depth on surveys and words from the creator of the original Agile Manifesto which illustrate what has gone wrong with this development mindset. Essentially, she articulates how the agile manifesto tends to upset more experienced software engineers for whom flexibility, realistic estimations, and thorough workflows are paramount to their work. Further, she explains that, from the point of view of those who initially conceptualized Agile, the way it is being employed is dangerous and unwieldy. The original intent for Agile was for it to be employed by a development team in situations where they saw it fit to do so. Using Agile as a means by which to accomplish a goal, not a goal in and of itself. To its creator, the idea of a project manager attending a few seminars on Agile and then attempting to run an entire team with it in mind is absolutely gormless.
This debate seems to illustrate something of a false dichotomy. People in favor of Agile defend it in comparison to Waterfall, a method which is very obviously flawed and most definitely the worst of the bunch. However, to do so is to assume that there is no other way. People who are against Agile do not even attempt to claim that Waterfall is good, as they would be fighting a losing battle. The reality is that while Agile may work in a bubble, it cannot work everywhere. Where Agile does produce a desirable outcome, it is hailed as the greatest development method ever conceived, with little to no regard given to the software engineers who had to work through this method.
Church’s comparison to video game development’s “crunch time” problem is one most apt. In most AAA video game studios, crunch time is a huge issue. Deadlines are imposed by executives and most of the creativity and polishing that should go into game mechanics wind up being eliminated with a preference going towards speed and output. While one could- unethically- argue that the overworking of developers with little pay increase does not matter so long as the output is good, often even that cannot be assumed.
A recent example- Cyberpunk 2077- had a studio who imposed far too harsh of a deadline in order to ensure that the game’s release corresponded to the release of the PlayStation 5. The game that they released cost around $70 and contained sections entirely incomplete which were intended to be fixed via later patches. Once those patches were rushed to release, they broke the game for several users. This issue is still ongoing, and with time and further patching, more and more issues are arising from Cyberpunk 2077.
Further, this is not to speak of the engineers whose livelihood and passion is being turned into a quick profit incentive.
Under Agile, little care is put into the desires of the engineer. If an individual wished to learn more about encrypting communications with servers, and wanted more experience in that field, but was currently most knowledgeable about MySQL, most employments of Agile would incentivize project managers to ensure that individual worked with MySQL in order to achieve the ideal outcome in the shortest timespan. Engineers under Agile are overworked, under considered, and misrepresented. Agile creates an environment in which reliability of an engineer is valued less than consistency. One could ask what the difference is therein, to which an adequate response would be that a reliable engineer can be trusted to self manage, take directions, and make durable software. A consistent engineer simply gives a result that is expectable. Being able to expect a constant outcome should be less important than reliability and trust, however when an outcome is valued more than an individual, this is not the case.
To argue for outcomes under Agile, the problem which Agile wishes to solve is that inexperienced clients will not be able to properly utilize the intent of Waterfall. Additionally, Waterfall’s reliance on engineers working based on user stories and returning with a product is less marketable, as the client will not see development until it has completed.
However, the tradeoff Agile makes is that the software produced can be referred to as “spaghetti code,” which is to say, poorly planned out and often bodged together. For an example of this, I would recommend watching a video called “The Art of the Bodge” by Tom Scott, in which a former software engineer explains a project he worked on and how he got it to work in such a way that it can perform the task he desired, no more, no less. Such methodologies may make sense for his application, but for a public release of a product which works and works well, Agile simply is not the way to go. Thoroughly planned code ensures that every line is written with intent and every implementation has a grander scheme. Agile ensures that each segment of code exists in a bubble, with the engineer’s focus being on what they are currently doing, rather than how it fits into the bigger picture.
So what is the solution? Well, certainly not Agile. Probably not Waterfall. What needs to be employed is a method in which constant communication and collaboration is encouraged, but where stages of development happen with autonomy. Engineers are not simply “grunt work,” they are designers and creators. They also are not machines, they cannot give a constant output without making sacrifices. While oversight is needed, a core premise of software engineering needs to be trust in the people you work with, and, as Church illustrates, that is something simply not present in Agile methodology.
Sources:
https://michaelochurch.wordpress.com/2015/06/06/why-agile-and-especially-scrum-are-terrible/
www.objectstyle.com/agile/why-developers-hate-agile
www.agile42.com/en/agile-info-center/scrum/10-good-reasons-implement-scrum/
