Agile

The Downside of Agile / Scrum Software Development Methodology

Agile

In software application development, Agile Software Development (ASD) is a methodology for the creative process that anticipates the need for flexibility and applies a level of pragmatism into the delivery of the finished product. Agile focuses on keeping code simple, testing often, and delivering functional bits of the application as soon as they're ready. The goal of Agile is to build upon small client-approved parts as the project progresses, as opposed to delivering one large application at the end of the project. With Agile, requirements and solutions evolve through collaboration between self-organizing, cross-functional teams. It promotes adaptive planning, evolutionary development, early delivery, continuous improvement, and encourages rapid and flexible response to change. The idea is to somehow get to market faster by doing two or three little tasks instead of one big task. (This may be best represented by the curious equation: 3 * 2 > 1 * 6.)

Waterfall

The waterfall model is a sequential design process, used in software development processes, in which progress is seen as flowing steadily downwards (like a waterfall) through the phases of conception, initiation, analysis, design, construction, testing, production / implementation and maintenance.

While the waterfall method was popular in the 70s, 80s, and 90s, Agile has really taken hold over the past few years. Unfortunately, it is being forced upon most software development shops, many of which would do better by sticking to the waterfall method. I've even heard Agile gurus speak of the "evils" of waterfall, like an evil outdated empire. Before we examine the reasons for this, let's learn more how Agile works.

Sprints

With Agile, small teams attack big problems in little chunks. A "sprint" is a selected period of time during which a team agrees to produce these software chunks. The length of the sprint is decided by the organization, but they're usually two-week or three-week periods. The idea is that the little pieces of development can be demonstrated to the stakeholders at the end of every sprint, showing what the team is doing and making sure they're on the right track. Often it's for identifying when to shut down development because the product being built isn't what the user wanted after all.

Fail Fast, and Often

Believe it or not, this is not a typo. Agile teaches us to fail fast. You build a piece of a product in two weeks; you demo it for your stakeholders; they tell you that you're full of crap; so you chalk up a failure after only two weeks instead of spending two months on the failed design. That's good. It makes sense. But, fail often? Really? Somebody didn't think this through.

Stories

At the center of Agile development is the "story," the modern word for "requirement." However, unlike its predecessor, it's short, vague, and loosely define.

Story Points & Velocity

The "story point" is a way of estimating the effort of a story. It's a key metric in evaluating the efficiency of teams, where "velocity" indicates how many story points a team can complete during a sprint. The problem is that NOBODY knows what a story point is.

Although Agile evangelists don't agree on much, they seem to agree that a story point is "arbitrary." That's right. It's something in the air that developers keep grasping at but never can harness. I've actually heard it described as "unitless," supposedly to stress that it's not supposed to be tied to a period of time. Furthermore, it is said to "evolve over time."

Still, I've been on teams that have defined a story point as one hour, other teams that define it as one day (both violating the "time" rule"), and still others that define it according to complexity. It doesn't seem to matter, as there's always some ambiguity to it. In other words, it doesn't matter how much actual progress is made during a sprint, but rather how many arbitrary and unitless items get completed. The idea is for any particular team to be consistent in the way that it defines story points, so that over time it can measure velocity to determine how many story points it can complete in a sprint.

However, in my experience, a story point is not only arbitrary and unitless, but also useless. One reason for this (as if it was not adequately explained above) is that before enough time can pass in order for a team to establish its velocity, something is assured to change--for example, the mission of the team, the leadership, or team members start quitting.

The truth is that, by definition, a story point must indeed be a measure of time because the sprint is a measure of time, and "velocity" is the number of points in that given period of time. A story point is inherently a measure of time because two weeks IS a span of "time."

Documentation

Unfortunately, Agile implies a lack of documentation. This goes hand-in-hand with Agile theory--requirements and solutions "evolving," "adaptive planning," and "evolutionary development." Students of Agile are taught to keep requirements and documentation to a minimum. They even learn special ways of writing requirements--"As a user, I want ..., so that ..." This keeps the acceptance criteria vague enough so that the testers can't be
too detailed about their expected results. This way everybody's happy with whatever is delivered, right?

Agile is particularly rough on QA because most stories aren't ready for test until the last day or two of the sprint, resulting in 16-hour days for the tester for the last couple of days of the sprint.

Meetings

Now to the meetings! Oh, the meetings! Agile gurus, who have trouble defending all of the meeting time, will say that there are only three meetings: planning meetings, stand-ups, and retrospectives. What they mean is that there are three "types" of meetings (although there are others such as grooming meetings that they conveniently categorize under planning meetings). In reality, there are dozens of meetings during each sprint. A two-week (ten-day) sprint includes the following meetings: a 2-hour sprint planning meeting; ten 30-minute daily "stand-up" scrum meetings (they're supposed to be 15 minutes, but they usually go over 30 minutes), a one-hour high-level grooming meeting, a one-hour team grooming meeting, and a 2-hour sprint retrospective / demo meeting. (I'm not sure, but I think the purpose of the demo is to make sure QA really tested what they said they did.)

Then consider other meetings such as code reviews, bug triages, and unplanned meetings, as well as the prep time included for all of these meetings such as the time to plan what you're going to say and what you're going to demo. Then that demo prep leads to a meeting where you discuss what you're actually going to demo. Then there are meetings to discuss things like, "What's a story point?" You can safely say that at least 24 hours (three of the ten days) of the sprint are consumed by meetings.

Agile Meetings

Week Sprint
# Time # Time
Stand-up Meetings 5.0 2.5 hours 10.0 5.0 hours
High Level Grooming 1.0 1.0 hour 2.0 2.0 hours
Team Grooming 1.0 1.0 hour 2.0 2.0 hours
Sprint Planning 0.5 1.0 hour 3.0 2.0 hours
Demo Prep 0.0 1.0 hour 0.0 2.0 hours
Sprint Retro / Demo 0.5 1.0 hour 1.0 2.0 hours
Misc.--Demo, Devops 1.0 1.0 hour 2.0 2.0 hours
Daily Triage 5.0 2.5 hours 10.0 5.0 hours
Code Reviews 1.0 1.0 hour 2.0 2.0 hours
Totals 15.0 12.0 hours 30 24.0 hours (30%)


So, in reality, each person has only seven days instead of ten days to develop software. With 26 sprints in a year, 624 of my hours, or 78 days, are spent in meetings every year. That's 30% of my time. For my current ten-person team, that's 780 person-days spent in meetings, or the equivalent of three person years--three more people (ten, instead of seven) needed to do the same amount of work. As a member of an Agile team, you are a meeting attender--seemingly your primary role, instead of a software engineer.

Because of the nature of all of the Agile meetings, software engineers are treated like undisciplined children rather than like adults. For example:

Stand-up Meeting Parody

The intimidating daily stand-up meetings take on a format where each member of the team, in turn, is required
to do the following:

- Literally, stand up.
- Recite what you did yesterday.
- Recite what you will do today.
- Recite what your roadblocks are.

This is akin to a kindergarten teacher (the scrummaster) asking a child:

Scrummaster: "Johnny, what did you do yesterday?"

Agile Johnny: "I went to ..."

Scrummaster: "Stand up, Johnny."

Agile Johnny (proudly): "I went to ...six meetings."

Scrummaster: "Johnny, are you getting smart with me?"

Agile Johnny: "No, sir. I almost finished two or my tasks."

Scrummaster: "Well now, that doesn't sound like very much work, does it? Why didn't you do more? Why didn't you finish those task? Didn't I see you looking up something on the internet that was completely unrelated to your tasks? Furthermore, didn't you call home to check on your sister who stayed home sick today?"

Agile Johnny: "Well, I do like my sister. Umh, oh, I tested and closed four bugs. But then, if you had just read your e-mail, you'd already know that from the automated messages you received when I documented them."

Scrummaster: "Don't get smart, Johnny. We're Agile, and we don't give a shoeshine spit about documentation.

So, what are you going to do today?"

Agile Johnny: "I'm going to work extra hard on another task, and stay off the internet."

Scrummaster: "OK Johnny, now, what is keeping you from getting your work done?"

Agile Johnny: "Meetings like this one?"

Scrummaster: "Johnny?"

Agile Johnny: "Nothing."

Scrummaster: "What, you have no roadblocks?"

Agile Johnny: "Well, I had one right after yesterday's meeting, but I discussed it with Billy and we resolved it."

Scrummaster: "You what? You were supposed to save that for the roadblock part of the stand-up meeting.

That's the most fun time of the whole day--humiliation all around."

Agile Johnny: "OK, I guess next time I'll sit around on the internet instead of being proactive."

Scrummaster: "Don't be a wise-guy, Johnny."

Planning Meeting Parody

Then there are the planning meetings where you all get to play "planning poker." Each player is dealt six cards showing the first six numbers of the fibonacci sequence--1, 2, 3, 5, 8, and 13. Then the scrummaster says, "How much effort do you think this story will take?" Then all team members simultaneously hold up one of their cards to answer the question, indicating how many arbitrary, unitless, and useless story points they think it's worth. The object of the game is to arrive at the same number through appropriate reasoning and discussion. (Have we no dignity?) I usually just try to hurry the game along by being the last one to hold up my card, and agreeing with the majority. After all, not only is a story point arbitrary, unitless, and useless, the QA folks have no freakin' idea about the complexity of the developers' Java code, and the developer mutually has
no clue about the level of manual and automated testing required:

Scrummaster: "Johnny, why do you think the effort is only two arbitrary and unitless story points? Billy thinks that it's three."

Johnny: "Because..."

Scrummaster: "Because why?

Johnny: "... Just because."

Scrummaster: "Just because why?"

Johnny: "Because it's easy?"

Scrummaster: "OK, Billy, why do you think it's three arbitrary and unitless story points?"

Billy: "Because it's hard."

Scrummaster: "Well Johnny, due to the time that we've already wasted on this, Billy is right--it's three arbitrary, unitless, and useless story points, thanks to you. Please try to be more accurate with your estimates from now on."

Retrospective Meeting Parody

Then comes the retrospective meeting when a (grown-up) software engineer is asked to "demo" the results of what he did. This is a lot like show-and-tell in the first grade when the teacher says:

Scrummaster: "Johnny, it's your turn. Go to the front of the room and tell us about the features and bugs you fixed in your software application."

Agile Johnny: "I fixed a typo. It used to say "The scrubmaster sucks," and I changed it to: "The scrummaster sucks."

Scrummaster: "Nobody cares about that crap, Johnny."

Agile Johnny: "Well you told me to show you what I did."

Scrummaster: "Get on with it, Johnny. What else did you do? Tell us something important this time."

Agile Johnny: "Well, I improved performance and enhanced security by refactoring all of our asynchronous calls."

Scrummaster" "Yeah, everyone in the room already knows that you already told us during this morning's stand-up meeting. Stop bragging. Anything else?"

Agile Johnny: "No, I think I've shown enough."

Rules Are Made to be Broken

Agile has many hard and fast rules that are always broken. For example:

When a sprint begins, it is "locked down" (like after yard time in prison). It's unacceptable to add anything to a sprint once the sprint begins, yet it's done all the time, without apology. This is often the result of the team being heavy on developers and light on testers. So, when one of the developers runs out of work, more stories are added to the sprint for him, which results in too much work for the testers, so it's the QA
guy who's working overtime.

Then of course there are the stories that need to have their estimates changed. Somehow the project manager finds a way to unlock the lock-down, crank up the estimate, lock it down again, and see how the team reacts to more work than they can do.

Sometimes they decide to run the sprints from Wednesdays through Tuesdays instead of Mondays through Fridays--Go figure. I think it's for added frustration.

Quality

One of the things you learn when you attend your initial full week of training on Agile is what happens when you can't get the work done (which is usually the case in every sprint)--like when you've underestimated the effort required for a story, or you have unexpected production crises or support issues to deal with. Something's got to give, and you have three choices: 1) Time / deadlines; 2) Content; or 3) Quality. You
can't allow more time because you're locked into your two-week sprint deadline, and another sprint starts on Monday. You can't remove content from the deliverable because the development manager has (over-) promised the functionality for the release. So, what else is left? Quality. Quality suffers because of inadequate time for testing, and everybody wonders why the stakeholders aren't satisfied with the product.

Agile Popularity

Yet, Agile is now used in some 90% of software development shops. So, why is Agile so prevalent? I think a lot of it has to do with personality types. Generally speaking, the Type A personalities like to talk, they like to hear themselves talk, they think that everybody likes to talk, and they like to have a lot of meetings. By definition, they're also the decision makers in the organization--executives, project managers, and product owners. So, they decide that meetings are the way to go, while the Type B personalities, including the folks who actually write and test the code, don't feel like they have the authority to push back on all the meetings. Yet I could build a case showing that Agile takes more people and more time (due to 12 extra hours of meetings each week, as well as failing fast and often) to build a product with lower quality than what the waterfall method would produce.

Lessons Learned

So, how do we Type B personalities cope with this? Well, after about 40 years of experience in this industry, I've found that the answer is to just wait it out. I've seen top-down programming, structured programming, object-oriented programming, waterfall, and now Agile. In a couple more years we'll all give up on it too, and start down another "arbitrary" path.

Video:

Agile is a software development methodology where, well, meetings take precedence over software development, documentation and productivity. Because of the nature of all of the Agile meetings, software engineers are treated like undisciplined children rather than like adults.

Meeting Skits:

For example, the intimidating daily stand-up meetings take on a format where each member of the team, in turn, is required to literally stand up, recite he did yesterday, recite what he will do today, and recite what his roadblocks are. It usually goes something like this:

Stand-up Meeting Skit:

Scrummaster: "Johnny, what did you do yesterday?"

Agile Johnny: "I went to ..."

Scrummaster: "Stand up, Johnny. Remember, All Hail the Agile Gods."

Agile Johnny: "All Hail the Agile Gods..." (proudly) "I went to ...six meetings."

Scrummaster: "Johnny, are you getting smart with me?"

Agile Johnny: "No, sir. I almost finished two or my tasks."

Scrummaster: "Well now, that doesn't sound like very much work, does it? Why didn't you do more? Why didn't you finish those task? Didn't I see you looking up something on the internet that was completely unrelated to your tasks? Furthermore, didn't you call home to check on your sister who stayed home sick today?"

Agile Johnny: "Well, I do like my sister. Umh, oh, I tested and closed four bugs. But then, if you had just read your e-mail, you'd already know that from the automated messages you received when I documented them."

Scrummaster: "Don't get smart, Johnny. We're Agile, and we don't give a shoeshine spit about documentation. So, what are you going to do today?"

Agile Johnny: "I'm going to work extra hard on another task, and stay off the internet."

Scrummaster: "OK Johnny, now, what is keeping you from getting your work done?"

Agile Johnny: "Meetings, like this one?"

Scrummaster: "Johnny?"

Agile Johnny: "Nothing."

Scrummaster: "What, you have no roadblocks?"

Agile Johnny: "Well, I had one right after yesterday's meeting, but I discussed it with Billy and we resolved it."

Scrummaster: "You what? You were supposed to save that for the roadblock part of the stand-up meeting.

That's the most fun time of the whole day--humiliation all around."

Agile Johnny: "OK, I guess next time I'll sit around on the internet instead of being proactive."

Scrummaster: "Don't be a wise-guy, Johnny."

Planning Meeting Skit:

Then there are the planning meetings where you all get to play "planning poker." Each player is dealt six cards showing the first six numbers of the fibonacci sequence--1, 2, 3, 5, 8, and 13. Then the scrummaster says, "How much effort do you think this story will take?" Then all team members simultaneously hold up one of their cards to answer the question, indicating how many arbitrary and unitless story points they think it's worth. The object of the game is to arrive at the same number through appropriate reasoning and discussion:

Scrummaster: "Johnny, why do you think the effort is only two arbitrary and unitless story points? Billy thinks that it's three."

Agile Johnny: "Because..."

Scrummaster: "Because why?

Agile Johnny: "... Just because."

Scrummaster: "Just because why?"

Agile Johnny: "Because it's easy?"

Scrummaster: "OK, Billy, why do you think it's three arbitrary and unitless story points?"

Agile Billy: "Because it's hard."

Scrummaster: "Well Johnny, due to the time that we've already wasted on this, Billy is right--it's three arbitrary, unitless, and useless story points, thanks to you. Please try to be more accurate with your estimates from now on."

Retrospective Meeting Skit:

Then comes the retrospective meeting when a (grown-up) software engineer is asked to "demo" the results of what he did. This is a lot like show-and-tell in the first grade:

Scrummaster: "Johnny, it's your turn. Go to the front of the room and tell us about the features and bugs you fixed in your software application."

Agile Johnny: "I fixed a typo. It used to say "The scrubmaster sucks," and I changed it to: "The scrummaster sucks."

Scrummaster: "Nobody cares about that crap, Johnny."

Agile Johnny: "Well you told me to show you what I did."

Scrummaster: "Get on with it, Johnny. What else did you do? Tell us something important this time."

Agile Johnny: "Well, I improved performance and enhanced security by refactoring all of our asynchronous calls."

Scrummaster" "Yeah, everyone in the room already knows that you already told us during this morning's stand-up meeting. Stop bragging. Anything else?"

Agile Johnny: "No, I think I've shown enough."

To read more, go to www.christiandataresources.com/agile.htm.