Heresy Against the Church of Agile Software Development

The Cranky Product Manager is a big fan of anything that will get quality, innovative, market-killing products out the door more quickly.

Sincerely. This statement is a paragon of truthiness. Hell, the Cranky Product Manager would even trade in her extensive work wardrobe of Lucky Brand jeans for a pile of poofy skirts if it would improve product quality. She would even stop swearing and referring to her colleagues as “beeyotches” if it would speed up development and reduce the number of half-finished features. No lie.

But alas, the Cranky Product Manager’s cantankerous, tomboyish lifestyle is hardly threatened, despite DysfunctoSoft’s move to Agile (specifically Scrum) two years ago.  Things are slightly better now, but not that much.

Shocking, the Cranky Product Manager knows.  Because Agile and Scrum are SO FRAKIN’ FASHIONABLE right now, that speaking of them in anything but the MOST obsequious “Agile-is-the-absolute-shizz-it’s-even-better-than-sex” manner, well, it’s kinda heresy.

You see, Agile is a pseudo-religion. It has a Manifesto and a bunch of zealots and everything. (gack)  In fact, the Cranky PM fully expects a pile of comments and trackbacks along the lines of:

  • “The CPM just doesn’t get it. Agile is indeed the shizz. Plus it’s better than sex. It’s new (well, kinda). It’s iterative. All the cool kids (claim) to do it.”
  • “The CPM is a waterfall-worshiping Luddite, afraid of change, out of touch, and a control freak — so typical of a product manager. Doesn’t she realize that Scrum liberates us developer-artistes to at long last build for intelligent audiences, people just like us?”
  • “The CPM must be doing Agile the wrong way. Not me — I know all the secrets. To learn how to do Agile right, visit my blog, read my book, hire my consulting firm, take my training class… With my help, you too can use Scrum to MAKE MONEY FAST, MELT AWAY THE POUNDS without diet or exercise, and become ABSOLUTELY IRRESISTIBLE to the opposite sex. Agile riches will exceed your wildest dreams!!!!!!!!!!”

Hmmmphhh. Such are the stones a heretic must endure.

Anyway, let the Cranky Product Manager continue and speak directly about the unspeakable:

Sure, Agile makes some things (ok, many things) better.  But it also makes some things — IMPORTANT things — worse.

Yes, Agile can speed up the development and improve the quality of small features.  But it’s too often at the expense of the Big Important Work — the heavy lifting, multi-month market analysis and architectural work that lead to REAL customer value and REAL competitive differentiation.

Yes, Agile can do wonders to improve product usability. But the results are often incremental in nature.  Opportunities to come up with innovative user interaction models that put the user experience on a different plane are not explored. Not enough time.

Yes, Scrum can improve the speed of decision making by requiring a “Product Owner” make on-the-spot decisions. But because it does not allow the Product Owner the customer face time to intimately understand the market, it too often results in products that addresses the verbatim enhancement requests of one or two specific customers (whoever spoke to the Product Owner last) while missing the real market opportunities. (See footnote 1.)

Yes, Agile can reduce the size of the development team. But instead of cutting costs, it moves the resource bottleneck to the product management and project management teams which are usually understaffed for the greater demands Agile places on them.

Yes, Agile is good at holding developers’ feet to the fire and jacking their lines-of-code-per-day rate through the roof.  It helps them fix bugs faster too. But because Agile keeps them in an always-on, semi-panicked state, it also leads to burn-out and prevents developers from doing the deep thinking required to solve the really thorny problems and to truly innovate.

Further, the War Room atmosphere and pair programming practices require a different, more outgoing personality than many developers naturally possess.  Granted, his research might be dated, but Tom DeMarco (author of Peopleware) found productivity was highest when developers had private offices with actual walls, windows, and doors that shut — the very antithesis of today’s War Room. In the War Room, the ideas of the more quiet, introverted, and thoughtful developers are too often drowned out by their louder, more obnoxious, and less gifted brethren.

So, despite the massive amount of hype currently surrounding Agile / Scrum, let the Cranky Product Manager remind you that — in the immortal words of Fred Brooks — There Is No Silver Bullet.

Agile is NOT the end-all-be-all for software product development. It is NOT the second coming. It is an improvement to be sure, but it has some big flaws – just like ALL software development methodologies.  And that is more than just “truthiness.”  It is the actual fraking TRUTH, beeyotches…

————-

1. The Cranky Product Manager knows that many Product Management bloggers believe that this problem is solved if you separate the market / customer-facing Product Manager role from the development-facing Product Owner. However, the Cranky Product Manager remains unconvinced.  How can the Product Owner gain the perspective to make the best product decisions without regular customer contact?  More on this later….

CORRECTION: In the Cranky Product Manager wrongly claimed that Rich Mironov of Enthiosys supported the idea of separating the product manager and product owner roles. Untrue. He says he supports this role split it in a world of infinite resources and perfect communication, but of course that ain’t reality. Read his article all the way through (and wait for part 2) for a more civil and scholarly discussion of the problems the Cranky Product Manager rants about. It is truly a WICKED AWESOME piece, so get over there and READ IT.

Related Posts:

22 comments

  1. Richard

    Agile was born from IT groups so solves problems product development problems from an IT perspective. I’ve never met an IT process person who really understood the complexities behind product design and specifications…

  2. Chip Overclock

    Like any process, Agile makes economic assumptions, like “communication between developers is cheap”, “communication with the customer is cheap”, “refactoring is cheap”, “testing is cheap”. If its assumptions don’t hold, Agile (or any other process) won’t work. I’ve worked in embedded environments where testing required hundreds of thousands of dollars of exotic equipment which had to be scheduled, and testing could not be automated. It’s hardly unusual for developers to be separated by many time zones, making effective (stress: effective) communication expensive to impossible. I’ve worked in multi-million line spaghetti code bases where refactoring was nearly impossible.

    Fred Brooks had it right: no silver bullet. And no process is going to make up for not having the absolutely best people possible, whether they be developers, project managers, product managers, or whatever. Development is fundamentally a creative process. A small proportion of people that make a living at it are actually any good at it.

  3. Chip Overclock

    One good thing about Agile: it’s gotten some folks to admit, or at least behave as if they admit, that development is by nature fractally iterative. You iterate at all levels: requirements, coding, testing, etc. Waterfall was pretty much built on a web of lies, pretending that this wasn’t the case.

    My experience using Windows Vista — my terrible, tragic, depressing, experience — suggests: if what is surely the largest and probably most successful software company on the planet can release a product that doesn’t work, and even if it did, is a huge step backward from prior efforts, then how can we say we have any idea how to produce a successful software product?

    OTOH, “No company has ever produced a perfect product, and we aren’t going to be the first.” — Me. Many times.

  4. Pingback: links for 2008-09-24 (Jarrett House North)
  5. Ken

    Totally agree.

    As you said, the agile treadmill keeps the developers hopping, trying to crank up the lines/day, but it doesn’t encourage a deep analysis that enables true innovation. Yes, I’ve been in a few sprints where a developer or architect is left alone for a sprint or two to think about how to do it when a user story says “make magic happen” because there’s no precedence for what’s been spec’ed. But often times it’s a real challenge for the team to more or less code peripheral things while the “magic” is being designed and integrated a few sprints later.

    Also, agile is well-matched with the pervasive practice of reusing open source libraries and tools. For example, as a developer, I have a 3-week sprint to do this code widget/feature — I’ll spend the first day or so searching for some sample open source code or libraries that’ll get me on the right track, then spend a week integrating it and customizing it to meet the needs of the story, then another week of spit and polish and testing. There’s nothing wrong with that — just an observation that waterfall was king when nearly all code was written from scratch.

  6. Mark

    IMO, Agile (and Scrum and all the rest) are just band-aids for a poor relationship between PM and Engineering. They are attempts to address the symptoms rather than the disease. Sure, they can have an effect, just like aspirin makes the tumor stop hurting, but the real problem (in many cases) is that PM and Eng often don’t trust each other and don’t respect each other. When each of us understands our role and respects the contribution of the other and genuinely likes the other, the specific development methodology becomes irrelevant. (Again, IMO.)

  7. Doug

    I find your cranky style amusing and agree with your logic about the limits of the benefits for Product Management in the current way businesses typically implement agile development methodologies. As another commenter noted, agile processes were worked out to improve developer/engineering IT pain but the squeeze on the toothpaste moved the pressure to Product Management.

    Since agile does solve important software engineering problems well, it seems wrong-headed to throw the baby out with the bath water. Rather let’s identify the mismatch between traditional Product Management and agile IT methodologies and then figure out what is required to effectively interface with an agile engineering group.

    In my view there are two aspects or roles of Product Management: Strategic Product Management and Tactical or Operational Product Management. The strategic folks interact with customers and think the “big picture” and long term thoughts about product direction. In my experience, these product managers aren’t the right people to work with IT. There needs to be another layer of Product Management that I refer to as tactical or operational Product Management.

    A Tactical Product Manager (TPM) is something akin to what used to be called a systems analyst. The key difference between the traditional systems analyst and the TPM is that the latter is schooled in how to decompose product requirements into agile user stories. The TPM, unlike the strategic Product Managers, has skin in the game – ie, is a participant in the weekly iterations and accountable with the dev team for completing the committed to work package of stories for the iteration.

    As part of an agile development team the TPM ensures that there are appropriately sized user stories prepared for each iteration that are aligned with the organization’s strategic product direction. The TPM is also a real-time clarifier of requirement issues and tries to stay a step ahead of the coders by analyzing work in progress for possible requirement gaps or conflicts. As part of the Product Management team the TPM is able to see how selected user stories build into releasable feature sets and serves as the interface between development and Product Management to determine the releasable stories or groups of stories for a release.

    My sense of your crankiness is that you’ll mellow some if a staffing strategy such as I propose above were implemented in your organization. The challenge is finding or training TPMs because traditional systems analysts (aka requirements analysts) aren’t always a good fit for agile shops – they often love big, up front analysis and thick MS Word documentation. Finding good people is a challenge on the IT side as well though so it should make you happy, in some cranky way, that the IT manager is struggling with the same issue you are.

  8. Pingback: A Drop In The Stream ›
  9. William Pietri

    Hi! Great post. I’ve been doing Agile development for quite a while, and I agree the current faddishness of it is annoying. In particular, it leads to a lot of half-assed, Agile-in-name-only implementations. I promise you, I’m even more annoyed about those than you are.

    That said I think you’ve got a few wrong notions.

    First, the problem of people not spending enough time thinking broadly about the right things has nothing to do with agile methods. Teams using waterfall-ish processes fail equally at this, but differently. They spend oceans of time on documenting and arguing, rather than thinking. Or they go off in cloud-cuckoo-land, analyzing, researching, and designing pure fantasies, ones with little relation to actual problems and achievable solutions. Then they stop thinking entirely for months or years, because it’s the wrong phase for thought.

    If you want to take time to think, there is nothing in any Agile book that says you can’t. All they say is that if you do have developers around, you need to give them enough to do that they aren’t twiddling their thumbs. So regularly set aside time for research and examining the bigger picture. No Agile guru is making you work at a white-hot pace, and developers need time to think, too.

    Second, not hiring enough product managers is not the fault of the method you’ve chosen. That developers are working faster and focusing on what they’re good at isn’t a problem unless you let it become one. If not enough product management is getting done, hire more people. That can include full product managers to share the load, junior product managers to take care of grunt work, full-time researchers to do field studies and report back, and assistants to handle the donkey work of collecting data and reporting it to stakeholders.

    Third, plenty of people doing Agile processes take time to understand their customers. At the big Agile conference this year, there was an entire track filled with presentations about how people are making it work. The trick is to take it seriously and devote adequate resources to it. Product managers are the bridge between what people need and what gets made, and a bridge that’s narrow on one end but wide on the other just doesn’t carry traffic well.

    Fourth, if your teams are always vaguely panicked, that’s a cultural issue, not a process issue. Sustainable Pace was one of the twelve original Extreme Programming practices. The Yesterday’s Weather practice also acts to force a reasonable schedule. I’m constantly coaching teams to take breaks, go home on time, and generally lead sane lives. If your leaders don’t value a measured pace, no process will fix that, because you’ll just ignore those parts of the process. As many are with their Agile adoptions.

    Fifth, introverted developers work just fine in a well-run war room environment. DeMarco’s research didn’t study Agile teams at all, and loud people can run over quiet ones in any environment. Putting them all in the same room can make the problem more obvious, but it’s not like that can’t happen in the meetings, memos, email arguments, and political maneuverings of a waterfall project. The solution is to fix the culture and the relationships, and that’s easier to do when everything’s out in the open.

    So in sum, I’d agree: Agile isn’t a silver bullet, and you should stop expecting it to fix every problem a team has. And take the fad-driven wave of dodgy Agile implementations more as evidence that following fads is bad, and less that Agile itself is.

  10. Nils Davis

    I’ve tried to boil down the key story of agile for my own understanding (and for my customers’). My simplified version of agile is:

    1. Do the most important things first
    2. Always be ready to reassess what’s most important

    Or, “spend your resources on the 20% of capabilities that will get you the best return.”

    Waterfall is more like:
    1. Figure out what you want to accomplish
    2. Determine the most efficient way to accomplish it

    It’s easy to see waterfall’s problems when characterized this way:
    a. You have to know up-front what your end game is – which makes it hard to respond to market changes
    b. The most efficient way of building the full product is not necessarily the one that front-loads the value, so often low-value items are completed and high-value items end up deferred
    c. You do a lot of work up-front to document things that the developers never get to

    Now, given my characterization of agile’s key goals, you have to look at agile *methodologies* just as a way of accomplishing them. Indeed, if the most important capability to deliver takes multiple sprints and lots of architecture, you still have to do it. And if the methodology doesn’t give you a way to do it, then the methodology won’t work for that product.

    But, on the other hand, most large monolithic capabilities *can* be broken down sensibly into multiple passes through the “smallest thing that could possibly work” approach.

    Agile is *not* a silver bullet, and it’s hard to get right, but if it helps you focus on putting first things first and executing on the 80/20 rule, it’s done its job.

  11. Pingback: Agile and The Big Dog | Wait, I know this one...
  12. GRF

    I find this somewhat amusing that a product manager would blame Agile for all their problems when it’s obvious that the problems were all there way before Agile. And, the PM at least admits that they have seen improvements with Agile. Sounds like a paradigm shift that the PM can’t make. Sure it’s different and focuses on the customer/PM so why wouldn’t your workload go up? Do you think you were hugely successful in the past when your requirements were being coded and then you didn’t recognize them when the product was released? Remember those days?????

    Sure, Agile has it’s problems but they are more related to the poor implementation and lack of reorganization that’s required than the process itself.

    Man, you certainly have issues but they are probably more personal than related to Agile.

  13. Pingback:   Christian Bale And Matthew McConaughey Teach Product Managers About Agile Development by ChristopherCummings.com
  14. Pingback: Switching Teams | The Productologist: Exploring the Depths of Product Management
  15. Pingback: Poll Results: Software Development Methodologies (Agile vs Waterfall) | The Cranky Product Manager
  16. Pingback: Blog Pav Blog › Our Highest Priority

Post a comment

You may use the following HTML:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>