One of the main tenets of agile methodology is working software trumps extensive documentation. You get something to work, and then iterate based on the quick feedback. It sounds great in theory, and in my experience, works reasonably well in practice. All software estimates are wrong, so agile is also wrong, but it produces software and does it without inflicting too much damage on those who build it.
But how do you square this way of working with a long term vision? If an organization is aligned towards a vision, there has to be a roadmap that people follow. And a roadmap, by definition, is a long term plan. It guides what needs to be done months, and sometimes years in to the future.
These two ideas seem contradictory, and they can be confusing for especially inexperienced software engineers to wrap their head around, like it was for me years ago. But after spending several years in companies big and small, I found a way to reconcile the seemingly contradictory ways of thinking. For me, two ideas bridge this gap: a) Planning is for planning b) “Agile is a state of mind”. Let me explain.
In his seminal book High Output Management, the famed late Intel CEO talks about how Intel creates a five-year roadmap, every year. This seems insane on the surface; every year a bunch of high powered executives come in and spend many hours creating a five-year plan, only to do it next year, seemingly wasting 4 years of planning! Couldn’t they just do a one year plan?
Grove points out the output of the planning process is not really the plan itself, but the mental transformation of the people involved in the process and the organizational effects. The physical, literal exercise of having people sit around a table, and discuss the future establishes a shared vocabulary, and provides a starting point and a framework for all the future ad-hoc decisions that will need to be made.
In other words, once the planning process is finished, it is the people that is reformed, and the plan, on paper, itself is just a small residue in the crucible. That transformation is provides two main things; first is a sane default for all the future decisions and the second is a lingering sense of what needs to be done to keep the momentum. In my experience, the sane default aspect is more important. The key here is not that plan is a fallback for next decisions be followed blindly but it’s a shared framework, a common place to start the conversation to initiate a discussion. This is what Eisenhower meant when he said that “No battle was ever won according to plan, but no battle was ever won without one”. It’s not what happens that matters, it’s that something happens.
This leads me to my next point, namely “the agile mindset (man)”. In most software projects, especially those in consumer field, what matters is the cadence of development. And the hardest part of that momentum is always overcoming the inertia of doing nothing. Ironically, most of the time, this inactivity manifests itself as planning. We need plans, for ourselves, surely, but we also need the antidote.
Here is an example. One of the projects I worked on involved building a new transport layer that extend all the way from the mobile client almost to the storage layer on the backend of a major enterprise. Almost literally, there was a moving part on every single part of the stack, each owned by different teams on different schedules, sometimes different timezones, different set of hopes and dreams.
The number questions with a project like that is essentially infinite. Some aspects such as security and privacy are non-negotiable, but the tail end of requirements have no end in sight. What is the monitoring story? What about error handling? How do we handle rollbacks, exceptions? Typing? Code generation? Compression, and performance? Where do you even start?
This is the point where agile mindset comes in handy. The idea behind agile is not that documentation is not useful (it is definitely useful, which I’ve learned the hard way) but it comes after the working software. The trick lies in being able to identify what really matters and what that initial state of working software looks like. In my experience, it’s always better to err on the side of simpler. Anything more than just a bit, sometimes literally, needs justification that’s simply not worth it.
So here’s what we did: we defined the security and privacy guarantees we need to provide, and only those. Nothing else. And then started building out something where the client can talk to the server, and server can respond back. It was extremely uneventful, when I tapped a button on my phone, and the random string I typed on my laptop appeared back on it. But it worked, and rest of it just followed. We found a way to handle the error handling, and handled the performance bottlenecks as they came along, and some brave souls handled code generation and today, it all works.
This is not to say the process was simple. The art of saying “not today” when people come knocking with their pet feature ideas, either from up or down the management chart, and sounding similarly credible when saying “but tomorrow” is a delicate skill. It requires credibility, resolve, and yes, sometimes a thick skin.
The selling point of “Agile” to the management has been that it provides value instantly and is more amenable to a dynamic, fast changing marketplace. And those are all true, but such verbiage can throw off those working in the trenches as MBA-speak.
For me, the main guiding principle of “agile”, with an intentional lowercase-A, has been the idea of taking into account how humans who build the software work. This isn’t surprising, considering “Agile Manifesto” was penned by actual software developers in the field. The open embrace of the messiness of doing anything that involves flesh-and-bones people is what makes the process more bearable than other forms of building software.
There are known knowns, there are known unknowns, and yes, there are unknown unknowns. There are temper tantrums, there are executive demands that come from nowhere on the 11th hour, there are teams that forgot they are involved, and there are those that casually ignore everything until the last moment.
The best we can do is aligning everyone on the same goals as best as we can, make sure people feel involved in the decisions that affect them, form the personal and organizational connections they will surely need, and have some sense of what success looks like. Rest will follow.