Developing Shared Code with Principles

One of the most high-leverage work in a technical organization is building shared libraries or frameworks. A common library, a piece of code that can be used as is, or a framework, a system that codifies certain decisions and allows further work to be built on top, has the opportunity to benefit many people at once. Not only that, they also institutionalize shared knowledge, put knowledge that’s in people’s head in code for future employees. And of course, there are other benefits such as possibly open-sourcing such work, which comes with its set of benefits to hiring and on boarding.

Of course, there are risks with such a venture. The biggest risk is of course the value risk; that the work goes unused by other teams. That is bad enough, but sometimes it gets worse. Sometimes the adoption comes, but the work instead of enabling teams, hinders them. It gets in the way of actual work, the benefits of standardization is overshadowed by the pains of integration and customization.

So how do we make sure our shared work, be them libraries or frameworks, achieves its goal? In my experience, there are 3 principles that separate successful shared projects from failed ones. Two of them are about how to build the project, and a third one about how to get it in hands of others.

Start with a Real Project

Start with the origin. The ideal way to come up with a shared tool is to extract it from an actual project. This seems straightforward enough. The biggest benefit of this approach is the framework has an immediate customer, so its creators are incentivized to solve actual problems, for actual people. In practice, the most important value comes from the “developer ergonomics”, or essentially the usability of the project is automatically improved. This is basically the cornerstone of the agile movement anyway.

Take Rails, which has been extracted out of Basecamp, a project management software. Or Django, again which was extracted from a content-management system built for a local news site. These two frameworks have different cultures, but they attack the same problem; how do you make sure your developers are more productive?

I realize this way of working is not always easy, or even possible For example, it’s easier to imagine a front-end library like Bootstrap being extracted from Twitter’s internal UI, than, say a library used for internal communications platform. But it is possible.

Keep It Small

The second way to ensure success in a shared product is to keep the surface area as small as possible. In other words, the shared project should try to provide value as soon as possible. In practice, what this means is that the framework should probably undershoot its feature set.

This might sound counter-intuitive. If we are aiming for high adoption, should we not try to cover all the possible use-cases? Shouldn’t we try to solve as many problems as possible, to make sure people find some use in our work?

The problems with trying to solve too many problems at once are plenty. First of all, as modern software development methodologies have discovered, the problem discovery is really a continuous process. Instead of trying to predict what the problems would be and trying to solve them, we should try to deliver value as soon as possible, and then iterate further.

The more subtle problem with a large feature-set is that in my experience, especially more tenured teams see it as more of a liability than leverage. They realize a big investment, especially in a new project will likely result in more work down the line.

A small word of caution here. Especially in the front-end world, this advice to keep the feature set small is taken to an unfortunate extreme. No project should need a library to add left padding to a bunch of strings. The line between a small project that does one thing and does it really well and a comically tiny project is a fine one. A good rule of thumb is to make sure the project should provide some immediate value, and be meaningful by itself. That is, one should be able to do something “production” level with your project and only that.

Take a look at first version of Rails, which is essentially a bunch of ActiveRecord classes that uses Ruby’s dynamism to build an Object Relational Mapper (ORM). All the other features that most Rails developers take for granted came many years later. Similarly, React (in addition to being extracted from an internal Facebook project), barely had many of the features it has now; support for many of the common HTML tags, ES6 classes came later.

Evangelize Constantly

And lastly, but maybe most importantly, teams should actively evangelize their projects to be adopted within a company. This might be initially uncomfortable. Many technical teams have a negative opinion of any sort of marketing. They believe that other teams should be able to evaluate their work on its merits, and everything else is either throwaway or disingenuous.

This is a short-sighted way of looking at it. Developers of shared tools should consider evangelism as not only adverting but also forming a two-way communication channel with their customers.

Take marketing. Many of your potential customers inside your company might have heard of your project, but unless they know the project is well supported and actively maintained, they will probably not consider using it. When you market your project via emails, presentations and such, you are not only letting people know of your project, but sending an active signal that it’s active, maintained project. Many times, just having a face attached to the project that is known inside the company is the difference between real adoption and leverage as opposed to a repository full of insights bit-rotting away.

Moreover, evangelism is also about forming that feedback loop with your customers. When you actively work with your prospective customers, you are getting immediate feedback about the pain points they are having. You see what some of the under-invested parts of your project might have low-hanging fruits for future wins.

Again, a bit of warning here is in order. Evangelism doesn’t come naturally to most developer teams. Moreover, with evangelism sometimes comes sacrifices. It is not unheard of to make one or two small one-off work for a big internal customer, to get some initial adoption. This might feel impure, but sometimes might be necessary. The key here is to keep the scope of the one-off demands as small as possible, and really do it for customers who would be game-changers.

Building shared libraries or frameworks is extremely fulfilling; seeing your code be adopted by others, making their lives easier is why many people get into software development in the first place. And it is an amazing way to create high leverage work in a technical organization. Ability to positively impact the work of tens, hundreds, or even thousands of fellow developers is something most executives would be excited about.

I believe with these guidelines in mind, you would be in a much better place to both deliver value for your company, and have some fun doing it.

Planning for Agile

One of the main tenets of agile methodology is working software trumps extensive documentation. You get something to work, and then iterate based on the quick feedback. It sounds great in theory, and in my experience, works reasonably well in practice. All software estimates are wrong, so agile is also wrong, but it produces software and does it without inflicting too much damage on those who build it.

But how do you square this way of working with a long term vision? If an organization is aligned towards a vision, there has to be a roadmap that people follow. And a roadmap, by definition, is a long term plan. It guides what needs to be done months, and sometimes years in to the future.

These two ideas seem contradictory, and they can be confusing for especially inexperienced software engineers to wrap their head around, like it was for me years ago. But after spending several years in companies big and small, I found a way to reconcile the seemingly contradictory ways of thinking. For me, two ideas bridge this gap: a) Planning is for planning b) “Agile is a state of mind”. Let me explain.

In his seminal book High Output Management, the famed late Intel CEO talks about how Intel creates a five-year roadmap, every year. This seems insane on the surface; every year a bunch of high powered executives come in and spend many hours creating a five-year plan, only to do it next year, seemingly wasting 4 years of planning! Couldn’t they just do a one year plan?

Grove points out the output of the planning process is not really the plan itself, but the mental transformation of the people involved in the process and the organizational effects. The physical, literal exercise of having people sit around a table, and discuss the future establishes a shared vocabulary, and provides a starting point and a framework for all the future ad-hoc decisions that will need to be made.

In other words, once the planning process is finished, it is the people that is reformed, and the plan, on paper, itself is just a small residue in the crucible. That transformation is provides two main things; first is a sane default for all the future decisions and the second is a lingering sense of what needs to be done to keep the momentum. In my experience, the sane default aspect is more important. The key here is not that plan is a fallback for next decisions be followed blindly but it’s a shared framework, a common place to start the conversation to initiate a discussion. This is what Eisenhower meant when he said that “No battle was ever won according to plan, but no battle was ever won without one”. It’s not what happens that matters, it’s that something happens.

This leads me to my next point, namely “the agile mindset (man)”. In most software projects, especially those in consumer field, what matters is the cadence of development. And the hardest part of that momentum is always overcoming the inertia of doing nothing. Ironically, most of the time, this inactivity manifests itself as planning. We need plans, for ourselves, surely, but we also need the antidote.

Let me give an example. One of the projects I worked on involved building a new transport layer that extend all the way from the mobile client almost to the storage layer on the backend of a major enterprise. Almost literally, there was a moving part on every single part of the stack, each owned by different teams on different schedules, sometimes different timezones, different set of hopes and dreams.

The number questions with a project like that is essentially infinite. Some things security and privacy are non-negotiable, but the tail end of requirements have no end in sight. What is the monitoring story? What about error handling? How do we handle rollbacks, exceptions? Typing? Code generation? Compression, and performance? Where do you even start?

This is the point where agile mindset comes in handy. The idea behind agile is not that documentation is not useful (it is definitely useful, which I’ve learned the hard way) but it comes after the working software. The trick lies in being able to identify what really matters and what that initial state of working software looks like. In my experience, it’s always better to err on the side of simpler. Anything more than just a bit, sometimes literally, needs justification that’s simply not worth it.

So here’s what we did: we defined the security and privacy guarantees we need to provide, and only those. Nothing else. And then started building out something where the client can talk to the server, and server can respond back. It was extremely uneventful, when I tapped a button on my phone, and the random string I typed on my laptop appeared back on it. But it worked, and rest of it just followed. We found a way to handle the error handling, and handled the performance bottlenecks as they came along, and some brave souls handled code generation and today, it all works.

This is not to say the process was simple. The art of saying “not today” when people come knocking with their pet feature ideas, either from up or down the management chart, and sounding similarly credible when saying “but tomorrow” is a delicate skill. It requires credibility, resolve, and yes, sometimes a thick skin.

The selling point of “Agile” to the management has been that it provides value instantly and is more amenable to a dynamic, fast changing marketplace. And those are all true, but such verbiage can throw off those working in the trenches as MBA-speak.

For me, the main guiding principle of “agile”, with an intentional lowercase-A, has been the idea of taking into account how humans who build the software work. This isn’t surprising, considering “Agile Manifesto” was penned by actual software developers in the field. The open embrace of the messiness of doing anything that involves flesh-and-bones people is what makes the process more bearable than other forms of building software.

There are known knowns, there are known unknowns, and yes, there are unknown unknowns. There are temper tantrums, there are executive demands that come from nowhere on the 11th hour, there are teams that forgot they are involved, and there are those that casually ignore everything until the last moment.

The best we can do is aligning everyone on the same goals as best as we can, make sure people feel involved in the decisions that affect them, form the personal and organizational connections they will surely need, and have some sense of what success looks like. Rest will follow.

On Being a Builder

One of the recurring themes in any technical team is the tension between designers and developers. Many designers complain about how their beautifully designed and well-thought out mocks aren’t faithfully implemented but merely considered as guidelines. A lot of the time, the design details takes a back seat to the ease-of-implementation and how detail-oriented the developer is. While there are a lot of developers who don’t mind going the extra mile to get the design “just right”, most of the time, the result ends up less than satisfactory to the designer.

On the flip side of the coin, a lot of the developers complain about the seeming disconnect of designers from the realities of building an application. Sometimes this happens in the form of designer designing something that can take an inordinate amount of time to implement or simply impossible. Other times, while the design looks great on the mocks where every piece of data is the way it is supposed to be, when the design is built and tested against real-world data, it just breaks down in unexpected ways and has to change dramatically.

While I have been mostly been on the developer side of this conundrum and definitely did my fair share of my complaining, it’s clear that this is a common problem with a lot of negative effects like inferior products that don’t feel right, unnecessary tensions between designers and developers, and wasted iteration cycles.

Different companies seem to be attacking this problem in different ways; some companies require their “designers” to actually code their designs, with Quora being the one of the well-known proponent of that approach. Quora’s job description for their product designer position explicitly lists “Ability to build what you design” amd their product designer Anne Halsall’s answer on the topic pretty much argues that the most important thing is being a builder.. Similarly, 37signals’ David Heinemer Hansson notes in a blog post that “all 37signals designers work directly with HTML and CSS“.

Yet another approach seems to be the rise of the “front-end engineer” position. As more business and consumer applications that were once desktop applications are built as web-applications where the meat of the interaction happens in the browser, people who were once simply considered “webmasters” have rightfully claimed their titles as real developers and became front-end engineers. While this position is generally considered an engineering position, it’s also always assumed (and implied in the job descriptions) that these people will have strong design sense, attention to detail to bring those intricate designs to reality as faithfully as possible.

I think both of these approaches, which aren’t mutually exclusive, are valid and have their uses. Especially in a sizable organization where there are tens or hundreds of people working together, some extreme specialization is not only desired but almost required to make sure people can work without stepping on each others’ toes.

However, I think the distinction between those who design and build application is an arbitrary one that is one that is slowly eroding. As there are better abstractions are built, the barrier to entry for realizing your idea and sharing it with the world becomes much, much lower. For developers, this means that they can prototype things out much faster and iterate on things themselves.

The real benefit, of course, is for the designers. For them, this means that they can just build what they had in mind without having to convince or wait for someone else to do it for them. I believe this is a game-changing freedom and it will only get better from here.

As we keep building better frameworks that encapsulate years of decision making, abstractions that hide things under the hood under under another plastic cover, and have simply better tools to get our work done, more and more people will be empowered to do things that once were within the technical reach of the few.

Today, anyone who can open up a Terminal window and type rails generate scaffold Post name:string title:string content:text can have a very basic blog up and running in less than a couple minutes. Top this off with some Heroku action and you have a live site running on a real database on a server somewhere in a minutes.

While the iOS space is a lot younger than the web and its reach is a lot less limited, better tools and abstractions that lower the barrier of entry are coming up fast. The XCode 4 interface is a huge improvement over the XCode 3 interface that make building an app require interacting with 1 app instead of 2; story boards are making building basic, brochure like applications essentially a drag-drop exercise, appearance proxies and callback based animations are making building iOS apps feel a bit more like using stylesheets and those familiar jQuery animations. For those who are more adventerous, tools like Pixate and RubyMotion are taking the abstraction to a whole new level where building an iOS app is essentially no different than building a web app.

Discussing the pros and cons of using abstractions over specialized tools is beyond the scope of this essay; I think while abstractions built in the name of cranking out more products faster result in subpar products, abstractions and frameworks that come out of real-world needs end up getting real traction.

Essentially, I believe we are moving to a world where we will see more builders like Sam Soffes who can churn out an iPhone app, a web application and an externally available API all by himself. When a single person can both “design” and “build” an entire product by himself, it’s clear that our nomenclature hasn’t fully caught up with people’s newly-found abilities.

That is not to say we should abolish all specialized tools; I think there will always be need for going really under the hood and actually replacing some carburetors but today, it is possible to push that need a lot further down the line. Similarly, that is not to say there’s never going to be designers who will be working away from technical tools; establishing a brand and visual identity will continue to be a job that requires professionals. Nevertheless, I believe the role of a designer as as it stands will change.

I think a lot of the responsibility will lie with the designers who will be expected to comfortable with the technology they are working with. While this seems like an exact opposite of the argument I am making, it will be more due to the adjusted expectations. Just like we are expecting better times from our Olympic athletes, designers will be expected to simply do a bit more.

As for developers, they will be freed from working as pure implementors of other people’s ideas but instead work on things that they find exciting. It is hard to make a general statement as to what this could be as it’s very domain specific but in general, I think in the future a lot more development effort will be focused on both building better abstractions for those who build on them and solving brand-new problems such as personalization and mining huge amounts of data.

It is an exciting time to be working in the tech industry right now. As we have built ourselves better tools, it is getting easier to simply work on the problems themselves. We’ll all have to learn a couple new tricks but hey, to me, that’s a small price to pay for progress.

Visual vs. Interaction Design

I for one generally hate getting into discussions about titles, especially in the realm of design. However, I’ll mention this one. Before you venture further, I’ll also disclaim that I am not a designer by trade but merely an interested, somewhat educated bystander.

As fluffy and pretentious as the name may sound, interaction design is a skill that is very, very different from visual design. The thought process that goes into creating an application that is going to be used is a lot different from the one that goes into making a jaw-dropping mockup or illustration.

In one sense, the term designer is definitely overloaded. Beyond the obvious technical differences between designing for print vs. designing for digital there is a certain overlooked difference between designing an application where people click on things, type in data; where things seemingly appear out of nowhere, change colors and designing a visual look.

That is not to say you can separate one from the other in a clean cut fashion; you definitely cannot. A great user interaction can go a long ways by itself but if you want to delight your users, you will want to couple that interaction with some great aesthetics of your own.

However, the point I am trying to make is that interaction design is a whole different animal that can be sub par no matter how good your product looks.

I do not know what makes a good interaction designer –and I still have moral qualms about using the term interaction designer–. However, I’ll make a few relatively educated guesses:

  • You need experience. This is a no-brainer. You just cannot learn this stuff in school (trust me, I tried). The challenges that you face when you actually create and support a product that are so weird, for the lack of a better term, compared to what your problems would be, that you’ll try to design things better the next time round.
  • You need education. This seems to counter the previous point but not really. I am not arguing that everyone needs to go to a HCI or design (I did but I was in the right place at the right time) but you need to know some hard facts to back up your intuitions. You need to know what an ideal line length is or if people care about below-the-fold. You can definitely make shit up as you design and probably convince yourself and people around you that your opinions are right but where’s the fun in that?
  • You need to iterate while designing. Again, a no-brainer, if you ask me. This stuff just takes time.
  • You need to work on your target medium. This is me saying, in a fancy way, that you need to be able to build your designs. A lot of people disagree with this. I think the best analogy I can think for this is translations, although that fails to capture the full breadth of what I am trying to say. Translation implies that as more touch and tweak an artifact, the more it will lose some of its initial value, which is correct. However, the real value in building what you design is that you’ll have a better feel for what works faster, which ties back to the previous point.
  • You need to get out and look at how people use other things than just a computer. I know this sounds a bit pretentious but watching people operate physical things just reinforces the importance of things like affordances, gestures, how people make mistakes, why Fitt’s law matters, what frustrates people.

Again, I’ll reiterate: a great designer (or product designer, whatever term you want to use for yourself) will need to have both good visual design skills and good interaction design skills.

The point I am trying to make here is that great interaction design is not something that is just a natural extension of great visual design.