David Boyle on achieving a flow of software features out to production

Imagine you’ve identified a bug in a financial model. Perhaps one percent is represented as 1 in the code when it should have been represented as 0.01. At least the fix is quick when measured in keystrokes, but how much elapsed time is there between those keystrokes and the fix being deployed to production?

The answer to this question - a question around mean time to value - is influenced by whether you’re flowing features out to production or using a batch-oriented approach.

With a batch-oriented approach to software deployments, we identify, up front, a set of features that we think constitutes a cohesive whole that should be deployed as a versioned unit. The batch of features is deployed only after all features have been fully implemented and tested.

With a flow-oriented approach, we simply deploy whenever a feature has been completed, ignoring any larger set to which it might conceptually belong.

This flow-oriented approach immediately leads to a problem. If our development team is working on, say, two features concurrently, A and B, with these features taking two and five days effort, respectively, then how do we ensure feature B doesn’t get deployed in a half-implemented state when A is complete? One approach would be to develop features A and B on separate branches within our repository, and only merge them to a main branch when they are completed. One branch per feature works in theory, but, in practice, gives rise to frequent merge conflicts, dependencies between branches, and long-lived branches that often become neglected if short-term priorities shift.

Other solutions exist that allow us to continuously integrate our features into a single branch (often referred to as the trunk) while avoiding the problem of features being deployed prematurely.

The first is feature flags. This is the use of a flag, defined in a configuration file or database somewhere, that is queried in code to determine if a given feature should be surfaced to users or not. In the example above, feature B can be worked on and committed to trunk, hidden behind a feature flag. When A is ready, the code in trunk (including the hidden feature B) can be deployed. Once B is completed, its feature flag can be flipped to make it available to users.

Secondly, we could just look for natural ways of ensuring inchoate features are hidden from users until ready. For instance, we might work on a new page of a web application but have no routes defined leading to it. We might be implementing a feature that requires the renaming of some columns in a relational database. We could create new columns, move data over to them and then, when ready, remove the old columns. I call these quiescent or latent features (because I never like to miss an opportunity to be pretentious!). The effect is the same as with feature flags, it’s just the mechanism used to hide unfinished features is different.

Finally, we could use an approach called branch by abstraction. Here we build new features as special cases of some abstract concept. For instance, we might want to implement a new interest rate model to replace an old one. We first create an abstraction layer that any interest rate model can implement. We then shift all client code that calls the old interest rate model to do so via the abstraction layer (an application of the dependency inversion principle). Our new interest rate model can now be implemented behind an instance of this abstraction layer with client code repointed to use the new abstraction layer when the new model is complete.

These and many other techniques can be used to ensure features can be worked on and flow to production using a single branch, even when some features are not ready for prime time.