As a software consultant, I spent a lot of time going into big, slow-moving organisations with legacy software and helping them sort it out. One persistent feature of these organisations is regular but infrequent releases of their platform and a fear of moving to anything more rapid.
By infrequent, I mean they might release a few times a week (Tuesday and Thursday are fairly common), or maybe each weekday, or maybe even just once a week. In the world of agile, all of these schedules are infrequent. Modern, agile platforms release constantly.
Typically, these companies will be afraid to move to anything more agile because they have a system in place and they think that it works. They say things like “we can’t risk continuous delivery (CD), people depend on our platform”.
This, in my view, is a mistake. And in this post, I am going to set out the reasons why it is safer to use CD. Not why it is better for the product owners, makes more money and keeps your developers happy, though those are all good reasons. I will make this case purely from the view of change management and their worry that it will damage the integrity of their system.
Big bang releases do not work
The old model involves people from every team merging their code into a release branch, that branch being put on a staging environment and then manual tests being run against this.
This is a terrible way to do things. As everyone merges in their code you get conflicts. Some of which will be resolved correctly, some will not.
The changes interfere with each other in ways that you cannot predict and there is just too much ground for the manual test engineers to cover.
Worse, when everything does break because you have pushed 20 features live at the same time, it is then really difficult to do anything about it because you have to check whether you can roll back, then check whether there is anything critical that needs to go out, then do a fix branch or a new release branch and rush through the whole process again.
And it produces a huge number of incidents. If you have zero incidents right now, you have a good system. But does anybody have that?
It created an automated testing culture
Such companies often say “we will move to CD when we have 100% automated test coverage”. But this is an unrealistic standard because they do not have 100% manual test coverage now.
Worse, because people rely on the manual test engineers to do the regression test, they don’t bother to put in place the correct level of automation. Maybe someday there will be a company that magically finds out how to do that. But nobody I have seen has so far.
The only way to force your engineers to do it is to move to a CD model and let them know that if they don’t put the automation in place there is no safety net and it will be traced back to them.
You don’t get features interfering with each other
Under the CD model, you release one feature at a time. So, gone are the days when two changes are merged in and don’t play nicely with each other. Each change goes out separately having passed all of the tests.
Critical features don’t get blocked
Sometimes, you have to push something out that is really important.
Under the traditional model, this is a major issue. Either you push it out as part of the scheduled release, and risk another feature breaking and you having to rollback your critical change. Or you block out the entire release and stop everyone else releasing for a few days. Which, as you can imagine, creates an even bigger big bang release later down the line.
These problems are eliminated with the one-feature-at-a-time CD model.
It is easier to roll back
If you do get a problem, it is super easy to roll back. You just hit the rollback button.
Under the traditional model, you have to check if you can roll back (due to all of the dependencies) and then if you are allowed to roll back (checking with the product owners that they are all okay with it) and then do some complicated rollback script.
All of this is simplified under a one-feature-at-a-time CD model where if it doesn’t work, you just roll it back straight away and don’t block anything else from releasing their features.
You can get fixes out faster
If something does slip through the net, you can get a fix out of the door faster than ever before. Gone are the days when you make the fix, try to work out what release branch it needs to go in, do all of your manual testing and then push it out the door.
Instead, you just write the fix and release it. And it’s fixed, way faster than it could be using the traditional model.
Summary
Yes, continuous delivery will make for happier product owners, happier developers and a faster-moving business.
But, and this is most important of all, it will also make your platform safer and more reliable. People think it will make things riskier, but, as I have outlined above, this is simply not the case.
With the CD model, you isolate every feature and every release, which is the gold standard of good change management. And, if anything does go wrong, it’s easier than ever to rollback and push a fix out.
Companies often believe that they cannot risk moving to a continious delivery model. However, if their platform truly is important, then they cannot risk not moving to the CD model.
Don't have time to check my blog? Get a weekly email with all the new posts. This is my personal blog, so obviously it is 100% spam free.
Tags: agile, best practice, continuous delivery, software
This entry was posted on Monday, October 23rd, 2017 at 11:00 am and is filed under Tech. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.