PH.DO

Paul Hammant's Delivery Optimization

For ThoughtWorks, I was ‘Technical Principal’ for colleagues in Chicago, Dallas and Porto Allegre (Brazil). We were leading the client’s website development. Specifically building a replacement website in Java, SpringMVC, and JSP, to replace a previous site made in C++ from the mid 90’s.

The complexity is that the client was doing “concurrent development of consecutive releases”. When I arrived on the project in 2009 the first ThoughtWorks-developed release was being finalized for a production push and a sharing of load and responsibilities with the legacy Apache/CGI-BIN solution. The method to migrate from old to new was a ThoughtWorks classic: “Application Strangulation”1,2.

Changing the branching model

The challenge was to develop concurrently for releases that were spread out over years into the future, that involved two other consultancies, as well as the client’s permanent staff, organized into separate teams. Given all of those releases were of the same code, a single source repository was in play. The client was previously advised to have a “cascade” branching model, and I lobbied hard to the client and the leads from the other consultancies to move to Trunk-Based Development (TBD). At a certain point, the client acquiesced. A small amount of merging got us from cascade to TBD, and all teams carried on with that without delay. Associated practices “Feature Flags/Toggles” and “Branch by Abstraction” (on which I wrote the first online article in 20073) were taught to all involved and expertly adhered to, as well as “Don’t break the build”.

TBD facilitating a hedge bet

I rolled off the project after a couple of years. ThoughtWorks stayed, and a fellow that replaced me as Tech Principal ultimately got to prove the wisdom of the TBD choice in my absence, when the airline had to re-sequence some major releases, given a partner providing new services they intended to integrate with was going to be some months late. The consequence of that was to not release something that was imminently going out (and ready from the airline’s viewpoint), go live with a couple of other major releases, then go love with the delayed release. The flipping of some flags/toggles and some work with a new Jenkins pipeline to represent that unanticipated permutation of toggles/flags proved that the re-sequencing did not adversely change any timelines, or in itself jeopardize anything. The worst case scenario for such things with other branching models would have been many weeks of “unmerge” and other unsavory coding practices like commenting out code. I don’t know for sure, but I’ll bet there were some new converts to the wisdom of Trunk-Based Development that day.

Referenced in presentation at MERGE 2014 on TBD

At Perforce’s MERGE conference in San Francisco in 2014, I did a keynote presentation which touched on this moment for the airline. The Perforce page for that conference is down presently, so I recorded an updated version:

(the airline case study section is 6m 51s in)

Selenium cloud and BDD

I also set up the airline’s BDD tests that pulled in browsers from SauceLabs over VPN to allow for parallelized testing.

  1. StranglerApplication (Martin Fowler; 2004, 2014) 

  2. Legacy Application Strangulation : Case Studies (Jul, 2013, my blog) 

  3. Introducing Branch By Abstraction (published Apr, 2007)