The Tuenti Release and Development Process: Development Branch Integration

Publicado el 26/8/2013 por Víctor García, DevOps Engineer

Blog post series: part 3
You can read the previous post here.

In the previous blog post, we mentioned that one of the requisites a development branch must fulfill is a pull request. This is the only process in Tuenti to merge code to the integration branch.
We really want to have an evergreen integration branch that is always in a good state and with no tests failing. To accomplish that, we created a pull request system managed by a tool called Flow.

The Flow Pull Request System

Flow is a tool that fully orchestrates the whole integration, release and deployment process and offers dashboards to show everyone the release status, integration branch, pull requests results, etc. A full blog post will explain this, so here we’re focusing on branch integration.

Although Flow has the logic and performs the operations, every action is triggered through Jira.

Following the above diagram, here are the steps performed:

  • The developer creates an “integration” ticket in Jira. This ticket is considered a unique entity that represents a developer’s intention to integrate code, so the ticket must contain some information regarding the code that will be merged:

    • Branch contents
    • Author
    • QA member representative
    • Risks
    • Branch name
    • Mercurial revision to merge
  • Then, the ticket must be transitioned to the “Accepted” status by a QA member, so we can keep track and be assured that at least one of them has reviewed this branch.
  • To integrate the branch, the ticket must be transitioned to the “Pull request” status. This action will launch a pull request in Flow.

    • Flow and Jira are integrated and they “talk” in both directions. In this case, Jira sends a notification to Flow informing of a new pull request.
    • Additionally, in the background, Flow will gather the tickets involved in the branch and will link all of these tickets with the integration ticket using the Jira ticket relationships.
    • This provides a good overview of what it’s included in that branch to be merged.
  • Flow has a pull request queue, so they are processed sequentially.
  • Flow starts processing the pull request:

    • It creates a temporary branch with the merge of the current integration branch and the pull request branch that will be merged.
    • It configures Jenkins and triggers a build for that temporary branch to run all tests.
    • It waits until Jenkins has finished and when it’s done, Jenkins notifies Flow.

      • Jenkins executes all tests in 40 minutes, using a special and more parallelized configuration.
    • Then, it checks the Jenkins results and decides whether or not the branch can actually be merged to the integration branch.

      • If successful, it performs the merge, transitions the Jira ticket to “Integrated,” and starts with the next pull request.
      • Otherwise, the pull request is marked as failed and transitions the Jira ticket to “Rejected”.
    • Every operation Flow sends an email to the branch owner notifying him or her of the status of pull request and adds a Jira comment to the ticket.
  • If the pull request fails, Flow shows the reason for this in its dashboard and in the ticket (failed tests, merge conflicts), so the developer must:

    • Fix the problems
    • Change the Mercurial revision in the Jira ticket because the fix would require a new commit.
    • Transition the ticket to the pull request status again to launch a new pull request.

As you can see, this process requires minimum manual intervention, only clicking some Jira buttons is enough. It’s quite stable and more importantly, it lets the developers work on other projects while Flow is working, so they don’t have to worry about merges anymore. They just launch the pull request in their Jira ticket, and can forget about everything else.

So, this is how we assure the integration branch is always safe and that tests don’t fail. Therefore, every integration branch changeset is a potential release candidate.

It’s been proved that this model has improved the integration workflow, the developers performance throughput has increased so it can be considered as a total success.

You can keep reading the next post here.