The Atlassian suite is a family of web applications used at ECMWF to facilitate collaboration, both internally and with our Member and Co-operating States. An exponential increase in its use has led ECMWF to reconfigure the way in which the Centre provides these tools.
Growth in use
The suite includes Jira, a project and issue management system; Jira Service Desk, an incident and support system; Confluence, a wiki and collaboration platform; Bitbucket, a git server and code collaboration system; and finally, Bamboo, a continuous integration platform. These tools are all provided by the same vendor, Atlassian, and integrate with each other seamlessly. For example, it is possible to access Confluence knowledge base articles from Jira Service Desk, or to link together a Jira bug report with the code that was committed to Bitbucket to fix that bug and to access the results of the unit tests in Bamboo.
In 2010, ECMWF started using the Atlassian tools to help improve the way we supported our software packages. The initial user base was rapidly extended to ECMWF users beyond the software development and support groups, and from there to many external collaborators and partners across many projects. As illustrated by the charts, over the last eight years we have seen exponential growth in the usage and amount of content stored in Confluence and Jira.
New architecture
The original architecture, which involved running all services in a single VMware virtual machine, was not designed for such an intense mission-critical usage. We needed to migrate to a ‘container’ solution where each tool is isolated into its own virtual system for increased performance, ease of recovery and future scalability. The migration to our new data centre in Bologna was also a concern. The new architecture provides an infrastructure/environment for running containerised applications on premises. This ensures that we can support the tools appropriately.
The container technology chosen for the Applications Group web deployment is Docker (https://www.docker.com). Docker enables us to package and version all Atlassian releases independently. An image for each version of each Atlassian product is packaged with all of ECMWF’s custom configurations and libraries and pushed to our internal Docker registry. By doing this we ensure consistency, guaranteeing that what is released and deployed in any of our stacks, pre-production and production, is the same.
Deployment
The current production versions of Jira and Confluence have already been migrated to new systems that are accessible at https://jira.ecmwf.int and https://confluence.ecmwf.int. However, we are aware that the old URLs (https://software.ecmwf.int/issues and https://software.ecmwf.int/wiki) are still referred to from many previous documents and websites, so we have provided an automatic redirection. The migration has not yet involved upgrading the software versions of Jira and Confluence, an exercise that will be performed separately once the new production infrastructure proves to be stable. The new upgrade procedure will involve equivalent automated redeployments in the development and pre-production environments, to be pushed to production only once unit and user tests have been passed.
The final steps in the re-architecture project will be to migrate our git server Bitbucket and finally Bamboo, the continuous integration system. We are confident that the new infrastructure will enable us to handle at least a few more years of increased usage of the Atlassian tools while maintaining the stability and good performance currently enjoyed by our users.
