Reduce, reuse, recycle: McDonald’s reusable workflows
To enable a fast and reliable continuous integration process, McDonald’s turns to reusable workflows and GitHub Actions.
By Michael Gorelik, Senior Solution Architect and Achintya Pillai, Software Engineer III
McDonald’s Engineering teams are at the forefront of digital innovation, creating seamless and engaging e-commerce applications that allow customers to conveniently order their favorite meals using mobile phones.
Our technology landscape is vast, encompassing many microservices, which are written in a wide range of languages and use different cloud-native services. Engineering teams around the globe must build, test, package and publish each of these services multiple times a day to integrate ongoing changes.
Our goal is to create fast, reliable, consistent yet flexible state-of-the-art, continuous integration (CI) processes for all our different applications.
To do this, we use a combination of reusable workflows and custom reusable actions provided by GitHub. While implementing the reusable workflows with GitHub Actions, we have focused on a few key objectives for improvements:
- Maximize readability and maintainability by reducing code duplication and complexity.
- Create a golden path for the continuous integration process.
- Maximize developer productivity by increasing autonomy without sacrificing quality and standardization.
- Implement pipeline observability and monitoring.
High-level process
Increased reliability and code maintainability
To reduce the pipeline code duplication, we grouped CI workflows for applications, based on their language types and using reusable workflows on GitHub Actions to make a central CI codebase. The centralized CI workflows include the bread-and-butter stages required for any continuous integration process, such as build, quality and security scans, and create and tag artifacts. This reusable structure provides us with a common location to maintain, update and reuse code and helps ensure our applications are following a certain standard. We have also used containers to store required utilities and libraries, allowing us to run various stages within our container. Using containers for workflow executions helps avoid downloading multiple external libraries during pipeline execution, reduce pipeline execution time and eliminate security risks from external libraries.
Golden path CI process
One of our goals was to create a standardized golden path for continuous integration stages, such as code quality, security, packaging and tagging. This helps us maintain an equal quality standard across all our applications and helps us maintain a standardized naming convention for deployment artifacts across the organization.
We aim to propagate the golden path to all our applications’ CI processes. With our central reusable workflows and actions, we can ensure all our applications follow a certain set of required steps. To achieve this setup, we inject a simple CI caller file into every application, so engineers can reference the golden path reusable workflows for the CI stages and ensure their delivered application is of a certain required standard. A high-level illustration of the process is shown below:
Developer productivity
A common issue with centralized codebases like the one described above, is the frequent requests for code changes, which can take a long time, due to the many approval processes and gates that need to be addressed by a central team. Centralized repositories can be impactful because they are being shared by many teams.
With our GitHub Actions codebase for CI, we aim to give our engineers a balance of centralized control and individual autonomy. This approach will allow the organization to create and sustain a central golden path for certain required steps and provide developers the flexibility to add new stages at any time, without impacting other operations and sacrificing agility.
Let us dive into the changes process. When a CI pipeline is run, the CI caller file first references our golden path, which executes a few required stages, to maintain a certain level of quality, security, and standards across all our applications. This part will need some level of central control by an organization’s platform leads. Below is an illustration of the pull request process our engineers would follow if they needed to make a change to the defined golden path in the central CI workflow.
Although, if engineers need to add a custom CI stage for their application-specific workflow, they can add a custom action to their CI caller YAML file, without impacting other teams and going through a pull-request process, as shown in the diagram below:
Observability and monitoring
To enable enterprise-level visualization of the workflows, workflow queues and workflow logs across multiple repositories, we use CI visibility tools that allow easy integration and visualization of GitHub action workflows.
With our reusable workflows, we can unlock central monitoring solutions to help DevOps teams and applications teams monitor their pipeline performance frequently and identify future improvements. We capture the following metrics from our pipelines:
- Pipeline count
- Repository commit history that triggered the pipeline
- Pipeline lead times
- Pipeline success and failure rate
- Monitor branches used in the GitHub Actions codebase
In conclusion, reusable workflows and GitHub Actions are powerful tools to help engineers create a robust process for running their CI pipelines for many applications with various languages, versions, and frameworks. With the GitHub Actions features and tools, we have implemented a reusable component-based CI system, which addressed reliability, maintainability, productivity, and monitoring. Continuous integration golden path helps to increase user’s agility and innovation while providing built-in standardization and quality.
