Github Reusable Workflows
Updated: Jul 15
Keep your workflows DRY Github Reusable Workflows
It is common in an organizational environment to have multiple applications built with the same technologies or frameworks, and even having them sharing a common CI/CD pipeline. This, most of the time ends up in having to repeat a lot of code to, for example, deploy a Terraform infrastructure, upload a container image to a registry and other similar jobs.
For this reason, to keep your pipelines DRY (Don’t Repeat Yourself) we bring you Github Reusable Workflows
How a normal workflow looks like
Here we have an example of how a normal workflow to upload a container image to ECR looks like:
build_n_push: name: Build and Push to ECR runs-on: ubuntu-latest outputs: image_tag: ${{ steps.set-image-tag.outputs.image_tag }} latest_tag: ${{ steps.set-latest-tag.outputs.latest_tag }} steps: - name: Checkout code uses: actions/checkout@v2 with: submodules: true fetch-depth: 0 - name: Set Image tag id: set-image-tag shell: bash run: | echo "::set-output name=image_tag::$(echo ${{ github.sha }} | cut -c1-12)" - name: Set LATEST tag id: set-latest-tag shell: bash run: | if [ ${{ github.ref }} == 'refs/heads/develop' ]; then echo "::set-output name=latest_tag::latest"; else echo "::set-output name=latest_tag::latest-staging"; fi - name: Configure AWS Credentials id: configure-credentials uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: us-east-1 - name: Login to Amazon ECR id: login-container-registry uses: aws-actions/amazon-ecr-login@v1 - name: Build and Push Container Image id: build-container-image env: ECR_REGISTRY: ${{ steps.login-container-registry.outputs.registry }} ECR_REPOSITORY: my_ecr_repo run: | docker build . \ --progress plain \ --file Dockerfile \ --tag $ECR_REGISTRY/$ECR_REPOSITORY:${{ steps.set-image-tag.outputs.image_tag }} docker tag \ $ECR_REGISTRY/$ECR_REPOSITORY:${{ steps.set-image-tag.outputs.image_tag }} \ $ECR_REGISTRY/$ECR_REPOSITORY:${{ steps.set-latest-tag.outputs.latest_tag }} docker push $ECR_REGISTRY/$ECR_REPOSITORY:${{ steps.set-image-tag.outputs.image_tag }} docker push $ECR_REGISTRY/$ECR_REPOSITORY:${{ steps.set-latest-tag.outputs.latest_tag }}
Now imagine having to copy this workflow on each one of your applications (even you will need to copy your deployment workflows too!). This is a lot of tedious and repetitive work, also this involves a lot of human interaction, therefore making our pipelines more prone to have errors.
It would be great to have all this standar code in a place, where we can maintain it more easily and which we can use from other repos in our organization. Here is where Reusable Workflows come handy.
How to make a workflow “reusable”
It is easy to create a reusable workflow for your Github Actions CI/CD pipeline, but we have to keep in mind some limitations related to them before starting:
Reusable workflows can’t call other reusable workflows
You can’t call reusable workflows in a private repository unless you are in the same repository.
Environment variables aren’t propagated from caller workflow to called workflow (don’t worry we can solve this with inputs)
So, now we can define our first reusable workflow:
name: Build and push a Docker image to ECR on: workflow_call: inputs: ECR_REPO: required: true type: string AWS_REGION: required: false type: string default: "us-east-1" outputs: image_tag: description: Image tag created on the workflow value: ${{ jobs.build_n_push.outputs.image_tag }} latest_tag: description: Latest tag created on the workflow value: ${{ jobs.build_n_push.outputs.latest_tag }} secrets: AWS_ACCESS_KEY_ID: required: true AWS_SECRET_ACCESS_KEY: required: true jobs:
As you can see, with just adding a few things we can create a reusable workflow. We have the following parameters:
on.workflow_call: this tells Github that this workflow will be triggered via call of another workflow.
inputs: this is non sensitive data that we can pass to our workflow, in this case we are passing the ECR Registry and the AWS Region.
outputs: this is data that our workflow will output to other jobs in the caller workflow. In this example we are setting as output the image tag and the latest tag for that image (we can chain this output to our deploy workflow 😉 )
secrets: sensitive data that will be used by our workflow.
After all these parameters, all we have to do is to define our workflow like a normal one, specifying all the jobs that will run along with their steps.
But… How can I use this workflow?
To make use of our new reusable workflow first of all you can have it on the same repo of your application, or a better solution would be to have this workflow on a public repository of your organization. This makes your workflow publicly accessible unless you do a trick I will teach you in the end 🤫.
We have two ways to invoke our workflow, the first one if it is on the same repo:
name: Continuous Deployment on: push: branches: - main - develop jobs: build_and_push_image: name: Build container image uses: .github/workflows/docker_build_and_push.yaml@master with: AWS_REGION: us-east-1 ECR_REPO: my-repo secrets: AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
And now if we have our public repo with the workflow:
name: Continuous Deployment on: push: branches: - main - develop jobs: build_and_push_image: name: Build container image uses: our-org/our-workflow-repo/.github/workflows/docker_build_and_push.yaml@master with: AWS_REGION: us-east-1 APP_NAME: my-repo secrets: inherit
About the inherit, you can use it when both your repos have access to the same secrets, and they have the same name in each repo (like using organizational secrets).
BONUS TRACK: private reusable workflows
Now… maybe you want to have this workflow on a public repository so you can call it from all your applications repositories but you may not like that everyone else can call your workflow. Well there is a simple trick to ensuring only your organization can call your workflow, you can use the following template:
jobs: check_org: name: Check Caller runs-on: ubuntu-latest steps: - name: Check the calling organization if: ${{ github.repository_owner != 'my-org'' }} uses: actions/github-script@v3 with: script: | core.setFailed('This reusable workflow can only be used by My Org.') my_normal_job: needs: check_org name: This is my normal job runs-on: ubuntu-latest steps: #....
As you can see, we are using a Github context variable called repository_owner, this ensures that if the calling repo isn’t from our organization it can’t use the workflow (at least from our public repo). Remember to add the needs: check_org on your actual job.
Juan Wiggenhauser
DevOps Teracloud
