my blog

$ cat github-actions/part-06.md

Reusable Workflows & Concurrency in GitHub Actions

After building custom actions, I discovered an even more powerful concept: reusable workflows. While custom actions let you reuse steps, reusable workflows let you reuse entire jobs and workflows. I’ll also show you how to manage concurrent workflow runs.

Reusable Workflows vs Custom Actions

Before diving in, let me clarify the difference because I was confused at first:

Custom Actions (from my last article):

  • Reuse steps within a job
  • Run on the same runner as the calling job
  • Can be composite, JavaScript, or Docker
  • Good for small, reusable tasks

Reusable Workflows:

  • Reuse entire workflows with multiple jobs
  • Each job runs on its own runner
  • Always YAML-based
  • Good for complete CI/CD pipelines

Creating a Reusable Workflow

Here’s my reusable deployment workflow that I can call from other workflows:

File: workflows/18-1-reusable-workflows.yaml

name: 18 - 1 - Reusable Workflows - Reusable Definition

on:
  workflow_call:
    inputs:
      target-directory:
        type: string
        required: true
    outputs:
      build-status:
        description: The status of the build process
        value: ${{ jobs.deploy.outputs.build-status }}
      url:
        description: The url of the deployed version
        value: ${{ jobs.deploy.outputs.url }}

jobs:
  deploy:
    runs-on: ubuntu-latest
    outputs:
      build-status: ${{ steps.build.outputs.build-status }}
      url: ${{ steps.deploy.outputs.url }}
    steps:
      - name: Checkout repo
        uses: actions/checkout@v4
      
      - name: Build
        id: build
        run: |
          echo "Building using directory ${{ inputs.target-directory }}"
          echo "build-status=success" >> "$GITHUB_OUTPUT"
      
      - name: Deploy
        id: deploy
        run: |
          echo "Deploying build artifacts"
          echo "url=https://www.google.com" >> "$GITHUB_OUTPUT"

Understanding the Key Differences

1. The trigger is workflow_call:

on:
  workflow_call:

This is the key! It tells GitHub this workflow can be called by other workflows, not triggered directly.

2. Inputs are defined at workflow level:

on:
  workflow_call:
    inputs:
      target-directory:
        type: string
        required: true

Unlike workflow_dispatch inputs, these are meant for other workflows to pass data.

3. Outputs are defined at workflow level:

on:
  workflow_call:
    outputs:
      build-status:
        description: The status of the build process
        value: ${{ jobs.deploy.outputs.build-status }}
      url:
        description: The url of the deployed version
        value: ${{ jobs.deploy.outputs.url }}

Notice the syntax: value: ${{ jobs.JOB_NAME.outputs.OUTPUT_NAME }}

The workflow exposes outputs from its jobs so callers can use them.

Calling a Reusable Workflow – Same Repository

Here’s how I call my reusable workflow from another workflow in the same repo:

File: workflows/18-2-reusable-workflows.yaml

name: 18 - 2 - Reusable Workflows

on: workflow_dispatch

jobs:
  deploy:
    uses: ./.github/workflows/18-1-reusable-workflows.yaml
    with:
      target-directory: dummmy-dir
  
  print-outputs:
    runs-on: ubuntu-latest
    needs: deploy
    steps:
      - name: Print outputs
        run: |
          echo "Build status: ${{ needs.deploy.outputs.build-status }}"
          echo "URL: ${{ needs.deploy.outputs.url }}"

How It Works

1. Call the workflow with uses:

deploy:
  uses: ./.github/workflows/18-1-reusable-workflows.yaml

Path format: ./.github/workflows/FILENAME.yaml

2. Pass inputs with with:

with:
  target-directory: dummmy-dir

3. Access outputs with needs.JOB_NAME.outputs

needs: deploy
steps:
  - name: Print outputs
    run: |
      echo "Build status: ${{ needs.deploy.outputs.build-status }}"
      echo "URL: ${{ needs.deploy.outputs.url }}"

The deploy job isn’t a regular job – it’s calling another workflow. But you can still use needs: and get its outputs!

Calling a Reusable Workflow – Different Repository

This is where it gets really powerful. You can call workflows from other repositories!

File: workflows/18-3-reusable-workflows.yaml

name: 18 - 3 - Reusable Workflows

on: workflow_dispatch

jobs:
  deploy:
    uses: ./.github/workflows/18-1-reusable-workflows.yaml
    with:
      target-directory: dummmy-dir
  
  e2e-tests:
    uses: udemy-lauromueller/github-actions-course-example-e2e/.github/workflows/e2e.yaml@main
    needs: deploy
    secrets:
      access-token: ${{ secrets.GH_TOKEN }}

Cross-Repository Workflow Calls

The syntax:

uses: OWNER/REPO/.github/workflows/FILENAME.yaml@REF
  • OWNER/REPO – GitHub username/organization and repository name
  • .github/workflows/FILENAME.yaml – Path to the workflow file
  • @REF – Branch, tag, or commit SHA (I use @main)

Example:

uses: udemy-lauromueller/github-actions-course-example-e2e/.github/workflows/e2e.yaml@main

This calls the e2e.yaml workflow from another repository’s main branch.

Passing Secrets to External Workflows

secrets:
  access-token: ${{ secrets.GH_TOKEN }}

Reusable workflows don’t automatically get access to secrets. You must explicitly pass them.

Here’s the reusable workflow that accepts secrets:

File: workflows/e2e.yaml

name: E2E tests

on:
  workflow_dispatch:
  workflow_call:
    secrets:
      access-token:
        required: true

jobs:
  e2e-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Print github owner and repo
        run: |
          echo "Github owner and repo: ${{ github.repository }}"
      
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          repository: udemy-lauromueller/github-actions-course-example-e2e
          ref: main
          token: ${{ secrets.access-token || secrets.GITHUB_TOKEN }}
      
      - name: Setup node
        uses: actions/setup-node@v4
        with:
          node-version: '20.x'
      
      - name: Install dependencies
        run: npm ci
      
      - name: Run E2E tests
        run: npm run test:e2e

Understanding the Secret Declaration

on:
  workflow_call:
    secrets:
      access-token:
        required: true

This workflow accepts a secret named access-token. The calling workflow must provide it.

Using the secret:

token: ${{ secrets.access-token || secrets.GITHUB_TOKEN }}

This tries to use the passed access-token, and falls back to GITHUB_TOKEN if not provided.

Reusable Workflows: What I Learned

1. You Can Have Multiple Triggers

on:
  workflow_dispatch:
  workflow_call:
    inputs:
      target-directory:
        type: string
        required: true

My e2e workflow can be called by other workflows OR triggered manually. This is useful for testing.

2. Reusable Workflows Have Limits

  • Maximum 4 levels of nesting (workflow calls workflow calls workflow…)
  • Cannot call workflows from private repos in different organizations
  • The calling workflow and reusable workflow share the same GITHUB_TOKEN permissions

3. Context Variables Behave Differently

When you call a reusable workflow:

  • github.repository in the reusable workflow shows the CALLING repository, not the reusable workflow’s repository
  • github.ref shows the CALLING workflow’s branch
  • github.sha shows the CALLING workflow’s commit

This was confusing at first! The reusable workflow runs in the context of the caller.

4. You Can’t Use needs Between Jobs in Different Workflows

This doesn’t work:

# In the calling workflow, you can't do this:
# (This would be trying to access a job inside the reusable workflow)
needs: job1  # This works within the reusable workflow

# But from the caller, you can only use:
needs: deploy  # The entire reusable workflow call

But you can’t have a job in the calling workflow that needs a job inside the reusable workflow. You can only use needs with the entire reusable workflow call.

Concurrency: Preventing Parallel Runs

Now let’s talk about concurrency. Sometimes you don’t want multiple workflow runs happening at the same time.

Workflow-Level Concurrency

File: workflows/19-1-concurrency.yaml

name: 19 - 1 - Managing Concurrency

on:
  workflow_dispatch:

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  ping:
    runs-on: ubuntu-latest
    steps:
      - name: ping
        run: |
          echo "Running workflow"
          sleep 15

Understanding Concurrency Groups

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

What this does:

  • Creates a concurrency group named after the workflow and branch
  • If another run with the same group starts, GitHub cancels the first one
  • This prevents multiple deployments to the same environment

The group name:

  • ${{ github.workflow }} – Workflow name (“19 – 1 – Managing Concurrency”)
  • ${{ github.ref }} – Branch name (“refs/heads/main”)
  • Combined: “19 – 1 – Managing Concurrency-refs/heads/main”

Example scenario:

  1. I trigger the workflow on main branch – Run #1 starts
  2. I trigger it again on main branch – Run #2 starts, Run #1 gets cancelled
  3. I trigger it on dev branch – Run #3 starts and runs in parallel with Run #2 (different group!)

Job-Level Concurrency

You can also set concurrency at the job level:

File: workflows/19-2-concurrency.yaml

name: 19 - 2 - Managing Concurrency

on:
  workflow_dispatch:

jobs:
  ping-with-concurrency:
    runs-on: ubuntu-latest
    concurrency:
      group: ${{ github.workflow }}-with-concurrency
    steps:
      - name: ping
        run: |
          echo "Running job with concurrency"
          sleep 10
  
  ping-without-concurrency:
    runs-on: ubuntu-latest
    steps:
      - name: ping
        run: |
          echo "Running job without concurrency"
          sleep 10

The Difference

Workflow-level concurrency:

  • Applies to the entire workflow
  • If a new run starts, ALL jobs in the old run get cancelled

Job-level concurrency:

  • Only applies to that specific job
  • Other jobs in the workflow continue running
  • Useful when only certain jobs need concurrency control

In my example:

  • ping-with-concurrency – Only one instance runs at a time
  • ping-without-concurrency – Multiple instances can run in parallel

Advanced Concurrency: cancel-in-progress

By default, GitHub cancels the old run when a new one starts. But you can change this behavior:

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: false

cancel-in-progress: true (default):

  • New run starts → Old run cancelled
  • Use this for deployments (you want the latest code deployed)

cancel-in-progress: false:

  • New run starts → New run waits for old run to finish
  • Use this for expensive operations you don’t want to restart

Real-World Concurrency Patterns

Pattern 1: Deployment Concurrency by Environment

name: Deploy Application

on:
  push:
    branches: [main]

jobs:
  deploy-staging:
    runs-on: ubuntu-latest
    concurrency:
      group: deploy-staging
      cancel-in-progress: false
    steps:
      - run: echo "Deploying to staging"
  
  deploy-production:
    runs-on: ubuntu-latest
    concurrency:
      group: deploy-production
      cancel-in-progress: false
    steps:
      - run: echo "Deploying to production"

This ensures:

  • Only one deployment to staging at a time
  • Only one deployment to production at a time
  • But staging and production deployments can run in parallel (different groups)
  • New deployments wait rather than cancel (cancel-in-progress: false)

Pattern 2: Per-Branch Concurrency

name: Build and Test

on:
  push:

jobs:
  build:
    runs-on: ubuntu-latest
    concurrency:
      group: ${{ github.ref }}
      cancel-in-progress: true
    steps:
      - run: echo "Building ${{ github.ref }}"

This ensures:

  • Each branch has its own concurrency group
  • Multiple branches can build in parallel
  • New pushes to the same branch cancel old runs (you only care about the latest code)

Pattern 3: Global Lock for Shared Resources

name: Database Migration

on:
  push:

jobs:
  migrate:
    runs-on: ubuntu-latest
    concurrency:
      group: database-migration
      cancel-in-progress: false
    steps:
      - run: echo "Running migration"

This ensures:

  • Only ONE migration runs across ALL branches and workflows
  • Fixed group name means all runs share the same lock
  • Migrations queue up rather than cancel (critical for database changes)

Combining Reusable Workflows and Concurrency

Here’s how I combine both concepts for a production deployment:

name: Production Deployment Pipeline

on:
  workflow_dispatch:

concurrency:
  group: production-deployment
  cancel-in-progress: false

jobs:
  build:
    uses: ./.github/workflows/build.yaml
  
  test:
    needs: build
    uses: ./.github/workflows/test.yaml
  
  deploy:
    needs: test
    uses: ./.github/workflows/deploy.yaml
    secrets:
      access-token: ${{ secrets.GH_TOKEN }}

This workflow:

  1. Uses concurrency to prevent multiple production deployments
  2. Calls reusable workflows for each deployment stage
  3. Passes secrets to external workflows
  4. Uses outputs from one reusable workflow as inputs to another
  5. Won’t cancel deployments (cancel-in-progress: false)

What I Learned

About Reusable Workflows

  1. Reusable workflows save enormous time: Instead of copying YAML across repos, I write it once and call it everywhere.
  2. Version your reusable workflows: Using @main is convenient but risky. For production, pin to a specific tag or SHA. This prevents breaking changes:
  3. Document your inputs and secrets: When others use your workflows, they need to know what inputs are required and what secrets to pass. I learned this the hard way when a teammate couldn’t figure out what the workflow expects.
  4. Test reusable workflows independently: Add workflow_dispatch to your reusable workflows so you can test them standalone before integrating them. List every secret you need.
  5. Keep reusable workflows focused: One workflow should do one thing well. A workflow that builds, tests, deploys, and notifies is too complex to reuse. Break it into smaller workflows that can be composed together. A workflow that tries to do everything becomes hard to reuse because different callers need different subsets. Better to have small, focused workflows: one for building, one for testing, one for deploying. Then compose them as needed. Test the reusable workflow standalone:
  6. Don’t nest too deep: Maximum 4 levels is more than enough. If you’re hitting that limit, your architecture is probably too complex. Simplify and flatten your workflow dependencies between workflows.

About Concurrency

  1. Always use concurrency for deployments: Never let two deployments to the same environment run in parallel. This prevents race conditions and corrupted deployments.
  2. Choose cancel-in-progress wisely: Use true for fast operations where you only care about the latest code (builds, tests). Use false for critical operations that must complete (database migrations, deployments).
  3. Use descriptive group names: Include the workflow name and branch in your group names. This makes it clear what’s being locked and helps debug concurrency issues.
  4. Concurrency is not a queue: When cancel-in-progress is false, new runs wait. But if multiple runs are waiting, only the latest one will run after the current one finishes. The others get skipped.
  5. Test your concurrency behavior: Trigger multiple runs quickly and verify the behavior matches your expectations. I once had a deployment group that was too broad and it blocked unrelated workflows.

When to Use What

Use Custom Actions when:

  • Reusing steps within a job
  • The logic is small and focused
  • You need Docker or JavaScript capabilities
  • The reuse is within a single repository

Use Reusable Workflows when:

  • Reusing entire jobs or multi-job pipelines
  • Sharing workflows across repositories
  • The pipeline is complex with multiple stages
  • Example: Build → Test → Deploy → Verify

Use Concurrency when:

  • Preventing parallel runs to shared resources
  • Managing deployments to the same environment
  • Controlling resource usage (API limits, database locks)
  • Ensuring only the latest code gets deployed

$ ls -t root-access/ | head -3

Hello world! uncategorized 1 min Building a Job Board API, Part 1: The Foundation job-board-api · 1/1 16 min Part 13 – Kubernetes Autoscaling in Action: A FastAPI Microservice That Scales Itself home-lab · 13/13 6 min

Leave a comment

Your email address will not be published. Required fields are marked *