my blog

$ cat github-actions/part-03.md

Job Outputs, Caching & Artifacts in GitHub Actions

In my previous articles, I covered the basics and how to control workflows. Now I want to share how to pass data between jobs and make your workflows faster with caching.

The Problem: Jobs Don’t Share Data by Default

Each job runs on a separate runner. This means files and variables from one job don’t automatically exist in another job. Here’s how I solved this.

Job Outputs: Passing Data Between Jobs

Outputs let you send data from one job to another. Here’s my workflow that demonstrates this:

name: 12 - Working with Outputs

on:
  workflow_dispatch:
    inputs:
      build-status:
        type: choice
        options:
          - success
          - failure
        default: success

jobs:
  build:
    runs-on: ubuntu-latest
    outputs:
      build-status: ${{ steps.build.outputs.status }}
      output1: ${{ steps.build.outputs.output1 }}
    steps:
      - name: Print GITHUB_OUTPUT path
        run: echo "$GITHUB_OUTPUT"
      - name: Build
        id: build
        run: |
          echo "$GITHUB_OUTPUT"
          echo "status=${{ inputs.build-status }}" >> "$GITHUB_OUTPUT"
          echo "output1=value1" >> "$GITHUB_OUTPUT"
          echo "output2=value2" >> "$GITHUB_OUTPUT"
          cat "$GITHUB_OUTPUT"
      - name: Step with mistake
        run: |
          echo "mistake=true" > "$GITHUB_OUTPUT"
          cat "$GITHUB_OUTPUT"
  deploy:
    runs-on: ubuntu-latest
    needs: build
    if: ${{ needs.build.outputs.build-status == 'success' }}
    steps:
      - name: Deploy
        run: echo "Deploying"
      - name: Print Outputs
        run: |
          echo "Output 1: ${{ needs.build.outputs.output1 }}"

How It Works

Step 1: Set outputs in a step

- name: Build
  id: build  # You need this ID!
  run: |
    echo "status=${{ inputs.build-status }}" >> "$GITHUB_OUTPUT"
    echo "output1=value1" >> "$GITHUB_OUTPUT"

The $GITHUB_OUTPUT is a special file. When you write to it, GitHub captures those values.

Step 2: Expose step outputs at job level

jobs:
  build:
    outputs:
      build-status: ${{ steps.build.outputs.status }}
      output1: ${{ steps.build.outputs.output1 }}

Notice I only exposed two outputs. output2 exists in the step but not at job level.

Step 3: Use outputs in another job

deploy:
  needs: build
  if: ${{ needs.build.outputs.build-status == 'success' }}
  steps:
    - name: Print Outputs
      run: echo "Output 1: ${{ needs.build.outputs.output1 }}"

Access outputs using needs.JOB_NAME.outputs.OUTPUT_NAME.

A Common Mistake I Made

Look at this step in my workflow:

- name: Step with mistake
  run: |
    echo "mistake=true" > "$GITHUB_OUTPUT"  # Using > instead of >>
    cat "$GITHUB_OUTPUT"

I used > instead of >>. The single > overwrites the file, removing all previous outputs! Always use >> to append.

Why This Is Useful

I use job outputs for:

  • Conditional deployments: Only deploy if build succeeds
  • Version numbers: Generate version in one job, use it everywhere
  • Environment info: Calculate which environment to deploy to
  • Cache keys: Generate cache keys in one place, use them in multiple jobs

Caching: Speed Up Your Workflows

Caching saves files between workflow runs. This is huge for Node.js projects where npm install takes forever.

Here’s my caching workflow for a React app:

name: 13 - Using Caching

on:
  workflow_dispatch:
    inputs:
      use-cache:
        description: Whether to execute cache step
        type: boolean
        default: true
      node-version:
        description: Node version
        type: choice
        options:
          - 18.x
          - 20.x
          - 21.x
        default: 20.x

jobs:
  install-deps:
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: react-app
    outputs:
      deps-cache-key: ${{ steps.cache-key.outputs.CACHE_KEY }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: ${{ inputs.node-version }}
      - name: Calculate cache key
        id: cache-key
        run: |
          echo "CACHE_KEY=deps-node-modules-${{ hashFiles('react-app/package-lock.json') }}" >> "$GITHUB_OUTPUT"
      - name: Download cached dependencies
        uses: actions/cache@v3
        if: ${{ inputs.use-cache }}
        id: cache
        with:
          path: react-app/node_modules
          key: ${{ steps.cache-key.outputs.CACHE_KEY }}
      - name: Install dependencies
        if: steps.cache.outputs.cache-hit != 'true'
        run: npm ci

Understanding the Cache Key

The most important part is the cache key:

- name: Calculate cache key
  id: cache-key
  run: |
    echo "CACHE_KEY=deps-node-modules-${{ hashFiles('react-app/package-lock.json') }}" >> "$GITHUB_OUTPUT"

What’s happening:

  • hashFiles() creates a hash of package-lock.json
  • If package-lock.json changes, the hash changes
  • New hash = new cache key = cache miss = fresh install
  • Same hash = cache hit = skip install

Using the Cache

- name: Download cached dependencies
  uses: actions/cache@v3
  id: cache
  with:
    path: react-app/node_modules
    key: ${{ steps.cache-key.outputs.CACHE_KEY }}

- name: Install dependencies
  if: steps.cache.outputs.cache-hit != 'true'
  run: npm ci

The actions/cache@v3 action:

  • Tries to find a cache with that key
  • Sets cache-hit output to ‘true’ if found
  • If cache-hit is not ‘true’, we run npm ci

Reusing Cache in Other Jobs

Here’s how I use the same cache in multiple jobs:

jobs:
  lint-test:
    runs-on: ubuntu-latest
    needs: install-deps
    defaults:
      run:
        working-directory: react-app
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: ${{ inputs.node-version }}
      - name: Download cached dependencies
        uses: actions/cache@v3
        id: cache
        with:
          path: react-app/node_modules
          key: ${{ needs.install-deps.outputs.deps-cache-key }}
      - name: Testing
        run: npm run test
      - name: Linting
        run: echo "Linting..."
  build:
    runs-on: ubuntu-latest
    needs: install-deps
    defaults:
      run:
        working-directory: react-app
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: ${{ inputs.node-version }}
      - name: Download cached dependencies
        uses: actions/cache@v3
        id: cache
        with:
          path: react-app/node_modules
          key: ${{ needs.install-deps.outputs.deps-cache-key }}
      - name: Building
        run: npm run build

Both jobs use the same cache key from install-deps job output. This means:

  1. First job installs and caches dependencies
  2. Other jobs download the cache
  3. No reinstalling = faster workflows

Cache Limitations I Found

Cache size: Maximum 10GB per repository. GitHub deletes old caches automatically.

Cache scope: Caches are scoped to branches. A cache created on main branch is not available on feature-branch.

Cache is immutable: You can’t update a cache. You create a new one with a new key.

Artifacts: Sharing Files Between Jobs

Artifacts are files you want to keep after workflow finishes. Unlike cache, artifacts:

  • Survive after workflow completes
  • Can be downloaded from GitHub UI
  • Are meant for build outputs, test results, logs

Here’s my complete workflow:

name: 14 - Working with Artifacts

on: workflow_dispatch

env:
  build-artifact-key: app-${{ github.sha }}
  test-coverage-key: test-coverage-${{ github.sha }}

jobs:
  test-build:
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: react-app
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: '20.x'
      - name: Download cached dependencies
        uses: actions/cache@v3
        id: cache
        with:
          path: react-app/node_modules
          key: deps-node-modules-${{ hashFiles('react-app/package-lock.json') }}
      - name: Install dependencies
        if: steps.cache.outputs.cache-hit != 'true'
        run: npm ci
      - name: Unit tests
        run: npm run test -- --coverage
      - name: Upload test results
        uses: actions/upload-artifact@v4
        with:
          name: ${{ env.test-coverage-key }}
          path: react-app/coverage
      - name: Build code
        run: npm run build
      - name: Upload build files
        uses: actions/upload-artifact@v4
        with:
          name: ${{ env.build-artifact-key }}
          path: react-app/build
  deploy:
    runs-on: ubuntu-latest
    needs: test-build
    steps:
      - name: Download build artifact
        uses: actions/download-artifact@v4
        with:
          name: ${{ env.build-artifact-key }}
          path: build
      - name: Show folder structure
        run: ls -R

Uploading Artifacts

- name: Upload build files
  uses: actions/upload-artifact@v4
  with:
    name: ${{ env.build-artifact-key }}
    path: react-app/build

This uploads everything in react-app/build directory with a unique name using commit SHA.

Downloading Artifacts

- name: Download build artifact
  uses: actions/download-artifact@v4
  with:
    name: ${{ env.build-artifact-key }}
    path: build

Downloads the artifact to build directory in the deploy job.

Using github.sha for Unique Names

env:
  build-artifact-key: app-${{ github.sha }}
  test-coverage-key: test-coverage-${{ github.sha }}

I use github.sha (commit hash) to make artifact names unique. Each commit gets its own artifacts.

Cache vs Artifacts: When to Use What

Use Cache for:

  • Dependencies (node_modules, pip packages)
  • Things you can regenerate
  • Speeding up workflows
  • Data you don’t need after workflow finishes

Use Artifacts for:

  • Build outputs (compiled code, docker images)
  • Test results and coverage reports
  • Logs and debug information
  • Files you want to download from GitHub UI
  • Data you need to deploy

Real-World Example: Combining Everything

Here’s how I use all three concepts together:

name: Complete CI/CD Pipeline

on:
  push:
    branches: [main]

env:
  NODE_VERSION: '20.x'

jobs:
  install-and-cache:
    runs-on: ubuntu-latest
    outputs:
      cache-key: ${{ steps.cache-key.outputs.key }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ env.NODE_VERSION }}
      - name: Generate cache key
        id: cache-key
        run: echo "key=deps-${{ hashFiles('**/package-lock.json') }}" >> "$GITHUB_OUTPUT"
      - uses: actions/cache@v3
        id: cache
        with:
          path: node_modules
          key: ${{ steps.cache-key.outputs.key }}
      - if: steps.cache.outputs.cache-hit != 'true'
        run: npm ci
  test:
    needs: install-and-cache
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ env.NODE_VERSION }}
      - uses: actions/cache@v3
        with:
          path: node_modules
          key: ${{ needs.install-and-cache.outputs.cache-key }}
      - run: npm run test -- --coverage
      - uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ github.sha }}
          path: coverage
  build:
    needs: [install-and-cache, test]
    runs-on: ubuntu-latest
    outputs:
      build-success: ${{ steps.build.outputs.success }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ env.NODE_VERSION }}
      - uses: actions/cache@v3
        with:
          path: node_modules
          key: ${{ needs.install-and-cache.outputs.cache-key }}
      - name: Build
        id: build
        run: |
          npm run build
          echo "success=true" >> "$GITHUB_OUTPUT"
      - uses: actions/upload-artifact@v4
        with:
          name: build-${{ github.sha }}
          path: build
  deploy:
    needs: build
    if: needs.build.outputs.build-success == 'true'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/download-artifact@v4
        with:
          name: build-${{ github.sha }}
          path: dist
      - name: Deploy
        run: echo "Deploying files from dist/"

This workflow:

  1. Installs and caches dependencies once
  2. Test and build jobs reuse the cache
  3. Test results are saved as artifacts
  4. Build output is saved as artifacts
  5. Deploy job downloads build artifacts
  6. Deploy only runs if build succeeded (using job outputs)

What I Learned

  1. Always use >> for GITHUB_OUTPUT: Using > will delete previous outputs.
  2. Job outputs don’t need to match step outputs: You can pick which step outputs to expose at job level.
  3. Cache keys are critical: Use hashFiles() to make cache keys change when dependencies change.
  4. Cache early in your workflow: Install dependencies in the first job, cache them, and all other jobs can use that cache.
  5. Artifacts cost storage: GitHub gives you free storage, but artifacts count against it. They expire after 90 days by default.
  6. Use github.sha for uniqueness: It’s perfect for creating unique artifact names for each commit.
  7. Working directory matters: When using actions/cache, make sure the path is correct. I use defaults.run.working-directory to avoid mistakes.

$ ls -t root-access/ | head -3

Hello world! uncategorized 1 min Building a Job Board API, Part 1: The Foundation job-board-api · 1/1 16 min Part 13 – Kubernetes Autoscaling in Action: A FastAPI Microservice That Scales Itself home-lab · 13/13 6 min

Leave a comment

Your email address will not be published. Required fields are marked *