Skip to content

GitHub Action Integration

github-actions-logo

In this guide we'll show you how to set up a GitHub Actions Workflow so Mayhem can automatically test your Code or API on every push.

You will need the following to run Mayhem in your GitHub Action workflows:

Note: You can use the same token to test your Code or API.

  1. Create a Mayhem API token.
  2. Add the newly created token to your GitHub secrets using a variable named MAYHEM_TOKEN.

Workflow Configuration for Testing your Code

The Mayhem GitHub Action tests every change to your code by kicking off a CI workflow (below we show example usage through GHCR and Docker Hub), building and pushing a Docker image for your target application, automatically testing it, with a final security testing or SARIF report generated at the end of the workflow.

Getting Started

Integrating the GitHub Action into your GitHub repository is easy! If you're using either the GitHub Container Registry (GHCR) or Docker Hub as your image registry, follow the corresponding steps:

mCode Action using GHCR:

  1. Add the following to your GitHub Secrets for your GitHub repository.

    1. MAYHEM_TOKEN: Your Mayhem token.
    2. MAYHEM_URL: The URL of the Mayhem instance, such as https://app.mayhem.security.

    3. Ensure your repository's package visibility settings are set to Public to give Mayhem permissions to ingest your Docker image from the GitHub Container Registry.

      Click on your package in the right-hand pane of your GitHub repository and go to Package Settings. Then, scroll down to Package Visibility and set the package to Public.

  2. Create a Dockerfile at the root of the repository for your containerized application.

  3. Create a Mayhemfile at the root of the repository for configuring the Mayhem run on your containerized target application invoked by the mCode GitHub Action.
  4. Create a mayhem.yml workflow at .github/workflows/mayhem.yml that builds and pushes a Docker image and uses the mCode GitHub Action to fuzz your containerized application in a CI pipeline.

mCode Action using Docker Hub:

  1. Add the following to your GitHub Secrets for your GitHub repository.

    1. DOCKER_USERNAME: Your Docker Hub username (this is case-sensitive).

    2. DOCKER_PASSWORD: Your Docker Hub password.

      If you have 2FA enabled, you must set up an access token for your Docker Hub account. Specify the access token as your DOCKER_PASSWORD in your GitHub Secrets.

    3. MAYHEM_TOKEN: Your Mayhem token.

    4. MAYHEM_URL: The URL of the Mayhem instance, such as https://app.mayhem.security.

  2. Create an associated Docker Hub repository for your GitHub repository.

    For example, the GitHub repository mayhemuser/lighttpd-demo will require an associated Docker Hub repository named mayhemuser/lighttpd-demo, as the Docker image built and pushed to Docker Hub during the pipeline will be tagged according to the GitHub repository.

  3. Create a Dockerfile at the root of the repository for your containerized application.

  4. Create a Mayhemfile at the root of the repository for configuring the Mayhem run on your containerized target application invoked by the mCode GitHub Action.
  5. Create a mayhem.yml workflow at .github/workflows/mayhem.yml that builds and pushes a Docker image and uses the mCode GitHub Action to fuzz your containerized application in a CI pipeline.

Configuring the mCode Action

To use the mCode Action, you'll need to add a mayhem.yml file to the fully qualified path .github/workflows/mayhem.yml in your GitHub repository. This is the configuration file for the mCode GitHub action workflow.

Your mayhem.yml should look similar to the following:

Note

The following mayhem.yml is fully-qualified and displays both required and optional input parameters (required fields are highlighted). To understand more about the available input parameters to mayhem.yml, check out mCode Action Usage.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
name: Mayhem
on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
  workflow_dispatch:

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}
  BRANCH_NAME: ${{ github.head_ref || github.ref_name }}

jobs:
  build:
    name: 'build'
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest]
        shared: [false]
        build_type: [Release]
        include:
          - os: ubuntu-latest
            triplet: x64-linux

    steps:
      - uses: actions/checkout@v3
        with:
          submodules: recursive

      - name: Log in to the Container registry
        uses: docker/login-action@v2.1.0
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Extract metadata (tags, labels) for Docker
        id: meta
        uses: docker/metadata-action@v4.1.1
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}

      - name: Set lowercase image name
        run: |
          echo "IMAGE_NAME=${GITHUB_REPOSITORY,,}" >> ${GITHUB_ENV}

      - name: Build and push Docker image
        uses: docker/build-push-action@v3.2.0
        with:
          context: .
          push: true
          file: Dockerfile
          tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.BRANCH_NAME }}
          labels: ${{ steps.meta.outputs.labels }}

    outputs:
      image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.BRANCH_NAME }}

  mayhem:
    needs: build
    name: 'fuzz'
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        mayhemfile:
          - mayhem/Mayhemfile.lighttpd
          - mayhem/Mayhemfile.mayhemit
          # Specify one or many Mayhemfiles here

    steps:
      - uses: actions/checkout@v3

      - name: Start analysis for ${{ matrix.mayhemfile }}
        uses: ForAllSecure/mcode-action@v1
        with:
          mayhem-url: ${{ secrets.MAYHEM_URL }}
          mayhem-token: ${{ secrets.MAYHEM_TOKEN }}
          args: --image ${{ needs.build.outputs.image }} --file ${{ matrix.mayhemfile }} --duration 300
          sarif-output: sarif

      - name: Upload SARIF file(s)
        uses: github/codeql-action/upload-sarif@v2
        with:
          sarif_file: sarif

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
name: Mayhem
on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
  workflow_dispatch:

env:
  REGISTRY: docker.io
  IMAGE_NAME: ${{ github.repository }}
  BRANCH_NAME: ${{ github.head_ref || github.ref_name }}

jobs:
  build:
    name: 'build'
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest]
        shared: [false]
        build_type: [Release]
        include:
          - os: ubuntu-latest
            triplet: x64-linux
        mayhemfile:
          - mayhem/Mayhemfile
          # Specify one or many Mayhemfiles here

    steps:
      - uses: actions/checkout@v3

      - name: Set up Docker BuildX
        uses: docker/setup-buildx-action@v2.2.1

      - name: Docker meta
        id: meta
        uses: docker/metadata-action@v4.1.1
        with:
          images: ${{ github.repository }}

      - name: Log in to Docker Hub
        uses: docker/login-action@v2.1.0
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      - name: Build and push
        uses: docker/build-push-action@v3.2.0
        with:
          context: .
          push: true
          file: Dockerfile
          tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.BRANCH_NAME }}
          labels: ${{ steps.meta.outputs.labels }}

      - name: Start analysis for ${{ matrix.mayhemfile }}
        uses: ForAllSecure/mcode-action@v1
        with:
          mayhem-url: ${{ secrets.MAYHEM_URL }}
          mayhem-token: ${{ secrets.MAYHEM_TOKEN }}
          args: --image ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.BRANCH_NAME }} --file ${{ matrix.mayhemfile }} --duration 300
          sarif-output: sarif

      - name: Upload SARIF file(s)
        uses: github/codeql-action/upload-sarif@v2
        with:
          sarif_file: sarif

We can see from the mayhem.yml that the following GitHub Action plugins are executed in sequence for the pipeline runner node:

Info

  1. actions/checkout: Your code is checked out from your GitHub repository.
  2. docker/login-action: Logs into the GitHub Container Registry using your GitHub Secrets credentials.
  3. docker/metadata-action: Used to extract metadata from Git reference and GitHub events for tagging and labeling Docker images.
  4. docker/build-push-action: Your code is built and pushed to the GitHub Container Registry as a Docker image.
  5. ForAllSecure/mcode-action: Mayhem ingests your GitHub Container Registry image and begins fuzzing your containerized target applications.
  6. github/codeql-action/upload-sarif: A SARIF report is generated from fuzz testing results and uploaded to GitHub for your repository.

Info

  1. actions/checkout: Your code is checked out from your GitHub repository.
  2. docker/setup-buildx-action: Docker is installed.
  3. docker/metadata-action: Sets up the tag for the soon-to-be uploaded Docker Hub image.
  4. Log in to Docker Hub: Logs into Docker Hub using your GitHub Secrets credentials.
  5. Build and Push: Your code is built and pushed to Docker Hub as a Docker image.
  6. Mayhem Scan: Mayhem ingests your public Docker Hub image and begins fuzzing your containerized target applications.
  7. Upload SARIF file(s): A SARIF report is generated from fuzz testing results and uploaded to GitHub for your repository.

🤔 Still have questions? Take a look at the mcode-actions-examples repository to view working examples integrated with the mCode GitHub Action that you can use as a template. Feel free to fork the repository as needed!

Analyzing SARIF Reports

After Mayhem has fuzzed your containerized target application via the uploaded Docker image, issues that are found by Mayhem will be generated as a SARIF report and uploaded to GitHub, allowing you to view these results either in the pull request itself or in the Security tab of your repository.

For example, if we click Details for the required Mayhem code scanning check in the pull request, we can view additional info about what Mayhem has found.

pr-security-vulns

Otherwise, users can view security issues under Security > Code scanning alerts for their respective GitHub repositories.

security-tab

And if we click on one of the following code scanning alerts, we can view the specific commits, timelines, and branches of when these security issues were introduced or fixed.

code-scanning-alert

Generating SARIF Reports via Mayhem CLI

Alternatively, users have the option to generate SARIF reports via the Mayhem CLI. Users can invoke a mayhem run and mayhem wait --sarif <SARIF> combination to do this.

Info

Check out the Mayhem CLI Commands page for more information on the mayhem run and mayhem wait commands.

Once the SARIF report has been generated, users can upload a SARIF file to GitHub to then view the results in the Security tab of their GitHub repositories.

Workflow Configuration for Testing your API

With our GitHub Action, you'll get Mayhem testing every API change in no time.

At the base directory of your code repository, add a .github/workflows/mapi.yml file to configure GitHub Actions to run Mayhem. Your file should look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
name: Mayhem
on:
  push:
    branches: [main, master]
  pull_request:
jobs:
  security:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          # fetch a greater number of commits for computing diffs between jobs
          fetch-depth: 50

      - name: Start your API
        run: ./run_your_api.sh &
        #           ^-------------- update this, but keep the '&' to run
        #                           the API in the background.

      - name: Run Mayhem on your API to check for vulnerabilities
        uses: ForAllSecure/mapi-action@v1
        with:
          mapi-token: ${{ secrets.MAYHEM_TOKEN }}
          api-url: http://localhost:8000 # <- update this
          api-spec: your-openapi-spec-or-postman-collection.json # <- update this

This configuration tells GitHub Actions to run a workflow which checks out your code, starts your API, and then runs Mayhem against your API. We pass the Mayhem token from the GitHub secret, MAYHEM_TOKEN.

Here's a fully working example repository showing you an ideal integration that you can use as a template. For more details on the Mayhem Action, visit the MAPI Action in the GitHub Actions marketplace.

SARIF Reports and GitHub Code Scanning

If you are testing a public repository, or a private repository on a GitHub Enterprise plan, we recommend generating a SARIF report. Mayhem can generate SARIF files with the --sarif argument to mapi run. SARIF files are a great way to upload issues to GitHub, if you happen to use GitHub.

GitHub

Mayhem Issues in GitHub

SARIF files can also be loaded in VSCode using a SARIF extension from Microsoft:

GitHub

Mayhem Issues in VSCode

Info

To give accurate file and line information for each issue, Mayhem requires your API to include stacktraces in the body of "500 Internal Server Error" responses. Only do that in dev & testing -- never in production!

Uploading SARIF reports to GitHub allows you to see any issue found by Mayhem in the "Security" tab of your repository. In addition, if your API is sending back stacktraces as part of the error response body, Mayhem will show you exactly where in your code the issue happened, right in your PR diffs.

Using GitHub action

To do so with our GitHub Action, after starting your API in the workflow, run the Mayhem action followed by github/codeql-action/upload-sarif@v1 to upload the report.

Note

continue-on-error needs to be set to true in the Mayhem action in order to run the next step, even if Mayhem finds issues.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
- name: Run Mayhem to check for vulnerabilities
  uses: ForAllSecure/mapi-action@v1
  continue-on-error: true
  with:
    mapi-token: ${{ secrets.MAPI_TOKEN }}
    api-url: http://localhost:8000 # <- update this
    api-spec: your-openapi-spec-or-postman-collection.json # <- update this
    sarif-report: mapi.sarif

# Upload SARIF file (only available on public repos or github enterprise)
- name: Upload SARIF file
  uses: github/codeql-action/upload-sarif@v1
  with:
    sarif_file: mapi.sarif

Using our CLI

To upload our results to GitHub code scanning from anywhere, locally or from a CI, you can use our CLI. You'll need to pass a GitHub token with the security_event scope so that Mayhem can upload findings. See GitHub documentation on creating access tokens for more information. Once you have your token, simply run:

mapi run "<target>" "<duration>" "<spec>" --sarif "mapi.sarif" --github-token "<github-token>"

Mayhem will attempt to infer the git information from the environment set by the CI, and the git repository on the filesystem. If those are not available, Mayhem will ask you to pass the missing information so that it can upload the findings to the right place.

In addition, Mayhem also supports GitHub Enterprise on premise! Just pass --github-api-url <your-github-api-url> to mapi run so that Mayhem knows where to upload the results. Your GitHub instance needs to be accessible from the host running our CLI, but it does not need to be accessible by our cloud infrastructure.

Helpful Tips

The below sections are some helpful tips that we recommend (but are not required) and are intended to maximize your experience with the mCode GitHub Action.

Organizing Mayhem Assets in GitHub

When using the mCode GitHub Action, it's important to know how to structure your files in your GitHub repository so that the mCode GitHub Action can properly build, push, and test your containerized target application.

The following files are required in order to use the mCode GitHub Action:

  1. mayhem.yml: The configuration file for the mCode GitHub Action workflow. This must be located at .github/workflows/mayhem.yml.
  2. Dockerfile: The build file for creating a Docker image for the containerized target application.
  3. Mayhemfile: The configuration file for executing a Mayhem run for the specified containerized target.

However, overall we recommended the following file hierarchy for storing Mayhem assets in GitHub:

├── GitHub Repository
    └── .github
        └── workflows
            └── mayhem.yml
    └── README.md
    └── mayhem
         └── testsuite <-- The compilation of test cases for the target. Can be used to seed a Mayhem run.
         └── Mayhemfile.mayhemit-c
         └── Mayhemfile.lighttpd
    └── Dockerfile
    └── Mayhemit-c
    └── lighttpd

Note

We recommend structuring your repo file organization in such a manner, but of course, this will depend on your personal preference.

As an example, see the file structure of the mcode-action-examples repository:

mcode-action-examples

Adding Mayhem Project Badges in GitHub

Want to display a Mayhem project badge in your GitHub repository to quickly identify if Mayhem has found defects? Now you can do this in only a few easy steps! 🎉

To add a project badge to a GitHub repo, perform the following steps:

  1. Navigate to the Mayhem UI and go to Project at the top of the home page. Select an existing project and then go to Settings > General in the left nav-pane for the project.

    project-view

  2. Scroll down to the Project Badges section: a markdown link for a dynamically generated SVG badge and description about what the badge is should be available.

    project-badges

  3. Navigate to the GitHub repo where you want to add a project badge and edit the README.md file.

    commit-changes

  4. Paste the badge link from the previous step and confirm the badge is properly rendered using the Preview button.

    edit-readme

  5. Once you confirm the badge is properly rendered, commit the changes to the main / master branch.

    readme-preview

  6. And you’re done! You just added a Mayhem project badge to your GitHub repo!

Now you can display Mayhem project badges for your GitHub repositories! Use these project badges to quickly show information about your Mayhem project and its associated runs.

Protecting Branches and Setting Required Checks

Optionally, users can set required pull requests, approvals, and Mayhem required checks to better secure their code and prevent instances in which code changes with potential vulnerabilities are merged into the main or master branch of their GitHub repositories.

To set these protections, go to Settings > Branches for your respective GitHub repository and click on Add rule. Then check the following criteria:

  1. Require a pull request before merging
  2. Require approvals
    • Set the required number of approvals, default is 1.
  3. Require status checks
    • Search and set Mayhem as the required status check.

Note

You may need to run the mCode Action at least once for Mayhem to show up as a required status check.

Once configured, your branch protection settings should look like the following:

branch-protection
required-checks

Once these protections have been set, we can see how this functions within a pull request where proposed changes are intended to merge into the main branch. The mCode GitHub Action will test target applications for security vulnerabilities and prevent code changes from merging into the primary branch should any vulnerabilities be detected.

Info

The following screenshot shows a pull request whose proposed code changes have been found to have vulnerabilities. Therefore, a developer cannot merge to the main branch (as intended).

pr-checks