Skip to content

Mayhemfile Configuration

When executing a Mayhem run to analyze a target application, Mayhem uses a configuration file (called a Mayhemfile) that designates the required and optional parameters a user can specify for testing their custom targets.

With simple applications, users can use the mayhem package command to automatically generate a Mayhemfile for their target application; however, for more complex software, oftentimes users will need to specify additional customizations to their Mayhemfile in order to fuzz their targets correctly. Therefore, we've included the following information to help you configure your Mayhemfile for your Mayhem runs.

Mayhemfile Source vs. Run

For debugging purposes, users should be aware of the difference between a Mayhemfile that a user creates/uploads to Mayhem and the fully qualified Mayhemfile that Mayhem actually uses for a Mayhem run.

In particular, Mayhem will add final changes to a user-uploaded Mayhemfile by adding, modifying, or removing certain fields to build out the full configuration used for executing a Mayhem run. Mayhem will adjust a user-uploaded "source" Mayhemfile to take into account the following:

  • Environment Variables: Variables within the Mayhemfile, such as ${MAYHEM_PROJECT} that are substituted with the value of the user's corresponding environment variable (if available).
  • CLI Overrides: Values within the Mayhemfile that are overridden as a result of executing a Mayhem run via the Mayhem CLI with specific parameters/flags.
  • Default Values: Default parameters and values that are automatically added to the Mayhemfile if no custom values are specified.

Environment Variables

Take a look at the following source Mayhemfile:

1
2
3
4
project: ${FOO}
target: ${BAR}
cmds:
  -  cmd: /bin/grep @@

Here we see that the ${FOO} and ${BAR} environment variables have been set.

Note

Users can use either syntax when setting environment variables within their Mayhemfiles: ${FOO} or $FOO, which are equivalent.

Assuming ${FOO} is set to foo and ${BAR} is set to bar for the user's environment, executing the mayhem run command will ask you for the following environment variable interpolation prompt. Type "Y" to proceed.

$ export FOO=foo
$ export BAR=bar
$ mayhem run .
Performing environment variable interpolation based on the following values from the local environment:
    FOO
    BAR
Proceed? [Y/n]? Y
Run started: foo/bar/1
Run URL: https://tutorial.forallsecure.com:443/mayhemuser/foo/bar/1

Tip

If you'd prefer to skip prompts, you can use the optional -y or --noninteractive arguments for Mayhem CLI commands such as mayhem run. For example mayhem -y run . will execute a Mayhem run for a Mayhemfile in the current directory in non-interactive mode.

The run should execute, and if we check the Mayhemfile for this run, we see that the ${FOO} and ${BAR} environment variables have been substituted with the correct values.

1
2
3
4
project: foo
target: bar
cmds:
  -  cmd: /bin/grep @@

Info

You can escape characters in your source Mayhemfile by using a backslash \. For example, if you had a target literally called $TARGET, you would escape the $ by using a backslash:

# Source Mayhemfile
project: foo
target: bar
cmds:
  -  cmd: \$TARGET @@

Which would result in the following:

# Run Mayhemfile
project: foo
target: bar
cmds:
  -  cmd: $TARGET @@

CLI Overrides

Users can specify the following override parameters from the Mayhem CLI when executing a mayhem run.

-h, --help            Show this help message and exit.
--regression          Run regression tests on available test cases.
--static              Run static checks on the entrypoint of the target.
--dynamic             Run dynamic analysis on the target.
--coverage            Perform coverage analysis on the target.
--all                 Enable all supported analyses.
-f FILE, --file FILE  Path to the Mayhemfile used (default:
                      <package>/Mayhemfile).
-b BUILD_ID, --build-id BUILD_ID
                      Build id to associate with this specific run.
--docker              Indicates whether the package is a docker image tag or
                      hash.
--warning-as-error    Have the warnings be treated as errors.
--testsuite TESTSUITE Specify a testsuite directory.
--project PROJECT     Name of the project.
--owner OWNER         The owner for this project.
--target TARGET       Name of the target.
--image IMAGE         Docker image you want to analyze.
--duration DURATION   How long to run for in seconds (wall clock time).
--uid UID             User id for running the target.
--gid GID             Group id for running the target.
--advanced-triage ADVANCED_TRIAGE
                      Set extra advanced triage analysis. Increases the
                      number of CWEs Mayhem finds but also increases test
                      case processing time.
--cmd CMD             Command to invoke the target.
--cwd CWD             Current working directory for running the target.
--env KEY=VALUE       Environment variable to include while running.
--filepath FILEPATH   Input file path where the target reads from.
--network-url NETWORK_URL
                      Network URI where the target reads from.
--network-timeout NETWORK_TIMEOUT
                      The time (in seconds) for Mayhem to wait for the
                      target to accept network input.
--network-client NETWORK_CLIENT
                      Specify whether this network target is a client or
                      server.
--libfuzzer LIBFUZZER
                      Specify whether this is a libfuzzer target or not.
--honggfuzz HONGGFUZZ
                      Specify whether this is a honggfuzz target or not.
--sanitizer SANITIZER
                      Specify whether sanitization is compiled in or not.
--max-length MAX_LENGTH
                      Maximum length for test cases.
--memory-limit MEMORY_LIMIT
                      How much memory to allow the target in megabytes (MB).
--url URL             URL to running Mayhem API.
--token TOKEN         Authentication token for accessing Mayhem API.
-k, --insecure        Disable SSL verification.
--cacert CACERT       Path to the mayhem server's certificate.
--timeout TIMEOUT     Seconds to wait for API responses (useful for slow
                      connections).

For example, given the following source Mayhemfile:

1
2
3
4
5
project: foo
target: bar
image: mock_image
cmds:
-  cmd: /bin/grep @@

The subsequent Mayhem CLI command mayhem run . --image forallsecure/tutorial (executed within the current directory as the above Mayhemfile) will override the image parameter with the forallsecure/tutorial image from Docker Hub, producing the following Mayhemfile for the run:

1
2
3
4
5
project: foo
target: bar
image: forallsecure/tutorial
cmds:
-  cmd: /bin/grep @@

Info

Override parameters will either modify or add parameters depending on whether the override key already exists in the source Mayhemfile.

Default Values

Mayhem will ensure the following default values are added to a Mayhem run Mayhemfile if a user does not specify a custom value for one of the following parameters:

1
2
3
4
5
6
7
8
9
testsuite:
  - https://${MAYHEM_DOMAIN}/${MAYHEM_PROJECT}/${MAYHEM_TARGET}/testsuite.tar
  - https://${MAYHEM_DOMAIN}/${MAYHEM_PROJECT}/${MAYHEM_TARGET}/uploaded/<hash>/testsuite.tgz
image: ${MAYHEM_DOCKER_REGISTRY}/forallsecure/debian-buster:latest
advanced_triage: false
tasks:
  - name: exploitability_factors
  - name: regression_testing
  - name: behavior_testing

In order to construct the default values above, Mayhem will substitute the values of the following in-built values, which are auto-populated if not specified as custom environment variables. These are substituted in various places such as the testsuite and image values:

  • ${MAYHEM_DOCKER_REGISTRY}: Set as the value of the Mayhem Docker Registry URL for the Mayhem instance executing the run (e.g. tutorial.forallsecure.com:5000).
  • ${MAYHEM_DOMAIN}: Set as the value of the URL for the Mayhem instance executing the run (e.g. tutorial.forallsecure.com).
  • ${MAYHEM_PROJECT}: Set as the value derived from the required project field in the Mayhemfile with the name of the user or organization prepended to it (for example forallsecure/project1).
  • ${MAYHEM_TARGET}: Set as the value derived from the required target field in the Mayhemfile.
  • ${MAYHEM_URL}: Set as the value based on the following syntax: https://${MAYHEM_DOMAIN} (e.g. https://tutorial.forallsecure.com).
  • ${MAYHEM_OWNER}: Set as the value for either the owner provided as a part of the project field in the Mayhemfile (e.g. project: forallsecure/project1), or the username of the logged in Mayhem user. In this case, the owner would be set to forallsecure.

For example, if the user mayhemuser uploaded the following Mayhemfile when executing a Mayhem run:

1
2
3
4
project: foo
target: bar
cmds:
  -  cmd: /bin/grep @@

Mayhem would automatically add the following defaults (and substitute the in-built variables accordingly) to the Mayhemfile for the resulting run:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
project: foo
target: bar
cmds:
  - cmd: /bin/grep @@
    env: {}
testsuite:
  - https://tutorial.forallsecure.com/mayhemuser/foo/bar/testsuite.tar
  - https://tutorial.forallsecure.com/mayhemuser/foo/bar/uploaded/f3364b2ee694e27d558aa38ce325b624a1f11e8718da0d5b1ff16419a61eeb3e/testsuite.tgz
image: $MAYHEM_DOCKER_REGISTRY/forallsecure/debian-buster:latest
advanced_triage: false
tasks:
  - name: exploitability_factors
  - name: regression_testing
  - name: behavior_testing

Mayhemfile Examples

Mayhem can fuzz target applications via file input, standard input, or TCP/UDP input. This section includes some examples of the different types of Mayhemfile configurations.

File Input

Simple single input on command-line (from objdump):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
# Project name that the target belongs to
project: objdump

# Target name (should be unique within the project)
target: objdump

# Base image to run the binary in.
image: forallsecure/debian-buster:latest

# Time that analysis will run for in seconds - if absent, run forever
duration: 90

# List of commands used to test the target
cmds:

  # Command used to start the target, "@@" is the input file
  - cmd: /usr/bin/objdump -x @@

Standard Input (stdin)

Running forever on djpeg while accepting input from /dev/stdin:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
# Mayhemfile: configuration file for testing your target with Mayhem
# Format: YAML 1.1

# Project name that the target belongs to
project: libjpeg

# Target name (should be unique within the project)
target: djpeg

# Base image to run the binary in.
image: forallsecure/debian-buster:latest

# Commented out duration to run forever
# duration: 90

# List of commands used to test the target
cmds:

  # Command used to start the target. "@@" is missing so
  # input will be read from /dev/stdin.
  - cmd: /usr/bin/djpeg

TCP/UDP Input

Simple single input from network on TCP port 80 using a public docker hub image (from sebp/lighttpd):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
project: lighttpd-docker
target: lighttpd
image: sebp/lighttpd
duration: 90
cmds:
  - cmd: /usr/sbin/lighttpd -D -f /etc/lighttpd/lighttpd.conf
    network:
      client: false
      url: tcp://127.0.0.1:80
      timeout: 1.0

Mayhemfile Parameters

Below are all of the available parameters to the Mayhemfile.

Important

There are multiple levels of Mayhemfile parameters, therefore be cognizant of the parameter hierarchy.

version

Specifies the version of Mayhem.

  • Optional
  • Datatype: string
  • Example:

    version: '2.5'
    

    Info

    Mayhemfile versions can be set to 0.4 and above. Note, however, that Mayhemfiles that specify earlier versions of Mayhem may not include the functionality of more up-to-date versions.

project

The unique name of the project organization and project the target belongs to.

  • Owner (Optional), Projectname (Optional)
  • Datatype: string
  • Example:

    project: owner/projectname
    

target

The name of the target you are testing. A single target could have multiple targets.

  • Optional
  • Datatype: string
  • Example:

    target: targetname
    

image

Warning

Formerly baseimage, this field is now image in mCode version 2.X.

image is the name of the Docker image that will be used. The image can contain the targets (if using Docker packaging) or they may be contained in a root/ folder, in which case they are overlayed into the image (see mayhem package command). If image is not specified, ForAllSecure’s default Docker image based on Debian buster will be used: $MAYHEM_DOCKER_REGISTRY/forallsecure/debian-buster. Docker images can be uploaded to Mayhem via an internal docker registry or through the public Docker Hub registry.

Info

To more easily refer to Docker images that have been uploaded to Mayhem, the $MAYHEM_DOCKER_REGISTRY variable can be used to reference the location of the server’s internal docker registry. Mayhem comes with ForAllSecure's default image preinstalled.

  • Optional
  • Datatype: string
  • Default: $MAYHEM_DOCKER_REGISTRY/forallsecure/debian-buster
  • Example:

    image: $MAYHEM_DOCKER_REGISTRY/forallsecure/debian-buster
    

duration

Time in seconds to run the application.

Info

If the duration parameter is set to null (i.e. no default duration was specified) the duration will have an unlimited timeframe.

  • Optional
  • Datatype: integer
  • Default: null
  • Example:

    duration: 90
    

advanced_triage

Runs additional analysis via extra test case processing to find defects that do not raise a UNIX signal, such as memory leaks or using an uninitialized variable.

Info

Due to the nature of adding extra test case processing, enabling advanced_triage may reduce the performance of the overall Mayhem run. In contrast, normal triage runs can be up to 10x faster than runs with advanced_triage enabled.

  • Optional
  • Datatype: boolean
  • Default: false
  • Example:

    advanced_triage: false
    

tasks

Sets the tasks that will be executed for the Mayhem run. The tasks parameter can be set to include the following tasks:

  1. Exploitability Factors
  2. Regression Testing
  3. Behavior Testing
  4. Coverage Analysis
  • Optional
  • Datatype: object
  • Default:
    tasks:
        - name: exploitability_factors
        - name: regression_testing
        - name: behavior_testing
    
  • Examples:

    tasks:
        - name: exploitability_factors
        - name: regression_testing
        - name: behavior_testing
        - name: coverage_analysis
    

Note

Executing the mayhem run command with task parameters will override the tasks key in the Mayhemfile. For example, mayhem run --all . will execute all tasks for a Mayhem run, even if the tasks key in the Mayhemfile is only set to behavior_testing.

testsuite

Note

Formerly corpus, this field is now testsuite in mCode version 2.X.

Defines the location of the initial test suite for a Mayhem run, and is treated as such:

  • If the element begins with file://, it is a local file source, representing a file or directory on the same system as the CLI starting the run.

    • The path can be a relative path (file://foo for example). If so, this file is resolved relative to the directory containing the Mayhemfile.
    • The path can be an absolute path (file:///, three slashes indicating path starts with a leading slash).
  • If your Mayhemfile specifies a Docker image with a /corpus or /testsuite folder in the root of the image filesystem, then the contents of these folders will be added to the initial test suite.

  • Optional

  • Datatype: list of strings
  • Default:

    • https://$MAYHEM_DOMAIN/$MAYHEM_PROJECT/$MAYHEM_TARGET/testsuite.tar
    • file://testsuite

    Info

    https://$MAYHEM_DOMAIN/$MAYHEM_PROJECT/$MAYHEM_TARGET/testsuite.tar specifies a URL pointing to a testsuite.tar file stored on a Mayhem instance for a given project and target (if available). This ensures test cases from previous Mayhem runs are carried over into the next Mayhem run for a given project and target.

  • Example:

    testsuite:
      - file://testsuite
    

uid

An optional numeric UNIX user id to use as the target uid. If unset, an unprivileged UNIX user id will be used that could cause issues accessing files. If a uid other than 0 is used, the target will generally not be able to perform privileged operations such as bind to a port lower than 1024. Docker defaults this to 0, although it can be overridden by the USER command. If using the default base image, this is 1000.

  • Optional
  • Datatype: integer
  • Default: derived from image
  • Example:

    uid: 0
    

gid

An optional numeric UNIX group id to use as the target gid. If unset, an unprivileged UNIX group id will be used that could cause issues accessing files. If unspecified, the gid is inherited from the base image. Docker defaults this to 0, although it can be overridden by the USER command. If using the default base image, this is 1000.

  • Optional
  • Datatype: integer
  • Default: derived from image
  • Example:

    gid: 0
    

cmds

A list of cmd objects to specify the different ways to test the packaged application. Each cmd object tells Mayhem how to analyze the target executable. Each cmd object contains the following optional fields that should only be used if the target requires additional information or changes to its environment prior to execution.

  • Required
  • Datatype: dictionary

cmd

Invocation of the target executable.

If you omit the @@ identifier from the command line and do not specify the target as a networking one, Mayhem will assume that standard input (/dev/stdin) is the main input source.

Note

If you need to manipulate the fuzz data in the @@ variable as a filesystem file, look towards the filepath parameter to accomplish this use case. The cmd contents are executed directly and are not passed through the bash interpreter.

  • Required
  • Datatype: string
  • Example One:

    cmd: /usr/bin/objdump -x @@
    
  • Example Two:

    - cmd: /host/tinyxml2-2.0.1/parse-fuzzer
    libfuzzer: true
    - cmd: /host/tinyxml2-2.0.1/orig-collapse @@
    

    Info

    It is possible to run multiple cmd parameters in a single Mayhemfile for when it is desired to run a target multiple times using differing instances of instrumentation. For example, a target that runs both a libFuzzer harness cmd and an uninstrumented/base-executable cmd that reads in a file and calls the same code. This would be beneficial as symoblic execution does not support libFuzzer, but test cases that it finds in the uninstrumented/base-executable binary can be shared with the libFuzzer run and help discover new coverage.

max_length

Maximum length (in bytes) that will be provided as input to the fuzz target and should not exceed 100MB, otherwise the Mayhemfile will fail to validate. Test cases in the starting tests larger than the specified maximum length will be truncated to the value set.

Info

If a user uses Mayhemfile V1.10 and does not provide a max_length field explicitly in Mayhemfile, it is by default 8192 bytes. Otherwise for Mayhemfile V1.9 and below, the default will be 8192 bytes for libFuzzer targets and no default for uninstrumented targets.

  • Optional
  • Datatype: integer
  • Default: 8192
  • Example:

    max_length: 8192
    

cwd

Warning

Formerly chdir, this field is now cwd in mCode version 2.X.

The path Mayhem should switch to prior to executing the target. (Also known as the current working directory - CWD)

  • Optional
  • Datatype: string
  • Default: derived from image
  • Example:

    cwd: /usr/bin
    

filepath

Warning

Formerly target_path, this field is now filepath in mCode version 2.X.

The name that Mayhem should give the input file when it is needed to manipulate the fuzz data in the @@ variable as a filesystem file.

  • Optional
  • Datatype: string
  • Default: null
  • Example:

    filepath: /tmp/my_custom_file
    

env

Specify the environment variables which will be set when your program is run. Additionally, special environment variables can be specified if your program is not compatible with some performance enhancements used by Mayhem. While this is often not necessary, some multi-threaded programs can be problematic to analyze.

Special environment variables for use during a Mayhem run are as follows:

Environment Variable Value Description
DISABLE_SMOKETEST 1 Setting 1 disables smoke testing, where Mayhem will perform quick sanity checks to make sure that a target works, and if not, detects and reports common misconfigurations that require user intervention in the package/Mayhemfile.
MAYHEM_AUTORESOLUTION true or false Setting true enables Mayhem auto-resolution, where Mayhem will attempt to auto-resolve Mayhemfile configuration issues that may arise during the Mayhem run. Setting false disables Mayhem auto-resolution.
MFUZZ_COMPAT_LEVEL 1 or 2 Setting 1 disables some of Mayhem's performance optimizations (specifically deferred execution of the fork server). Setting 2 completely disables optimizations that may interfere with multithreaded programs.
MFUZZ_JAVA_MAIN_CLASS e.g. com.forallsecure.FuzzMe Sets the main class for the specified JAR file.
MFUZZ_JAVA 1 Setting 1 instructs Mayhem to fuzz a Java target.
MFUZZ_CUSTOM_MUTATOR e.g. /libmutator.so Sets the shared object for custom mutator support.
MFUZZ_DOCKER_ENTRYPOINT 1 Setting 1 instructs Mayhem to run an available docker-entrypoint.sh script for a Docker image target.
MAYHEM_TIMEOUTS_AS_DEFECTS 1 Setting 1 instructs Mayhem to treat test case timeouts as defects and report them under the Defects/Test Case pane in the Mayhem UI run page. The CWE for these defects will be 407 Inefficient Algorithmic Complexity.

Info

The value format for the env parameter can either be in flow or block YAML format, and supports multiple environment variables.

  • Optional
  • Datatype: object
  • Example One:

    env: { "ENV_VAR1": "value1", "ENV_VAR2": "value2", "ENV_VAR3": "value3" }
    
  • Example Two:

    env:
      ENV_VAR1: value1
      ENV_VAR2: value2
      ENV_VAR3: value3
    
  • Default: derived from environment variables already present in the Docker image specified by the image parameter.

network

Signify the target is a network target

  • Required if the target will receive input over a network
  • Datatype: string
  • Default: {}
url

Specify the protocol, host, and port to analyze. Supported protocols are: "tcp", "udp", "unix+stream", "unix+dgram" and "unix+seqpacket"

For Mayhem, localhost means any local interface, IPv4 or IPv6 and will attempt to perform auto-detection on which protocol should be tested. If you need to use specifically one or the other, you may instead write 127.0.0.1 for IPv4 or [::1] for IPv6. For example tcp://127.0.0.1:80 will feed input to an IPv4 connection to port 80.

Make sure you are aware how your target is communicating. In particular, using localhost inside of a container may require a hostname lookup for localhost which may not resolve to what you expect. If possible, stick to an IP address for binding and connecting in internal target configurations to avoid ambiguity.

  • Required if the target will receive input over a network
  • Datatype: string
  • Default: empty
  • Example:

    url: tcp://localhost:8080
    
timeout

The time (in seconds) for Mayhem to wait for the target to accept network input during triage. If the target does not accept network input within this time limit, the testcase will be discarded. If the time is too low, the testcase may report back as normal. Targets that take a while to start may need a longer timeout.

Note

This field refers to the timeout for network targets and is different from the timeout parameter for non-network targets.

  • Optional
  • Datatype: number
  • Default: 2
  • Example:

    timeout: 10
    
client

Warning

Formerly is_client, this field is now client in mCode version 2.X.

Set to 'true' if the executable to be tested is a client. A client is expected to initiate the network connection, while a server is expected to wait for Mayhem.

  • Optional
  • Datatype: boolean
  • Default: false
  • Example:

    client: true
    

timeout

The time (in seconds) for Mayhem to wait for a single test to complete.

Info

The max timeout allowed is 60 seconds unless overridden by the deployment/install-time environment variable MAX_CMD_TIMEOUT_SECONDS to increase the max timeout limit further. Values greater than the set maximum (either 60 seconds by default or specified by MAX_CMD_TIMEOUT_SECONDS) will be automatically downsized to the maximum timeout limit. Contact your administrator to set the MAX_CMD_TIMEOUT_SECONDS environment variable if needed.

  • Optional:
  • Datatype: number
  • Default: automatic
  • Example:

    timeout: 1.5
    

memory_limit

Warning

Formerly memory_limit_in_mb, this field is now memory_limit in mCode version 2.X.

The maximum amount of memory (in MB) to allow the target to allocate. Mayhem will kill a test run that goes above this memory limit and reports it as a crash.

  • Optional
  • Datatype: integer
  • Default: automatic
  • Example:

    memory_limit: 8192
    

sanitizer

Set to true only if the target executable is compiled with a sanitizer

  • Optional
  • Datatype: boolean
  • Default: automatic
  • Example:

    sanitizer: false
    

libfuzzer

Set to true only if the target executable is compiled with libFuzzer.

  • Optional
  • Datatype: boolean
  • Default: automatic
  • Example:

    libfuzzer: false
    

afl

Set to true only if the target executable is compiled with AFL/AFL++ instrumentation.

  • Optional
  • Datatype: boolean
  • Default: automatic
  • Example:

    afl: false
    

honggfuzz

Set to true only if the target executable is compiled with Honggfuzz.

  • Optional
  • Datatype: boolean
  • Default: automatic
  • Example:

    honggfuzz: false
    

gcov

Set to true only if the target executable is compiled with gcov support. Also, gcno files should be located in the same path as the target executable.

  • Optional
  • Datatype: boolean
  • Default: false
  • Example:

    gcov: true
    

sancov

Set to true only if the target executable is compiled with sancov support.

  • Optional
  • Datatype: boolean
  • Default: false
  • Example:

    sancov: true
    

dictionary

The path to an AFL/AFL++-compatible dictionary file. The path should be relative to the root of the container. For instance, if the dictionary is under /etc/foo/bar/http.dict inside the image, the dictionary path in the Mayhemfile should be /etc/foo/bar/http.dict.

Tip

Take a look at the following AFL-compatibile dictionaries for real world examples on using a dictionary file as fuzzer input.

  • Optional
  • Datatype: string
  • Default: empty
  • Example:

    dictionary: /tmp/my-dictionary