Mayhem CLI Commands¶

Here we have a comprehensive reference guide for each the Mayhem CLI commands.

Info

You will need to have the mayhem binary installed to use the Mayhem CLI. Check out Installing the Mayhem CLI for more information on getting set up.

Use the mayhem --help command to get more information on all the available Mayhem CLI commands.

(base) andrew@Andrews-MacBook-Pro mcode % mayhem --help
usage: mayhem [-h] [-y] [--verbosity {info,debug}] [--version] CMD ...

Command line client for interacting with Mayhem

*positional arguments*:
  CMD
    check               Check targets to see if they are Mayhem-eligible.
    docker-registry     Gets the URI for mayhem's docker registry.
    download            Download a target and its test cases.
    init                Generate a Mayhemfile.
    list                List projects and targets you have run.
    login               Login to a Mayhem server.
    logout              Logout from a Mayhem server.
    package             Package the given target and dependencies for Mayhem.
    run                 Run a target through Mayhem.
    show                Show Mayhem run(s).
    stop                Stop a Mayhem run.
    sync                Sync a package to its latest state.
    validate            Validate a Mayhemfile.
    wait                Wait for a run to finish.

*optional arguments*:
  -h, --help            Show this help message and exit.
  -y, --noninteractive  Accept the default options for all prompts. Equivalent 
                        to setting the environment variable `MAYHEM_NONINTERACTIVE=1`.
  --prompt              Force prompt text to display. Useful when no 
                        TTY/interactive shell is connected, such as in a CI/CD pipeline,
                        and a user wants to control prompt input. Equivalent 
                        to setting the environment variable `MAYHEM_PROMPT=1`.
  --verbosity {info,debug}
                        Set mayhem verbosity level.
  --version             Get the version of the Mayhem CLI.

additional information:
    By default, mayhem stores its configuration in the directory
    `~/.config/mayhem`.
    If the environment variable `XDG_CONFIG_HOME` is set, it will
    store configuration under `$XDG_CONFIG_HOME/mayhem`.

check¶

Checks to see if targets are Mayhem-eligible.

Usage: mayhem check <file>
Positional Arguments:
- file: Path to local file(s) to check.
Optional Arguments:
- -h, --help: Show this help message and exit.
- --format {pretty,json}: Format for returned results.

Use Case Example: file = ./testme

The command mayhem check ./testme prints various characteristics of the testme binary in the current directory, including:

Architecture, word size, and endianness
Dynamically-loaded libraries
Exploitability factors such as DEP and stack canary support
Memory map when loaded
Presence of fuzzer instrumentation
System calls the binary may make

~/tutorial/testme/v1# mayhem check ./testme
Key        Value
---------  -------------------------------
File       /root/tutorial/testme/v1/testme
Type       ELF/x86_64
Version    8.3.0
PIE        ✖
DEP        ✖
Canary     ✖
Fortify    ✖
Static     ✔
Fuzz       ✔
LibFuzzer  ✖
HonggFuzz  ✖
SymbExec   ✔
AFL        ✖
ASAN       ✖
MSAN       ✖
UBSAN      ✖
LSAN       ✖
Rust       ✖
Golang     ✖

docker-registry¶

Gets the URI for mayhem's docker registry.

Usage: mayhem docker-registry
Optional Arguments:
- -h, --help: Show this help message and exit.
Use Case Example:

The command mayhem docker-registry outputs the URI for the connected Mayhem Docker Registry.
```
~/tutorial/testme/v1# mayhem docker-registry
tutorial.forallsecure.com:5000
```

download¶

Downloads a target and its test cases.

Usage: mayhem download <target>
Positional Arguments:
- target: Specified as [<owner>/]<project>/<target>.
Optional Arguments:
- -h, --help: Show this help message and exit.
- -o OUTPUT, --output OUTPUT: Specify output directory for the target.
- -n NAMESPACE, --owner NAMESPACE: Owner to use for API interactions.
- --url URL: URL to running Mayhem API.
- --token TOKEN: Authentication token for accessing Mayhem API.
- -k, --insecure: Disable SSL verification.
- --cacert CACERT: Path to the mayhem server's certificate.
- --timeout TIMEOUT: Seconds to wait for API responses (useful for slow connections).

Use Case Example: <target> = testme/testme

The command mayhem download testme/testme will download the latest Mayhemfile, testsuite.tar, and root.tgz (if non-Docker target) for the testme target within the testme project for the connected Mayhem user.

~/tutorial/testme/v1# mayhem download testme/testme
Downloaded: Mayhemfile.
Downloaded: root.tgz.
Downloading testsuite.tar:   1.3 MiB |         #| Elapsed Time: 0:00:02 466.7 KiB/s
Extracting test 1281 of 1281 |#############| Time:  0:00:00
Target downloaded at: '/tmp/testmetestme-uk3lt2ql'.

init¶

Generates a Mayhemfile.

Usage: mayhem init <image>
Positional Arguments:
- image: Docker image tag or hash.
Optional Arguments:
- -h: Show this help message and exit.
- -o OUTPUT: Specify the file path of the Mayhemfile.
- --project PROJECT: Name of the project.
- --owner OWNER: The owner for this project.
- --target TARGET: Name of the target.
- --image IMAGE: Docker image you want to analyze.
- --duration DURATION: How long to run for in seconds (wall clock time).
- --uid UID: User id for running the target.
- --gid GID: Group id for running the target.
- --advanced-triage ADVANCED_TRIAGE: Set extra advanced triage analysis. Increases the number of CWEs Mayhem finds but also increases test case processing time.
- --cmd CMD: Command to invoke the target.
- --cwd CWD: Current working directory for running the target.
- --env KEY=VALUE: Environment variable to include while running.
- --filepath FILEPATH: Input file path where the target reads from.
- --network-url NETWORK_URL: Network URI where the target reads from.
- --network-timeout NETWORK_TIMEOUT: The time (in seconds) for Mayhem to wait for the target to accept network input.
- --network-client NETWORK_CLIENT: Specify whether this network target is a client or server.
- --libfuzzer LIBFUZZER: Specify whether this is a libfuzzer target or not.
- --honggfuzz HONGGFUZZ: Specify whether this is a honggfuzz target or not.
- --sanitizer SANITIZER: Specify whether sanitization is compiled in or not.
- --max-length MAX_LENGTH: Maximum length for test cases.
- --memory-limit MEMORY_LIMIT: How much memory to allow the target in megabytes (MB).
- --template TEMPLATE: Generate a harness template or wrapper. Support for C++ and AutoSAR currently.

Use Case Example One: <image> = nginx

The command mayhem init nginx will generate a Mayhemfile for a given Docker image from either Docker Hub or a private Mayhem Docker Registry and attempt to automatically fill in the required fields for the Mayhemfile (version, project, target, cmds, and cmd).

(base) andrew@Andrews-MacBook-Pro mcode % mayhem init nginx
View/edit generated Mayhemfile at: /Users/andrew/Desktop/mcode/Mayhemfile
To start a run with this configuration, run  mayhem run .

# Mayhem by https://forallsecure.com
# Mayhemfile: configuration file for testing your target with Mayhem
# Format: YAML 1.1

# Owned project name that the target belongs to
project: nginx

# Target name (should be unique within the project)
target: nginx-latest

# Base image to run the binary in.
image: nginx:latest

# List of commands used to test the target
cmds:

  # Command used to start the target, "@@" is the input file
  # (when "@@" is omitted Mayhem defaults to stdin inputs)
  - cmd: nginx -g 'daemon off;'
      network:
        client: false
        timeout: 2.0
        url: tcp://localhost:80
      env: {}

      ## Uncomment to change default dir (/) from which the target is invoked
      #cwd: /

      ## Max test case length (in bytes) to be taken into account. Test cases over
      ## that length will be truncated. Be very careful about increasing this
      ## limit as it can severely affect your fuzzer performance.
      # max_length: 8192

Use Case Example Two: <image> = null

The command mayhem init will generate a default Mayhemfile template.

(base) andrew@Andrews-MacBook-Pro mcode % mayhem init
View/edit generated Mayhemfile at: /Users/andrew/Desktop/mcode/Mayhemfile
To start a run with this configuration, run  mayhem run .

# Mayhem by https://forallsecure.com
# Mayhemfile: configuration file for testing your target with Mayhem
# Format: YAML 1.1

# Owned project name that the target belongs to
project: MYPROJECT

# Target name (should be unique within the project)
target: MYTARGET

# Base image to run the binary in.
image: MYIMAGE

# List of commands used to test the target
cmds:

  # Command used to start the target, "@@" is the input file
  # (when "@@" is omitted Mayhem defaults to stdin inputs)
  - cmd: MYBINARY
      env: {}

      ## Uncomment to change default dir (/) from which the target is invoked
      #cwd: /

      ## If this is a network target, uncomment the block below and remove
      ## the @@ input file reference in the cmd (you can either test network or
      ## file inputs, not both).
      #network:
      ## Use "127.0.0.1" instead of "localhost" below if you want to test only
      ## for IPv4. For IPv6, use "[::1]". By leaving as "localhost", Mayhem will
      ## attempt to autodetect the one used by the target.
      #  url: tcp://localhost:8080  # protocol, host and port to analyze
      #  client: false           # target is a client-side program
      #  timeout: 2.0               # max seconds for sending data

      ## Max test case length (in bytes) to be taken into account. Test cases over
      ## that length will be truncated. Be very careful about increasing this
      ## limit as it can severely affect your fuzzer performance.
      # max_length: 8192

list¶

Lists projects and targets you have run.

Usage: mayhem list
Optional Arguments:
- -h, --help: Show this help message and exit.
- -n NAMESPACE, --owner NAMESPACE: Owner to use for API interactions.
- --url URL: URL to running Mayhem API.
- --token TOKEN: Authentication token for accessing Mayhem API.
- -k, --insecure: Disable SSL verification.
- --cacert CACERT: Path to the mayhem server's certificate.
- --timeout TIMEOUT: Seconds to wait for API responses (useful for slow connections).

Use Case Example:

The command mayhem list returns the corresponding projects and targets for a connected Mayhem user in the form of <project>/<target>.

~/tutorial/testme/v1# mayhem list
afl/testme
atheris/fuzzme-out-of-bounds
atheris/testme
auto-resolution/buf
auto-resolution/testme-timeout
bacsrv/bacsrv
c/powerpc-c-uninstrumented
cpp/cpp-afl-gcc
dialserver/dialserver
dnsmasq/dnsmasq
docker/lighttpd-1-4
docker/testme
foo/bar
forallsecure-testme-docker/latest
forallsecure-tutorial/latest
...

Login to a Mayhem server.

Usage: mayhem login <MAYHEM_URL> <MAYHEM_TOKEN>
Positional arguments:
- MAYHEM_URL: URL to running Mayhem API.
- MAYHEM_TOKEN: Authentication token for accessing Mayhem API.
Optional Arguments:
- -h, --help: Show this help message and exit.
- --url URL: URL to running Mayhem API.
- --token TOKEN: Authentication token for accessing Mayhem API.
- -k, --insecure: Disable SSL verification.
- --cacert CACERT: Path to the mayhem server's certificate.
- --timeout TIMEOUT: Seconds to wait for API responses (useful for slow connections).
Use Case Example: MAYHEM_URL = tutorial.forallsecure.com MAYHEM_TOKEN = AT1.XAqBdE9bdo9YlMwDFdp-1Ku7wq8Cmek1

The command mayhem login tutorial.forallsecure.com AT1.XAqBdE9bdo9YlMwDFdp-1Ku7wq8Cmek1 will log into the Mayhem server for the given user designated by the API token. The Mayhem CLI will also attempt to log into the corresponding Mayhem Docker Registry if available.
```
Logged in successfully at 'https://tutorial.forallsecure.com:443' as 'mayhemuser'.
Syncing default settings: /root/.config/mayhem/mayhem.
Docker is not installed, skipping docker login
```

logout¶

Logout from a Mayhem server.

Usage: mayhem logout
Optional Arguments:
- -h, --help: Show this help message and exit.
Use Case Example:

The command mayhem logout will log a user out of the Mayhem server as well as the Mayhem Docker Registry if available.
```
Docker CLI not found, skipping logout from Docker registry.
Logged out.
```

package¶

Packages the given target and dependencies for Mayhem.

Usage: mayhem package <binary>
Positional Arguments:
- binary: Path to local target to package.
Optional Arguments:
- -h, --help: Show this help message and exit.
- -o OUTPUT, --output OUTPUT: Specify output directory for the package.
- -d DEPDIRS, --depdirs DEPDIRS: Comma-separated list of directories to search for dependencies.

Example Use Case: <binary> = ./testme

The command mayhem package ./testme will package the testme binary located in the current directory and statically infer its dependencies, including them in /tmp/testme-noleclq8 along with a generated Mayhemfile and tests folder.

~/tutorial/testme/v1# mayhem package ./testme
Packaging target: /root/tutorial/testme/v1/testme
Packaging dependency: /root/tutorial/testme/v1/testme -> /tmp/testme-noleclq8/root/root/tutorial/testme/v1/testme
Generating default configuration under: /tmp/testme-noleclq8/Mayhemfile
Packaging complete.
To upload the package do: `mayhem run /tmp/testme-noleclq8`.
Before uploading, you may wish to edit the config file at '/tmp/testme-noleclq8/Mayhemfile'.
/tmp/testme-noleclq8
~/tutorial/testme/v1# ls -l /tmp/testme-noleclq8
-rw-r--r-- 1 root root 1918 Jan  5 18:08 Mayhemfile
drwxr-xr-x 2 root root 4096 Jan  5 18:08 tests
drwxr-xr-x 3 root root 4096 Jan  5 18:08 root

run¶

Runs a target through Mayhem.

usage: mayhem run <package>

Positional arguments:
- package: Path to the directory containing the packaged target.
Optional Arguments:
- -h, --help: Show this help message and exit.
- --regression: Run regression tests on available test cases.
- --static: Run static checks on the entrypoint of the target.
- --dynamic: Run dynamic analysis on the target.
- --coverage: Perform coverage analysis on the target.
- --all: Enable all supported analyses.
- -f FILE, --file FILE: Path to the Mayhemfile used (default: <package>/Mayhemfile).
- -b BUILD_ID, --build-id BUILD_ID: Build id to associate with this specific run.
- --docker: Indicates whether the package is a docker image tag or hash.
- --warning-as-error: Have the warnings be treated as errors.
- --testsuite TESTSUITE: Specify a test suite directory.
- --project PROJECT: Name of the project.
- --owner OWNER: The owner for this project.
- --target TARGET: Name of the target.
- --duration DURATION: Behavior testing duration in seconds, or format 'XhYmZs' to specify X hours, Y mins, Z secs (default: None) / Mayhemfile override.
- --uid UID: User id for running the target.
- --gid GID: Group id for running the target.
- --advanced-triage ADVANCED_TRIAGE: Set extra advanced triage analysis. Increases the number of CWEs Mayhem finds but also increases test case processing time.
- --image IMAGE: Docker image you want to analyze.
- --cmd CMD: Command to invoke the target.
- --cwd CWD: Current working directory for running the target.
- --env KEY=VALUE: Environment variable to include while running.
- --filepath FILEPATH: Input file path where the target reads from.
- --network-url NETWORK_URL: Network URI where the target reads from.
- --network-timeout NETWORK_TIMEOUT: The time (in seconds) for Mayhem to wait for the target to accept network input.
- --network-client NETWORK_CLIENT: Specify whether this network target is a client or server.
- --libfuzzer LIBFUZZER: Specify whether this is a libfuzzer target or not.
- --honggfuzz HONGGFUZZ: Specify whether this is a honggfuzz target or not.
- --sanitizer SANITIZER: Specify whether sanitization is compiled in or not.
- --max-length MAX_LENGTH: Maximum length for test cases.
- --memory-limit MEMORY_LIMIT: How much memory to allow the target in megabytes (MB).
- -n NAMESPACE, --namespace NAMESPACE: Namespace to use for API interactions.
- --url URL: URL to running Mayhem API.
- --token TOKEN: Authentication token for accessing Mayhem API.
- -k, --insecure: Disable SSL verification.
- --cacert CACERT: Path to the mayhem server's certificate.
- --timeout TIMEOUT: Seconds to wait for API responses (useful for slow connections).

Use Case Example <package> = .

The command mayhem run . will search for a Mayhemfile in the current directory and execute a Mayhem run given the Mayhemfile configuration. Non-Docker targets will require a corresponding packaged binary with a valid file path to the target binary in the cmd field of the Mayhemfile.

~/tutorial/testme/v1# mayhem run .
/tmp/tmpgxuz_son/testsuite.tgz 100% |###########################| Time:  0:00:00 338.3 B/s
Syncing /tmp/tmpukob8cbx 100% |##############################| Time:  0:00:00
/tmp/tmp73rvnn8l/Mayhemfile 100% |###########################| Time:  0:00:00   3.1 KiB/s
Run started: testme/testme/66
Run URL: https://tutorial.forallsecure.com:443/mayhemuser/testme/testme/66
testme/testme/66

show¶

Shows Mayhem run(s).

usage: mayhem show <run>
Positional Arguments:
- run: Run specifier in [<owner>/]<project>/<target>/<run> format.
Optional Arguments:
- -h, --help: Show this help message and exit.
- --status STATUS: Filter runs based on status as a regular expression (default: '.*')
- --format {pretty,json,csv}: Format for returned results.
- -n NAMESPACE, --owner NAMESPACE Owner to use for API interactions.
- --url URL: URL to running Mayhem API.
- --token TOKEN: Authentication token for accessing Mayhem API.
- -k, --insecure: Disable SSL verification.
- --cacert CACERT: Path to the mayhem server's certificate.
- --timeout TIMEOUT: Seconds to wait for API responses (useful for slow connections).

Use Case Example One: <run> = testme/testme/66

The command mayhem show testme/testme/66 will return analysis results for the 66th run of the testme target within the testme project for the connected Mayhem user.

~/tutorial/testme/v1# mayhem show testme/testme/66
Statistics for testme/testme/66
Status: dynamic_analysis:failed, regression_testing:failed, static_analysis:completed
Run started: Wed Jan 5 18:24:45 2022 +0000
Time elapsed: 0:00:04
Tests performed: 0
Test reports: 0
Crash reports: 0
Defects: 0

Use Case Example Two: <run> = null

The command mayhem show will return analysis results for all runs of all projects and targets for the connected Mayhem user.

~/tutorial/testme/v1# mayhem show
Reports    Crashes    Defects  Date
----------------------------------------------------------------------------------------------------------------------------------------
testme/foo/1 dynamic_analysis:failed, static_analysis:completed, regression_testing:completed  82  59  1  Tue Oct 12 21:04:35 2021 +0000
testme/testme/68 dynamic_analysis:failed, regression_testing:failed, static_analysis:completed  0   0  0  Wed Jan 5 21:16:49 2022 +0000
testme/testme/67 static_analysis:stopped, regression_testing:stopped, dynamic_analysis:stopped  0   0  0  Wed Jan 5 20:06:20 2022 +0000
...

stop¶

Stops a Mayhem run.

Usage: mayhem stop <run_path>
Positional Arguments:
- run_path: Name of the run in [<owner>/]<project>/<target>/<run_number> format. To stop ALL runs for a target omit the <run_number>. For example "mayhem stop my-project/target". To stop runs for all targets omit both the <run_number> AND the <target>. For example "mayhem stop my-project".
Optional Arguments:
- -h, --help: Show this help message and exit.
- -n NAMESPACE, --owner NAMESPACE: Owner to use for API interactions.
- --url URL: URL to running Mayhem API.
- --token TOKEN: Authentication token for accessing Mayhem API.
- -k, --insecure: Disable SSL verification.
- --cacert CACERT: Path to the mayhem server's certificate.
- --timeout TIMEOUT: Seconds to wait for API responses (useful for slow connections).
Use Case Example: <run_path>: testme/testme/67

The command mayhem stop testme/testme/67 will stop the in-progress 67th run of the testme target for the testme project associated with the connected Mayhem user.
```
~/tutorial/testme/v1# mayhem stop testme/testme/67
Stopping Run testme/testme/67...
Stopping dynamic_analysis...
Stopping regression_testing...
Stopping static_analysis...
```

sync¶

Syncs a package to its latest state. Run mayhem sync path/to/target to retrieve the latest test cases from a target you have previously packaged and run.

Usage: mayhem sync <package>
Positional Arguments:
- package: Path to package directory.
Optional Arguments:
- -h, --help: Show this help message and exit.
- -n NAMESPACE, --owner NAMESPACE: Owner to use for API interactions.
- --url URL: URL to running Mayhem API.
- --token TOKEN: Authentication token for accessing Mayhem API.
- -k, --insecure: Disable SSL verification.
- --cacert CACERT: Path to the mayhem server's certificate.
- --timeout TIMEOUT: Seconds to wait for API responses (useful for slow connections).

Use Case Example: <package> = .

The command mayhem sync . will search the current directory for a Mayhemfile and download the latest testcases for the target and project combination within the Mayhemfile configuration.

~/tutorial/testme/v1# mayhem sync .
Downloaded: Mayhemfile.
Downloading testsuite.tar:   1.3 MiB |                            #| Elapsed Time: 0:00:02 456.2 KiB/s
Extracting test 1281 of 1281 |################################| Time:  0:00:00
Target synced at: '.'.

validate¶

Validates a Mayhemfile.

Usage: mayhem validate <package>
Positional Arguments:
- package: Path to the directory containing the packaged target.
Optional Arguments:
- -h, --help: Show this help message and exit.
- -f FILE, --file FILE: Path to the Mayhemfile used (default: <package>/Mayhemfile).
- --no-docker: Skips intrusive validation tests (that require docker).
- -n NAMESPACE, --owner NAMESPACE: Owner to use for API interactions.
- --url URL: URL to running Mayhem API.
- --token TOKEN: Authentication token for accessing Mayhem API.
- -k, --insecure: Disable SSL verification.
- --cacert CACERT: Path to the mayhem server's certificate.
- --timeout TIMEOUT: Seconds to wait for API responses (useful for slow connections).

wait¶

Waits for a run to finish.

Usage: mayhem wait <run>
Positional Arguments:
- run: Run to wait for (in [<owner>/]<project/target/run_number> format).
Optional Arguments:
- -h, --help: Show this help message and exit.
- -v, --verbose: Print status information while waiting.
- --regression: Wait on regression tests.
- --static: Wait on static analysis.
- --dynamic: Wait on dynamic analysis.
- --coverage: Perform coverage analysis on the target.
- --all: Wait for all analyses.
- --junit JUNIT: Generate junit XML report and write to the specified file.
- --sarif SARIF: Generate SARIF JSON report and write to the specified file.
- -n NAMESPACE, --owner NAMESPACE: Owner to use for API interactions.
- --url URL: URL to running Mayhem API.
- --token TOKEN: Authentication token for accessing Mayhem API.
- -k, --insecure: Disable SSL verification.
- --cacert CACERT: Path to the mayhem server's certificate.
- --timeout TIMEOUT: Seconds to wait for API responses (useful for slow connections).
Use Case Example <run> = testme/testme/68

The command mayhem wait testme/testme/68 will wait for the completion of the 68th run of the testme target for the testme project of a connected Mayhem user. The mayhem wait command is particularly useful within scripts where operations should wait for a Mayhem run to complete before proceeding.
```
~/tutorial/testme/v1# mayhem wait testme/testme/68
~/tutorial/testme/v1#
```