Mayhemfile Configuration¶
When executing a Mayhem run to analyze a target application, Mayhem uses a configuration file (called a Mayhemfile
) that designates the required and optional parameters a user can specify for testing their custom targets.
With simple applications, users can use the mayhem package command to automatically generate a Mayhemfile
for their target application; however, for more complex software, oftentimes users will need to specify additional customizations to their Mayhemfile
in order to fuzz their targets correctly. Therefore, we've included the following information to help you configure your Mayhemfile
for your Mayhem runs.
Mayhemfile Source vs. Run¶
For debugging purposes, users should be aware of the difference between a Mayhemfile
that a user creates/uploads to Mayhem and the fully qualified Mayhemfile
that Mayhem actually uses for a Mayhem run.
In particular, Mayhem will add final changes to a user-uploaded Mayhemfile
by adding, modifying, or removing certain fields to build out the full configuration used for executing a Mayhem run. Mayhem will adjust a user-uploaded "source" Mayhemfile
to take into account the following:
- Environment Variables: Variables within the
Mayhemfile
, such as${MAYHEM_PROJECT}
that are substituted with the value of the user's corresponding environment variable (if available). - CLI Overrides: Values within the
Mayhemfile
that are overridden as a result of executing a Mayhem run via the Mayhem CLI with specific parameters/flags. - Default Values: Default parameters and values that are automatically added to the
Mayhemfile
if no custom values are specified.
Environment Variables¶
Take a look at the following source Mayhemfile
:
1 2 3 4 |
|
Here we see that the ${FOO}
and ${BAR}
environment variables have been set.
Note
Users can use either syntax when setting environment variables within their Mayhemfiles: ${FOO}
or $FOO
, which are equivalent.
Assuming ${FOO}
is set to foo
and ${BAR}
is set to bar
for the user's environment, executing the mayhem run
command will ask you for the following environment variable interpolation prompt. Type "Y" to proceed.
$ export FOO=foo
$ export BAR=bar
$ mayhem run .
Performing environment variable interpolation based on the following values from the local environment:
FOO
BAR
Proceed? [Y/n]? Y
Run started: foo/bar/1
Run URL: https://tutorial.forallsecure.com:443/mayhemuser/foo/bar/1
Tip
If you'd prefer to skip prompts, you can use the optional -y
or --noninteractive
arguments for Mayhem CLI commands such as mayhem run
. For example mayhem -y run .
will execute a Mayhem run for a Mayhemfile in the current directory in non-interactive mode.
The run should execute, and if we check the Mayhemfile
for this run, we see that the ${FOO}
and ${BAR}
environment variables have been substituted with the correct values.
1 2 3 4 |
|
Info
You can escape characters in your source Mayhemfile
by using a backslash \
. For example, if you had a target literally called $TARGET
, you would escape the $
by using a backslash:
# Source Mayhemfile
project: foo
target: bar
cmds:
- cmd: \$TARGET @@
Which would result in the following:
# Run Mayhemfile
project: foo
target: bar
cmds:
- cmd: $TARGET @@
CLI Overrides¶
Users can specify the following override parameters from the Mayhem CLI when executing a mayhem run
.
-h, --help Show this help message and exit.
--regression Run regression tests on available test cases.
--static Run static checks on the entrypoint of the target.
--dynamic Run dynamic analysis on the target.
--coverage Perform coverage analysis on the target.
--all Enable all supported analyses.
-f FILE, --file FILE Path to the Mayhemfile used (default:
<package>/Mayhemfile).
-b BUILD_ID, --build-id BUILD_ID
Build id to associate with this specific run.
--docker Indicates whether the package is a docker image tag or
hash.
--warning-as-error Have the warnings be treated as errors.
--testsuite TESTSUITE Specify a testsuite directory.
--project PROJECT Name of the project.
--owner OWNER The owner for this project.
--target TARGET Name of the target.
--image IMAGE Docker image you want to analyze.
--duration DURATION How long to run for in seconds (wall clock time).
--uid UID User id for running the target.
--gid GID Group id for running the target.
--advanced-triage ADVANCED_TRIAGE
Set extra advanced triage analysis. Increases the
number of CWEs Mayhem finds but also increases test
case processing time.
--cmd CMD Command to invoke the target.
--cwd CWD Current working directory for running the target.
--env KEY=VALUE Environment variable to include while running.
--filepath FILEPATH Input file path where the target reads from.
--network-url NETWORK_URL
Network URI where the target reads from.
--network-timeout NETWORK_TIMEOUT
The time (in seconds) for Mayhem to wait for the
target to accept network input.
--network-client NETWORK_CLIENT
Specify whether this network target is a client or
server.
--libfuzzer LIBFUZZER
Specify whether this is a libfuzzer target or not.
--honggfuzz HONGGFUZZ
Specify whether this is a honggfuzz target or not.
--sanitizer SANITIZER
Specify whether sanitization is compiled in or not.
--max-length MAX_LENGTH
Maximum length for test cases.
--memory-limit MEMORY_LIMIT
How much memory to allow the target in megabytes (MB).
--url URL URL to running Mayhem API.
--token TOKEN Authentication token for accessing Mayhem API.
-k, --insecure Disable SSL verification.
--cacert CACERT Path to the mayhem server's certificate.
--timeout TIMEOUT Seconds to wait for API responses (useful for slow
connections).
For example, given the following source Mayhemfile:
1 2 3 4 5 |
|
The subsequent Mayhem CLI command mayhem run . --image forallsecure/tutorial
(executed within the current directory as the above Mayhemfile
) will override the image
parameter with the forallsecure/tutorial
image from Docker Hub, producing the following Mayhemfile
for the run:
1 2 3 4 5 |
|
Info
Override parameters will either modify or add parameters depending on whether the override key already exists in the source Mayhemfile.
Default Values¶
Mayhem will ensure the following default values are added to a Mayhem run Mayhemfile
if a user does not specify a custom value for one of the following parameters:
1 2 3 4 5 6 7 8 9 |
|
In order to construct the default values above, Mayhem will substitute the values of the following in-built values, which are auto-populated if not specified as custom environment variables. These are substituted in various places such as the testsuite
and image
values:
- ${MAYHEM_DOCKER_REGISTRY}: Set as the value of the Mayhem Docker Registry URL for the Mayhem instance executing the run (e.g.
tutorial.forallsecure.com:5000
). - ${MAYHEM_DOMAIN}: Set as the value of the URL for the Mayhem instance executing the run (e.g.
tutorial.forallsecure.com
). - ${MAYHEM_PROJECT}: Set as the value derived from the required
project
field in the Mayhemfile with the name of the user or organization prepended to it (for exampleforallsecure/project1
). - ${MAYHEM_TARGET}: Set as the value derived from the required
target
field in the Mayhemfile. - ${MAYHEM_URL}: Set as the value based on the following syntax:
https://${MAYHEM_DOMAIN}
(e.g.https://tutorial.forallsecure.com
). - ${MAYHEM_OWNER}: Set as the value for either the owner provided as a part of the
project
field in the Mayhemfile (e.g.project: forallsecure/project1
), or the username of the logged in Mayhem user. In this case, the owner would be set toforallsecure
.
For example, if the user mayhemuser
uploaded the following Mayhemfile
when executing a Mayhem run:
1 2 3 4 |
|
Mayhem would automatically add the following defaults (and substitute the in-built variables accordingly) to the Mayhemfile
for the resulting run:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
Mayhemfile Examples¶
Mayhem can fuzz target applications via file input, standard input, or TCP/UDP input. This section includes some examples of the different types of Mayhemfile
configurations.
File Input¶
Simple single input on command-line (from objdump
):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
Standard Input (stdin)¶
Running forever on djpeg while accepting input from /dev/stdin
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
|
TCP/UDP Input¶
Simple single input from network on TCP port 80 using a public docker hub image (from sebp/lighttpd
):
1 2 3 4 5 6 7 8 9 10 |
|
Mayhemfile Parameters¶
Below are all of the available parameters to the Mayhemfile
.
Important
There are multiple levels of Mayhemfile
parameters, therefore be cognizant of the parameter hierarchy.
version
¶
Specifies the version of Mayhem.
- Optional
- Datatype: string
-
Example:
version: '2.10'
Info
Mayhemfile versions can be set to 0.4 and above. Note, however, that Mayhemfiles that specify earlier versions of Mayhem may not include the functionality of more up-to-date versions.
project
¶
The unique name of the project organization and project the target belongs to.
- Owner (Optional), Projectname (Optional)
- Datatype: string
-
Example:
project: owner/projectname
target
¶
The name of the target you are testing. A single target could have multiple targets.
- Optional
- Datatype: string
-
Example:
target: targetname
image
¶
Warning
Formerly baseimage
, this field is now image
in mCode version 2.X.
image
is the name of the Docker image that will be used. The image can contain the targets (if using Docker packaging) or they may be contained in a root/ folder, in which case they are overlayed into the image (see mayhem package command). If image is not specified, ForAllSecureâs default Docker image based on Debian buster will be used: $MAYHEM_DOCKER_REGISTRY/forallsecure/debian-buster
. Docker images can be uploaded to Mayhem via an internal docker registry or through the public Docker Hub registry.
Info
To more easily refer to Docker images that have been uploaded to Mayhem, the $MAYHEM_DOCKER_REGISTRY
variable can be used to reference the location of the serverâs internal docker registry. Mayhem comes with ForAllSecure's default image preinstalled.
- Optional
- Datatype: string
- Default:
$MAYHEM_DOCKER_REGISTRY/forallsecure/debian-buster
-
Example:
image: $MAYHEM_DOCKER_REGISTRY/forallsecure/debian-buster
duration
¶
Time in seconds to run the application.
Info
If the duration
parameter is set to null
(i.e. no default duration was specified) the duration will have an unlimited timeframe.
- Optional
- Datatype: integer
- Default:
null
-
Example:
duration: 90
advanced_triage
¶
Runs additional analysis via extra test case processing to find defects that do not raise a UNIX signal, such as memory leaks or using an uninitialized variable.
Info
Due to the nature of adding extra test case processing, enabling advanced_triage
may reduce the performance of the overall Mayhem run. In contrast, normal triage runs can be up to 10x faster than runs with advanced_triage
enabled.
- Optional
- Datatype: boolean
- Default:
false
-
Example:
advanced_triage: false
tasks
¶
Sets the tasks that will be executed for the Mayhem run. The tasks
parameter can be set to include the following tasks:
- Exploitability Factors
- Regression Testing
- Behavior Testing
- Coverage Analysis
- Optional
- Datatype: object
- Default:
tasks: - name: exploitability_factors - name: regression_testing - name: behavior_testing
-
Examples:
tasks: - name: exploitability_factors - name: regression_testing - name: behavior_testing - name: coverage_analysis
Note
Executing the mayhem run
command with task parameters will override the tasks
key in the Mayhemfile
. For example, mayhem run --all .
will execute all tasks for a Mayhem run, even if the tasks
key in the Mayhemfile is only set to behavior_testing
.
testsuite
¶
Note
Formerly corpus
, this field is now testsuite
in mCode version 2.X.
Defines the location of the initial test suite for a Mayhem run, and is treated as such:
-
If the element begins with
file://
, it is a local file source, representing a file or directory on the same system as the CLI starting the run.- The path can be a relative path (
file://foo
for example). If so, this file is resolved relative to the directory containing the Mayhemfile. - The path can be an absolute path (
file:///
, three slashes indicating path starts with a leading slash).
- The path can be a relative path (
-
If your Mayhemfile specifies a Docker image with a
/corpus
or/testsuite
folder in the root of the image filesystem, then the contents of these folders will be added to the initial test suite. -
Optional
- Datatype: list of strings
-
Default:
https://$MAYHEM_DOMAIN/$MAYHEM_PROJECT/$MAYHEM_TARGET/testsuite.tar
file://testsuite
Info
https://$MAYHEM_DOMAIN/$MAYHEM_PROJECT/$MAYHEM_TARGET/testsuite.tar
specifies a URL pointing to atestsuite.tar
file stored on a Mayhem instance for a given project and target (if available). This ensures test cases from previous Mayhem runs are carried over into the next Mayhem run for a given project and target. -
Example:
testsuite: - file://testsuite
uid
¶
An optional numeric UNIX user id to use as the target uid. If unset, an unprivileged UNIX user id will be used that could cause issues accessing files. If a uid other than 0 is used, the target will generally not be able to perform privileged operations such as bind to a port lower than 1024. Docker defaults this to 0, although it can be overridden by the USER command. If using the default base image, this is 1000.
- Optional
- Datatype: integer
- Default: derived from
image
-
Example:
uid: 0
gid
¶
An optional numeric UNIX group id to use as the target gid. If unset, an unprivileged UNIX group id will be used that could cause issues accessing files. If unspecified, the gid is inherited from the base image. Docker defaults this to 0, although it can be overridden by the USER command. If using the default base image, this is 1000.
- Optional
- Datatype: integer
- Default: derived from
image
-
Example:
gid: 0
cmds
¶
A list of cmd objects to specify the different ways to test the packaged application. Each cmd object tells Mayhem how to analyze the target executable. Each cmd object contains the following optional fields that should only be used if the target requires additional information or changes to its environment prior to execution.
- Required
- Datatype: dictionary
cmd
¶
Invocation of the target executable.
If you omit the @@
identifier from the command line and do not specify the target as a networking one, Mayhem will assume that standard input (/dev/stdin
) is the main input source.
Note
If you need to manipulate the fuzz data in the @@
variable as a filesystem file, look towards the filepath
parameter to accomplish this use case. The cmd
contents are executed directly and are not passed through the bash interpreter.
- Required
- Datatype: string
-
Example One:
cmd: /usr/bin/objdump -x @@
-
Example Two:
- cmd: /host/tinyxml2-2.0.1/parse-fuzzer libfuzzer: true - cmd: /host/tinyxml2-2.0.1/orig-collapse @@
Info
It is possible to run multiple
cmd
parameters in a single Mayhemfile for when it is desired to run a target multiple times using differing instances of instrumentation. For example, a target that runs both a libFuzzer harnesscmd
and an uninstrumented/base-executablecmd
that reads in a file and calls the same code. This would be beneficial as symoblic execution does not support libFuzzer, but test cases that it finds in the uninstrumented/base-executable binary can be shared with the libFuzzer run and help discover new coverage.
max_length
¶
Maximum length (in bytes) that will be provided as input to the fuzz target and should not exceed 100MB, otherwise the Mayhemfile will fail to validate. Test cases in the starting tests larger than the specified maximum length will be truncated to the value set.
Info
If a user uses Mayhemfile V1.10 and does not provide a max_length field explicitly in Mayhemfile, it is by default 8192 bytes. Otherwise for Mayhemfile V1.9 and below, the default will be 8192 bytes for libFuzzer targets and no default for uninstrumented targets.
- Optional
- Datatype: integer
- Default:
8192
-
Example:
max_length: 8192
cwd
¶
Warning
Formerly chdir
, this field is now cwd
in mCode version 2.X.
The path Mayhem should switch to prior to executing the target. (Also known as the current working directory - CWD)
- Optional
- Datatype: string
- Default: derived from
image
-
Example:
cwd: /usr/bin
filepath
¶
Warning
Formerly target_path
, this field is now filepath
in mCode version 2.X.
The name that Mayhem should give the input file when it is needed to manipulate the fuzz data in the @@
variable as a filesystem file.
- Optional
- Datatype: string
- Default: null
-
Example:
filepath: /tmp/my_custom_file
env
¶
Specify the environment variables which will be set when your program is run. Additionally, special environment variables can be specified if your program is not compatible with some performance enhancements used by Mayhem. While this is often not necessary, some multi-threaded programs can be problematic to analyze.
Special environment variables for use during a Mayhem run are as follows:
Environment Variable | Value | Description |
---|---|---|
DISABLE_SMOKETEST |
1 |
Setting 1 disables smoke testing, where Mayhem will perform quick sanity checks to make sure that a target works, and if not, detects and reports common misconfigurations that require user intervention in the package/Mayhemfile. |
MAYHEM_AUTORESOLUTION |
true or false |
Setting true enables Mayhem auto-resolution, where Mayhem will attempt to auto-resolve Mayhemfile configuration issues that may arise during the Mayhem run. Setting false disables Mayhem auto-resolution. |
MFUZZ_COMPAT_LEVEL |
1 or 2 |
Setting 1 disables some of Mayhem's performance optimizations (specifically deferred execution of the fork server). Setting 2 completely disables optimizations that may interfere with multithreaded programs. |
MFUZZ_JAVA_MAIN_CLASS |
e.g. com.forallsecure.FuzzMe |
Sets the main class for the specified JAR file. |
MFUZZ_JAVA |
1 |
Setting 1 instructs Mayhem to fuzz a Java target. |
MFUZZ_CUSTOM_MUTATOR |
e.g. /libmutator.so |
Sets the shared object for custom mutator support. |
MFUZZ_DOCKER_ENTRYPOINT |
1 |
Setting 1 instructs Mayhem to run an available docker-entrypoint.sh script for a Docker image target. |
MAYHEM_TIMEOUTS_AS_DEFECTS |
1 |
Setting 1 instructs Mayhem to treat test case timeouts as defects and report them under the Defects/Test Case pane in the Mayhem UI run page. The CWE for these defects will be 407 Inefficient Algorithmic Complexity . |
Info
The value format for the env
parameter can either be in flow or block YAML format, and supports multiple environment variables.
- Optional
- Datatype: object
-
Example One:
env: { "ENV_VAR1": "value1", "ENV_VAR2": "value2", "ENV_VAR3": "value3" }
-
Example Two:
env: ENV_VAR1: value1 ENV_VAR2: value2 ENV_VAR3: value3
-
Default: derived from environment variables already present in the Docker image specified by the
image
parameter.
network
¶
Signify the target is a network target
- Required if the target will receive input over a network
- Datatype: string
- Default:
{}
url
¶
Specify the protocol, host, and port to analyze. Supported protocols are: "tcp", "udp", "unix+stream", "unix+dgram" and "unix+seqpacket"
For Mayhem, localhost
means any local interface, IPv4 or IPv6 and will attempt to perform auto-detection on which protocol should be tested. If you need to use specifically one or the other, you may instead write 127.0.0.1
for IPv4 or [::1]
for IPv6. For example tcp://127.0.0.1:80
will feed input to an IPv4 connection to port 80.
Make sure you are aware how your target is communicating. In particular, using localhost
inside of a container may require a hostname lookup for localhost
which may not resolve to what you expect. If possible, stick to an IP address for binding and connecting in internal target configurations to avoid ambiguity.
- Required if the target will receive input over a network
- Datatype: string
- Default: empty
-
Example:
url: tcp://localhost:8080
timeout
¶
The time (in seconds) for Mayhem to wait for the target to accept network input during triage. If the target does not accept network input within this time limit, the testcase will be discarded. If the time is too low, the testcase may report back as normal. Targets that take a while to start may need a longer timeout.
Note
This field refers to the timeout
for network targets and is different from the timeout
parameter for non-network targets.
- Optional
- Datatype: number
- Default:
2
-
Example:
timeout: 10
client
¶
Warning
Formerly is_client
, this field is now client
in mCode version 2.X.
Set to 'true' if the executable to be tested is a client. A client is expected to initiate the network connection, while a server is expected to wait for Mayhem.
- Optional
- Datatype: boolean
- Default:
false
-
Example:
client: true
timeout
¶
The time (in seconds) for Mayhem to wait for a single test to complete.
Info
The max timeout
allowed is 60 seconds unless overridden by the deployment/install-time environment variable MAX_CMD_TIMEOUT_SECONDS
to increase the max timeout
limit further. Values greater than the set maximum (either 60 seconds by default or specified by MAX_CMD_TIMEOUT_SECONDS
) will be automatically downsized to the maximum timeout
limit. Contact your administrator to set the MAX_CMD_TIMEOUT_SECONDS
environment variable if needed.
- Optional:
- Datatype: number
- Default: automatic
-
Example:
timeout: 1.5
memory_limit
¶
Warning
Formerly memory_limit_in_mb
, this field is now memory_limit
in mCode version 2.X.
The maximum amount of memory (in MB) to allow the target to allocate. Mayhem will kill a test run that goes above this memory limit and reports it as a crash.
- Optional
- Datatype: integer
- Default: automatic
-
Example:
memory_limit: 8192
sanitizer
¶
Set to true only if the target executable is compiled with a sanitizer
- Optional
- Datatype: boolean
- Default: automatic
-
Example:
sanitizer: false
libfuzzer
¶
Set to true only if the target executable is compiled with libFuzzer.
- Optional
- Datatype: boolean
- Default: automatic
-
Example:
libfuzzer: false
afl
¶
Set to true only if the target executable is compiled with AFL/AFL++ instrumentation.
- Optional
- Datatype: boolean
- Default: automatic
-
Example:
afl: false
honggfuzz
¶
Set to true only if the target executable is compiled with Honggfuzz.
- Optional
- Datatype: boolean
- Default: automatic
-
Example:
honggfuzz: false
gcov
¶
Set to true only if the target executable is compiled with gcov
support. Also, gcno
files should be located in the same path as the target executable.
- Optional
- Datatype: boolean
- Default: false
-
Example:
gcov: true
sancov
¶
Set to true only if the target executable is compiled with sancov
support.
- Optional
- Datatype: boolean
- Default: false
-
Example:
sancov: true
dictionary
¶
The path to an AFL/AFL++-compatible dictionary file. The path should be relative to the root of the container. For instance, if the dictionary is under /etc/foo/bar/http.dict
inside the image, the dictionary path in the Mayhemfile should be /etc/foo/bar/http.dict
.
Tip
Take a look at the following AFL-compatibile dictionaries for real world examples on using a dictionary file as fuzzer input.
- Optional
- Datatype: string
- Default: empty
-
Example:
dictionary: /tmp/my-dictionary