Skip to content

advanced

Target Input Methods

input

In this lesson, we'll show you how to use Mayhem to test binaries that use differing types of input such as file input, stdin, and TCP/UDP. Let's walk through how to package and test common network applications such as bacsrv and lighttpd.

Estimated Time: 20 minutes

By the end of this lesson, you will be able to:

  1. Compare and contrast the Mayhemfile configurations for binaries requiring differing input.
  2. Analyze the bacsrv network application and determine its listening port.
  3. Configure network options in Mayhemfile.
  4. Perform a Mayhem run on bacsrv.
  5. Manually package dependencies for lighttpd.
  6. Perform a Mayhem run on lighttpd.

Run through the lesson:

See prerequisites before beginning.

  1. Mayhem can fuzz target binaries via the following inputs:

    1. File input
    2. Standard input
    3. TCP/UDP input

  2. Package the bacsrv target using the Mayhem CLI:

    mayhem package ./bacsrv -o /tmp/bacsrv-pkg

  3. Execute a Mayhem run on the bacsrv target using the mayhem run command with the following Mayhemfile:

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
    project: bacsrv
target: bacsrv
duration: 90
advanced_triage: true

cmds:
  - cmd: /root/tutorial/bacsrv/bacsrv

    # For a network program, you add a "network" specification block
    network:
    url: udp://localhost:47808  # protocol, host and port to analyze
    client: false            # target is not a client program
    timeout: 2.0                # max seconds before timing out on network

  4. Package the lighttpd target using the mayhem package command:

    # Run the package command
mayhem package ./1.4.15/sbin/lighttpd -o /tmp/lighttpd-pkg

# Copy over all dependencies manually
cp -R ./1.4.15 /tmp/lighttpd-pkg/root/root/tutorial/lighttpd/
cp -R /www /tmp/lighttpd-pkg/root/

  5. Execute a Mayhem run on the lighttpd target using the mayhem run command with the following Mayhemfile:

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
    project: lighttpd
target: lighttpd-1-4
duration: 90

cmds:
  - cmd: /root/tutorial/lighttpd/1.4.15/sbin/lighttpd -D -f /root/tutorial/lighttpd/1.4.15/etc/lighttpd.conf
        network:
        client: false
        timeout: 5.0
        url: tcp://localhost:8080

You will need the following:

  • The Mayhem CLI installed and authenticated with the Mayhem server.

  • Access to the tutorial examples by running the pre-built Docker image.

    docker pull forallsecure/tutorial:2.10
docker run -ti --privileged --rm forallsecure/tutorial:2.10

Mayhemfiles for Input Types

Mayhem can fuzz target binaries via the following inputs:

  1. File input
  2. Standard input
  3. TCP/UDP input

Let's show you a few examples to explain the difference in Mayhemfile configurations. Feel free to execute the corresponding Mayhem runs for these Mayhemfiles.

Info

Have a more complex and custom target? Check out the Mayhemfile Configuration page for more detailed information on configuring the Mayhemfile.

File Input

Simple single input on command-line (from objdump):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
# Project name that the target belongs to
project: objdump

# Target name (should be unique within the project)
target: objdump

# Base image to run the binary in.
image: forallsecure/debian-buster:latest

# Time that analysis will run for in seconds - if absent, run forever
duration: 90

# List of commands used to test the target
cmds:

  # Command used to start the target, "@@" is the input file
  - cmd: /usr/bin/objdump -x @@

Standard Input (stdin)

Running forever on djpeg while accepting input from /dev/stdin:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
# Mayhemfile: configuration file for testing your target with Mayhem
# Format: YAML 1.1

# Project name that the target belongs to
project: libjpeg

# Target name (should be unique within the project)
target: djpeg

# Base image to run the binary in.
image: forallsecure/debian-buster:latest

# Commented out duration to run forever
# duration: 90

# List of commands used to test the target
cmds:

  # Command used to start the target. "@@" is missing so
  # input will be read from /dev/stdin.
  - cmd: /usr/bin/djpeg

TCP/UDP Input

Simple single input from network on TCP port 80 using a public docker hub image (from sebp/lighttpd):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
project: lighttpd-docker
target: lighttpd
image: sebp/lighttpd
duration: 90
cmds:
  - cmd: /usr/sbin/lighttpd -D -f /etc/lighttpd/lighttpd.conf
    network:
      client: false
      url: tcp://127.0.0.1:80
      timeout: 1.0

Analyzing bacsrv

bacsrv (sometimes called bacserv) is part of the BACnet suite of tools. To quote the BACnet website:

Developed under the auspices of the American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE), BACnet is an American national standard, a European standard, a national standard in more than 30 countries, and an ISO global standard. The protocol is supported and maintained by ASHRAE Standing Standard Project Committee 135 whose members.

Here we will be analyzing bacsrv, which listens on UDP port 47808. One advantage of Mayhem is that you do not need to write a special test driver for TCP and UDP servers. We will analyze the binary as-is, with no modifications.

The first step in analyzing an application is to make sure you can run it. Go to the examples/bacsrv directory, and run bacsrv

$ cd ~/tutorial/bacsrv
$ ./bacsrv &
BACnet Server Demo
BACnet Stack Version 0.9.1
BACnet Device ID: 260001
Max APDU: 1476

What if you're given a network application and you are not sure what port it uses? There are multiple ways to find out, such as using netstat, lsof, and strace. Let's use strace to confirm bacsrv listens on port 47808.

For example, we can find out bacsrv listens on port 47808 by looking for calls to bind:

# strace -Tfe trace=bind ./bacsrv
BACnet Server Demo
BACnet Stack Version 0.9.1
BACnet Device ID: 260001
Max APDU: 1476
bind(3, {sa_family=AF_INET, sin_port=htons(47808), sin_addr=inet_addr("0.0.0.0")}, 16) = 0 <0.000018>

Note

If you are following along in Docker, make sure you invoked it with the --privileged flag, or strace will not work.

Mayhemfile bacsrv

In previous lessons, you informed Mayhem that the testme application would read a file by appending @@ to the target path in the cmd parameter of the Mayhemfile:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# A user-meaningful name to group like targets
project: testme

# A user-meaningful name of the target
target: testme

# Time to spend fuzzing.
duration: 90

# Perform advanced triage and diagnosis of each test case.
# This will find many more defects, but add additional processing time.
advanced_triage: true

# List of executable programs to analyze.  For most use, you only
# need one.
cmds:

    # The full path to the executable program to test.
    #   - "@@" is the  to test.
    #     For programs that take input from a file, use '@@' to mark
    #     the location in the target's command line where the input
    #     file name should be placed. Mayhem will substitute this
    #     for you.
    - cmd: /root/tutorial/testme/v1/testme @@

When analyzing a network application, however, you need to tell Mayhem which network port to use and whether the application uses TCP or UDP. We show a Mayhemfile for bacsrv that shows how to specify this information.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
project: bacsrv
target: bacsrv
duration: 90
advanced_triage: true

cmds:
  - cmd: /root/tutorial/bacsrv/bacsrv

    # For a network program, you add a "network" specification block
    network:
      url: udp://localhost:47808  # protocol, host and port to analyze
      client: false               # target is not a client program
      timeout: 2.0                # max seconds before timing out on network
  • url is specified as TCP|UDP://<HOST>:PORT. You almost always want HOST to be localhost, since Mayhem will be both running and connecting to the service on the same host.
  • client specifies whether the app reaches out over the network, such as with curl and wget, or it is a server. We are analyzing a server, so we set the value to false.
  • timeout specifies how long Mayhem should wait after sending a request before it decides the service is no longer responding. 1.0 or 2.0 are good values, though if you have trouble with a network service timing out during analysis you should consider increasing it.

Mayhem bacsrv

Once you've written your Mayhemfile, you analyze bacsrv by packaging and performing a Mayhem run--similar to any other program:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
#!/bin/sh

# Package bacsrv
mayhem package ./bacsrv -o /tmp/bacsrv-pkg

# Write your Mayhemfile (here copying over provided one)
cp Mayhemfile /tmp/bacsrv-pkg/Mayhemfile

# Run Mayhem!
id=$(mayhem run /tmp/bacsrv-pkg)

# Wait for the run to finish
mayhem wait $id

# Sync the test suite to the "testsuite" directory.
mayhem sync /tmp/bacsrv-pkg

It should take less than 10 minutes (depending on your hardware) to reproduce a crash for an invalid read (CWE-125), as well as several other errors where variables used are uninitialized.

lighttpd

lighttpd is a fast, compliant, and flexible web server that scales several times better than alternate web servers.

In this example, we will demonstrate packaging and running lighttpd, and talk about common errors you may encounter when packaging. Recall a Mayhem package is the executable program and its full runtime environment, including any shared objects and configuration files.

Verify you can run lighttpd

We have pre-installed lighttpd-1.4.15, a version of lighttpd with an exploitable vulnerability, in the examples under tutorial/lighttpd.

./1.4.15/sbin/lighttpd -D -f ./1.4.15/etc/lighttpd.conf
2019-09-02 23:22:20: (log.c.75) server started
2019-09-02 23:22:21: (server.c.1252) [note] graceful shutdown started
2019-09-02 23:22:21: (log.c.135) server stopped

Info

Sending a SIGINT (signal interrupt) to lighttpd using Ctrl+C will initiate a graceful shutdown.

First Try: Packaging lighttpd

The mayhem package command inspects an executable for all dependencies it can resolve statically. However, lighttpd, like many web servers, resolves libraries at runtime. Note that shared libraries themselves are not the issue; it is when they are dynamically resolved at runtime, e.g., via dlopen().

Unfortunately, lighttpd uses dependencies that mayhem package cannot statically find. Execute the following to see why:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
#!/bin/sh

# Run the package command
mayhem package ./1.4.15/sbin/lighttpd -o /tmp/lighttpd-pkg

# Inspect the contents of the package
ls -R /tmp/lighttpd-pkg/root/

# Run mayhem on the package
id=$(mayhem run /tmp/lighttpd-pkg/)

# Wait for Mayhem to finish. It will return 'failed', indicating
# a failed run.
mayhem wait $id

The mayhem run will kick off a Mayhem run, but mayhem wait will indicate the run failed. If you go to the UI and click on the Event Log, you will see the following on why the lighttpd run failed:

  1. Behavior Testing for the Mayhem run failed.
  2. Mayhem detected that the program was not reading input.
  3. lighttpd printed an error.

failed-lighttpd-run.png

A little investigation into how lighttpd behaves indicates why:

  1. The package we uploaded did not include all lighttpd dependencies. Specifically, for this run, no configuration file could be found during lighttpd startup, and some libraries dynamically loaded by dlopen() were missing. As a result, mayhem package could not properly package the lighttpd target.
  2. The default Mayhemfile did not specify a network port to communicate with the lighttpd target.

A look at the generated Mayhemfile using the mayhem package command also reveals the limitation of this method of packaging:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# Namespaced project name that the target belongs to
project: lighttpd

# Target name (should be unique within the project)
target: lighttpd

# Base image to run the binary in.
# image: $MAYHEM_DOCKER_REGISTRY/forallsecure/debian-buster:latest

# Time that analysis will run for in seconds - if absent, run forever
duration: 90

# Turns on extra test case processing (completing a run will take longer)
advanced_triage: false

# List of commands used to test the target
cmds:

  # Command used to start the target, "@@" is the input file
  # (when "@@" is omitted Mayhem defaults to stdin inputs)
  - cmd: /root/tutorial/lighttpd/1.4.15/sbin/lighttpd @@
    env: {}

    ## Uncomment to change default dir (/) from which the target is invoked
    #cwd: /

    ## If this is a network target, uncomment the block below and remove
    ## the @@ input file reference in the cmd (you can either test network or
    ## file inputs, not both).
    #network:
    ## Use "127.0.0.1" instead of "localhost" below if you want to test only
    ## for IPv4. For IPv6, use "[::1]". By leaving as "localhost", Mayhem will
    ## attempt to autodetect the one used by the target.
    #  url: tcp://localhost:8080  # protocol, host and port to analyze
    #  client: false           # target is a client-side program
    #  timeout: 2.0               # max seconds for sending data

Here we can see that the generated Mayhemfile did not specify the network options for required for lighttpd.

Second Try: Successfully Analyzing lighttpd

In order to successfully analyze lighttpd, you need to:

  1. Write a Mayhemfile that correctly tells Mayhem to test lighttpd over TCP port 8080.
  2. Copy over all dependencies into the mayhem package.

In general, finding all dependencies can be a little tricky. In this lesson, we've simplified the problem by putting them all underneath one directory. In real life, dependencies may be scattered throughout the file system. You can use tools like strace, discussed above, to help find those dependencies.

The following Mayhemfile correctly tells Mayhem how to analyze lighttpd:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
project: lighttpd
target: lighttpd-1-4
duration: 90

cmds:
- cmd: /root/tutorial/lighttpd/1.4.15/sbin/lighttpd -D -f /root/tutorial/lighttpd/1.4.15/etc/lighttpd.conf
  network:
    client: false
    timeout: 5.0
    url: tcp://localhost:8080

Executing the following script will package up lighttpd and its associated dependencies correctly:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
#!/bin/sh

# Run the package command
mayhem package ./1.4.15/sbin/lighttpd -o /tmp/lighttpd-pkg

# Copy over all dependencies manually
cp -R ./1.4.15 /tmp/lighttpd-pkg/root/root/tutorial/lighttpd/
cp -R /www /tmp/lighttpd-pkg/root/

# Copy over the Mayhemfile needed to run lighttpd
cp Mayhemfile /tmp/lighttpd-pkg/

# Copy over an example test suite. This speeds up the example by
# seeding analysis with an example input.
cp testsuite/* /tmp/lighttpd-pkg/testsuite/

# Run mayhem on the package
id=$(mayhem run /tmp/lighttpd-pkg/)

# Wait for Mayhem to finish. It will succeed this time because we included
# lighttpd's dependencies.
mayhem wait $id

Warning

Recall Mayhem runs as an unprivileged user. If your network service binds below port 1024, you will need to set the top-level directive uid: 0 in the Mayhemfile to run your program as root.

✏️ Summary and Recap

In this lesson, you learned how to fuzz network targets such as bacsrv and lighttpd.

I learned how to...

1. Compare and contrast the Mayhemfile configurations for binaries requiring differing input.

  • Simple single input on command-line (from objdump): 

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
    # Project name that the target belongs to
project: objdump

# Target name (should be unique within the project)
target: objdump

# Base image to run the binary in.
image: forallsecure/debian-buster:latest

# Time that analysis will run for in seconds - if absent, run forever
duration: 90

# List of commands used to test the target
cmds:

  # Command used to start the target, "@@" is the input file
  - cmd: /usr/bin/objdump -x @@

  • Running forever on djpeg while accepting input from /dev/stdin: 

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
    # Mayhemfile: configuration file for testing your target with Mayhem
# Format: YAML 1.1

# Project name that the target belongs to
project: libjpeg

# Target name (should be unique within the project)
target: djpeg

# Base image to run the binary in.
image: forallsecure/debian-buster:latest

# Commented out duration to run forever
# duration: 90

# List of commands used to test the target
cmds:

# Command used to start the target. "@@" is missing so
# input will be read from /dev/stdin.
- cmd: /usr/bin/djpeg

  • Simple single input from network on TCP port 80 using a public docker hub image (from sebp/lighttpd): 

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
    project: lighttpd-docker
target: lighttpd
image: sebp/lighttpd
duration: 90
cmds:
  - cmd: /usr/sbin/lighttpd -D -f /etc/lighttpd/lighttpd.conf
    network:
      client: false
      url: tcp://127.0.0.1:80
      timeout: 1.0

2. Analyze the bacsrv network application and determine its listening port.
  • The strace utility can confirm bacsrv listens on port 47808. 
$ strace -Tfe trace=bind ./bacsrv
BACnet Server Demo
BACnet Stack Version 0.9.1
BACnet Device ID: 260001
Max APDU: 1476
bind(3, {sa_family=AF_INET, sin_port=htons(47808), sin_addr=inet_addr("0.0.0.0")}, 16) = 0 <0.000018>
3. Configure network options in Mayhemfile.
  • When analyzing a network application, you need to tell Mayhem which network port to use and whether the application uses TCP or UDP. We show a Mayhemfile for bacsrv that shows how to specify this information.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
project: bacsrv
target: bacsrv
duration: 90
advanced_triage: true

cmds:
  - cmd: /root/tutorial/bacsrv/bacsrv

    # For a network program, you add a "network" specification block
    network:
      url: udp://localhost:47808  # protocol, host and port to analyze
      client: false            # target is not a client program
      timeout: 2.0                # max seconds before timing out on network
4. Perform a Mayhem run on bacsrv.

  • The following script executes a Mayhem run on the bacsrv target:

    #!/bin/sh

# Package bacsrv
mayhem package ./bacsrv -o /tmp/bacsrv-pkg

# Write your Mayhemfile (here copying over provided one)
cp Mayhemfile /tmp/bacsrv-pkg/Mayhemfile

# Run Mayhem!
id=$(mayhem run /tmp/bacsrv-pkg)

# Wait for the run to finish
mayhem wait $id

# Sync the test suite to the "testsuite" directory.
mayhem sync /tmp/bacsrv-pkg
5. Manually package dependencies for lighttpd.

  • The following script packages up the dependencies for lighttpd correctly:

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
    #!/bin/sh

# Run the package command
mayhem package ./1.4.15/sbin/lighttpd -o /tmp/lighttpd-pkg

# Copy over all dependencies manually
cp -R ./1.4.15 /tmp/lighttpd-pkg/root/root/tutorial/lighttpd/
cp -R /www /tmp/lighttpd-pkg/root/

# Copy over the Mayhemfile needed to run lighttpd
cp Mayhemfile /tmp/lighttpd-pkg/

# Copy over an example test suite. This speeds up the example by
# seeding analysis with an example input.
cp testsuite/* /tmp/lighttpd-pkg/testsuite/

# Run mayhem on the package
id=$(mayhem run /tmp/lighttpd-pkg/)

# Wait for Mayhem to finish. It will succeed this time because we included
# lighttpd's dependencies.
mayhem wait $id
6. Perform a Mayhem run on lighttpd.

  • The following script executes a Mayhem run on lighttpd:

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
    project: lighttpd
target: lighttpd-1-4
duration: 90

cmds:
- cmd: /root/tutorial/lighttpd/1.4.15/sbin/lighttpd -D -f /root/tutorial/lighttpd/1.4.15/etc/lighttpd.conf
    network:
    client: false
    timeout: 5.0
    url: tcp://localhost:8080