Skip to content

advanced

Using libFuzzer, AFL, and honggfuzz OSS fuzzers with Mayhem

llvm-logo google-logo

In this lesson, we'll walk through how to use the three most popular open-source software fuzzing frameworks with Mayhem: libFuzzer, AFL, and honggfuzz.

Estimated Time: 15 minutes

By the end of this lesson, you will be able to:

  1. Define the AFL, libFuzzer, and honggfuzz fuzzing frameworks.
  2. Configure the Mayhemfile to use each respective fuzzing framework for the testme target.
  3. Perform Mayhem runs on the testme target using each fuzzing framework.

Run through the lesson:

See prerequisites before beginning.

  1. Package the libFuzzer-instrumented testme binary using the mayhem package command:

    mayhem package ./testme -o /tmp/libfuzzer-pkg

  2. Execute a Mayhem run on the libFuzzer-instrumented testme binary using the mayhem run command with the following Mayhemfile:

    1
2
3
4
5
6
7
8
9
    project: libfuzzer
target: testme
duration: 90
advanced_triage: false

cmds:
  - cmd: /root/tutorial/oss-fuzzers/libfuzzer/testme
    libfuzzer: true
    sanitizer: true

  3. Package the afl-instrumented testme binary using the mayhem package command:

    mayhem package ./testme -o /tmp/afl-pkg

  4. Execute a Mayhem run on the afl-instrumented testme binary using the mayhem run command with the following Mayhemfile:

    1
2
3
4
5
6
7
8
    project: afl
target: testme
duration: 90
advanced_triage: false

cmds:
  - cmd: /root/tutorial/oss-fuzzers/afl/testme @@
    afl: true

  5. Package the honggfuzz-instrumented testme binary (compiled with hfuzz-clang) using the mayhem package command:

    mayhem package ./testme -o /tmp/hfuzz-libfuzzer-pkg

  6. Execute a Mayhem run on the honggfuzz-instrumented testme binary (compiled with hfuzz-clang) using the mayhem run command with the following Mayhemfile:

    1
2
3
4
5
6
7
8
9
    project: honggfuzz-libfuzzer
target: testme
duration: 90
advanced_triage: false

cmds:
    - cmd: /root/tutorial/oss-fuzzers/honggfuzz-libfuzzer/testme
      honggfuzz: true
      sanitizer: true

  7. Package the honggfuzz-instrumented testme binary (compiled with hfuzz-gcc) using the mayhem package command:

    mayhem package ./testme -o /tmp/honggfuzz-pkg

  8. Execute a Mayhem run on the honggfuzz-instrumented testme binary (compiled with hfuzz-gcc) using the mayhem run command with the following Mayhemfile:

    1
2
3
4
5
6
7
8
    project: honggfuzz
target: testme
duration: 90
advanced_triage: false

cmds:
    - cmd: /root/tutorial/oss-fuzzers/honggfuzz/testme
      honggfuzz: true

You will need the following:

  • The Mayhem CLI installed and authenticated with the Mayhem server.

  • Access to the tutorial examples by running the pre-built Docker image.

    docker pull forallsecure/tutorial:2.5
docker run -ti --privileged --rm forallsecure/tutorial:2.5

Fuzzing with LibFuzzer

LibFuzzer is an in-process fuzzer produced by the LLVM project. An in-process fuzzer doesn't use fork() to re-execute the program on each invocation, but instead repeatedly calls a target function with new inputs within the same in-memory process.

To build and run this example, change directory (cd) to oss-fuzzers/libfuzzer. We use a new version of testme.c, which is shown below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
#include <stddef.h>
#include <stdint.h>

int testme(char *buf, unsigned len)
{
  unsigned ok;
  char *ptr = (char *) buf[0]; // Added for clang

  if(!ok) // Defect: uninitialized use of ok.
    ok = len;

  if (len < 3) return 1;

  if(buf[0] == 'b')
    if(buf[1] == 'a')
      if(buf[2] == 'd') {
        return (*ptr == 0x0);      // Defect: OOB reference.
      }
  return 0;
}

int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size)
{
  testme((char *) Data, Size);
  return 0;
}

The changes are:

  • Line 17: the previous testme.c used the abort() function to create an improper input validation defect. We've replaced the abort() function with an invalid memory access.

  • Line 22-26: LibFuzzer requires you write a test driver. We cover test drivers in more detail later. For now, the important part is that LibFuzzer requires you specify the function to test, and passes two arguments: Data and Size. The two arguments are filled in by the fuzzer.

To use LibFuzzer with Mayhem, you need to specify libfuzzer: true for the command (cmd) to execute. In addition, you need to specify that the target is built with a sanitizer with sanitizer: true.

Note

You use sanitizer: true for any sanitizer (UndefinedBehaviorSanitizer, AddressSanitizer, MemorySanitizer, etc.).

Here is a complete Mayhemfile:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
# A user-meaningful name to group like targets
project: libfuzzer

# A user-meaningful name of the target
target: testme-libfuzzer

# Time to spend fuzzing.
duration: 90

# Perform advanced triage and diagnosis of each test case.
advanced_triage: false

# List of executable programs to analyze.  For most use, you only
# need one.
cmds:

    # The full path to the executable program to test.
    #
    # libfuzzer targets do not accept a filename.
    - cmd: /root/tutorial/oss-fuzzers/libfuzzer/testme
      libfuzzer: true   # this is a libfuzzer target
      sanitizer: true        # A sanitizer was used (the average case)

To run Mayhem, execute the following commands:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
#!/bin/sh

# Package testme.
mayhem package ./testme -o /tmp/libfuzzer-pkg

# Copy over our Mayhemfile
cp Mayhemfile /tmp/libfuzzer-pkg

# Copy over our tests
cp testsuite/* /tmp/libfuzzer-pkg/testsuite/

# Run mayhem. The run ID is saved to $id
id=$(mayhem run /tmp/libfuzzer-pkg)

# Wait for the run to finish
mayhem wait $id

# Sync the test suite to the "testsuite" directory.
mayhem sync /tmp/libfuzzer-pkg

Go to the Mayhem UI, and you should see Mayhem fuzzing the target!

Fuzzing with AFL

AFL is a well-known process-based fuzzer. To use AFL, you need to recompile your application with the AFL toolchain. The toolchain inlines AFL instrumentation into the executable, which is used to guide the fuzzer.

To build and run this example, change directory (cd) to oss-fuzzers/afl. In that directory, you will find:

  • testme.c, which is the same vulnerable application from previous lessons.

  • Makefile, which is a UNIX Makefile to build the target with AFL instrumentation. We use the afl-gcc command, which uses gcc as in our previous examples. To build the target run make clean && make

  • The Mayhemfile, as shown below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# A user-meaningful name to group like targets
project: afl

# A user-meaningful name of the target
target: testme-afl

# Time to spend fuzzing.
duration: 90

# Perform advanced triage and diagnosis of each test case.
advanced_triage: false

# List of executable programs to analyze.  For most use, you only
# need one.
cmds:

    # The full path to the executable program to test.
    #   - "@@" is the  to test.
    #     For programs that take input from a file, use '@@' to mark
    #     the location in the target's command line where the input
    #     file name should be placed. Mayhem will substitute this
    #     for you.
    - cmd: /root/tutorial/oss-fuzzers/afl/testme @@
      afl: true

Note that we've added afl: true to the Mayhemfile. This tells Mayhem that the target is compiled with AFL.

To run Mayhem, execute the following commands:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
#!/bin/sh

# Package testme-v1 library dependencies.
mayhem package ./testme -o /tmp/afl-pkg

# Copy over our Mayhemfile
cp Mayhemfile /tmp/afl-pkg

# Run mayhem. The run ID is saved to $id
id=$(mayhem run /tmp/afl-pkg)

# Wait for the run to finish
mayhem wait $id

# Sync the test suite to the "testsuite" directory.
mayhem sync /tmp/afl-pkg

Go to the Mayhem UI, and you should see Mayhem fuzzing the target!

Fuzzing with honggfuzz

Honggfuzz is a security oriented, feedback-driven, evolutionary, easy-to-use fuzzer with several analysis options. Honggfuzz provides compiler wrappers for clang and GCC such as hfuzz-clang and hfuzz-gcc, respectively, which allow for compiling in different forms of compile time coverage instrumentation.

Users can use either the hfuzz-clang or hfuzz-gcc compiler wrappers interchangeably.

Info

Honggfuzz has several modes; however, we'll show you how to use LLVMFuzzerTestOneInput or HF_ITER style test drivers to compile honggfuzz persistent mode binaries for this lesson. See honggfuzz's persistent mode for more information.

Mayhem supports fuzzing honggfuzz persistent mode binaries compiled using hfuzz-clang. Here we'll show how to compile using libFuzzer-style test drivers (e.g. LLVMFuzzerTestOneInput).

To build and run this example, change directory (cd) to oss-fuzzers/honggfuzz-libfuzzer. You will find:

  • testme.c, which is the same vulnerable application from the libFuzzer example above.
  • Makefile, which is a UNIX Makefile to build the target. The Makefile uses hfuzz-clang.
  • The Mayhemfile, as shown below:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# A user-meaningful name to group like targets
project: honggfuzz-libfuzzer

# A user-meaningful name of the target
target: testme-honggfuzz-libfuzzer

# Time to spend fuzzing.
duration: 90

# Perform advanced triage and diagnosis of each test case.
advanced_triage: false

# List of executable programs to analyze.  For most use, you only
# need one.
cmds:

    # The full path to the executable program to test.
    # You do not need "@@" for libfuzzer, including under honggfuzz
    # when clang/libfuzzer is used to create.
    #
    # However, you *do* need to include the sanitizer flag with honggfuzz
    # libfuzzer targets.
    - cmd: /root/tutorial/oss-fuzzers/honggfuzz-libfuzzer/testme
      honggfuzz: true
      sanitizer: true

Note

You will need to specify honggfuzz: true and sanitizer: true. The honggfuzz: true instructs Mayhem to use the honggfuzz fuzzer, while sanitizer: true tells Mayhem that the binary is compiled with a sanitizer.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
#!/bin/sh

# Package testme.
mayhem package ./testme -o /tmp/hfuzz-libfuzzer-pkg

# Copy over our Mayhemfile
cp Mayhemfile /tmp/hfuzz-libfuzzer-pkg

# Copy over our tests
cp testsuite/* /tmp/hfuzz-libfuzzer-pkg/testsuite/

# Run mayhem. The run ID is saved to $id
id=$(mayhem run /tmp/hfuzz-libfuzzer-pkg)

# Wait for the run to finish
mayhem wait $id

# Sync the test suite to the "testsuite" directory.
mayhem sync /tmp/hfuzz-libfuzzer-pkg

Go to the Mayhem UI, and you should see Mayhem fuzzing the target!

Warning

If you do not have the sanitizer: true with an sanitizer target Mayhem will not correctly report found bugs.

Honggfuzz with hfuzz-gcc

Mayhem also supports fuzzing honggfuzz persistent mode binaries compiled using hfuzz-gcc. Here we'll show how to compile binaries using HF_ITER style test drivers.

Info

Mayhem can also run non-persistent mode binaries compiled with Honggfuzz instrumentation. However, non-persistent mode binaries will be treated the same as non-instrumented binaries and is not recommended.

Let's view a basic example! Change directory (cd) to oss-fuzzers/honggfuzz. Here you will find the following:

  • testme.c, which is a modified version of the testme application from previous lessons.
  • Makefile, which is a UNIX Makefile to build the target. The Makefile uses hfuzz-gcc. This adds special honggfuzz instrumentation.
  • Mayhemfile, which is the configuration file for the Mayhem run.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
#include <stddef.h>
#include <stdint.h>

int testme(char *buf, unsigned len)
{
  if(len >= 3)
    if(buf[0] == 'b')
      if(buf[1] == 'u')
        if(buf[2] == 'g') {
          return 1/0;      // Defect: divide-by-zero.
        }
  return 0;
}

extern int HF_ITER(uint8_t** buf, size_t* len);

int main(void) {
    for (;;) {
        size_t len;
        uint8_t *buf;

        HF_ITER(&buf, &len);

        testme(buf, len);
    }
}

Notice that the modified testme.c uses the HF_ITER function, indicating that this is a persistent mode binary.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
# A user-meaningful name to group like targets
project: honggfuzz

# A user-meaningful name of the target
target: testme-honggfuzz

# Time to spend fuzzing.
duration: 90

# Perform advanced triage and diagnosis of each test case.
advanced_triage: false

# List of executable programs to analyze.  For most use, you only
# need one.
cmds:

    # The full path to the executable program to test.
    # "@@" should be removed for persistent mode binaries.
    - cmd: /root/tutorial/oss-fuzzers/honggfuzz/testme
      honggfuzz: true

As a result the @@ should be removed from the cmd parameter for the persistent mode testme binary.

To run Mayhem, execute the following commands:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
#!/bin/sh

# Package testme-v1 library dependencies.
mayhem package ./testme -o /tmp/honggfuzz-pkg

# Copy over our Mayhemfile
cp Mayhemfile /tmp/honggfuzz-pkg

# Run mayhem. The run ID is saved to $id
id=$(mayhem run /tmp/honggfuzz-pkg)

# Wait for the run to finish
mayhem wait $id

# Sync the test suite to the "testsuite" directory.
mayhem sync /tmp/honggfuzz-pkg

Go to the Mayhem UI, and you should see Mayhem fuzzing the target!

✏️ Summary and Recap

In this lesson, you learned how to fuzz targets with the AFL, libFuzzer, and honggfuzz fuzzing frameworks.

I learned how to...

1. Define the AFL, libFuzzer, and honggfuzz fuzzing frameworks.
  • AFL is a well-known process-based fuzzer. To use AFL, you need to recompile your application with the AFL toolchain. The toolchain inlines AFL instrumentation into the executable, which is used to guide the fuzzer.
  • LibFuzzer is an in-process fuzzer produced by the LLVM project. An in-process fuzzer doesn't use fork() to re-execute the program on each invocation, but instead repeatedly calls a target function with new inputs within the same in-memory process.
  • Honggfuzz is a security oriented, feedback-driven, evolutionary, easy-to-use fuzzer with interesting analysis options. Honggfuzz has the ability to compile in different types of instrumentation, including it's own with hfuzz-gcc and libFuzzer.
2. Configure the Mayhemfile to use each respective fuzzing framework for the testme target.

  • AFL Mayhemfile:

    1
2
3
4
5
6
7
    project: afl
target: testme
duration: 90
advanced_triage: false
cmds:
  - cmd: /root/tutorial/oss-fuzzers/afl/testme @@
    afl: true

  • LibFuzzer Mayhemfile:

    1
2
3
4
5
6
7
8
9
    project: libfuzzer
target: testme
duration: 90
advanced_triage: false

cmds:
  - cmd: /root/tutorial/oss-fuzzers/libfuzzer/testme
    libfuzzer: true   # this is a libfuzzer target
    sanitizer: true        # A sanitizer was used (the average case)

  • Honggfuzz Mayhemfile:

    1
2
3
4
5
6
7
8
9
    project: honggfuzz-libfuzzer
target: testme
duration: 90
advanced_triage: false

cmds:
  - cmd: /root/tutorial/oss-fuzzers/honggfuzz-libfuzzer/testme
    honggfuzz: true
    sanitizer: true
3. Perform Mayhem runs on the testme target using each fuzzing framework.
  • Execute the mayhem run command for each of the AFL, libFuzzer, and honggfuzz targets.