beginner

Introduction

Welcome to your first lesson using Mayhem to test your code! In this lesson, we will introduce you to Mayhem and discuss its benefits, and explain the underlying fundamentals of how Mayhem uses fuzzing to test code.

---

Estimated Time: 10 minutes

Objectives

By the end of this lesson, you will be able to:

1. Define why you should use Mayhem to test your code.
2. Define what Mayhem is.
3. Understand how fuzzing works.
4. Explain how Mayhem tests targets.
5. Explain how to get started with Mayhem.

Why Use Mayhem to Test Your Code?

Now more than ever, ensuring the security of applications is paramount. Failure to conduct proper application security testing can expose vulnerabilities in code, giving hackers an opportunity to exploit these weaknesses and compromise the application's integrity and data.

Traditional application security testing methods were time-consuming and required significant manual effort, resulting in increased overhead costs for enterprises and development teams. However, Mayhem offers a solution that enables users to deliver secure and reliable software with greater precision, efficiency, and cost-effectiveness. Mayhem achieves this through its advanced fuzz testing solution, which automates the detection and validation of defects in application security testing. Specifically, Mayhem helps reduce the expenses associated with application development and auditing by:

1. Providing Proof of Defects: Mayhem automatically identifies bugs by generating a test case that demonstrates the existence and reachability of a defect.
2. Generating Test Suites: Mayhem automates the generation, indexing, and preservation of test cases during security testing. It also performs re-tests for each application version. Test suites significantly reduce the time and cost required to conduct comprehensive and repeatable security tests on your application.
3. Following Best Practices: Mayhem automatically checks compiled executable programs to see if they are hardened against exploits.

What Exactly is Mayhem?

Mayhem is a comprehensive application security testing platform designed for developers and security practitioners to collaborate seamlessly. It provides efficient organization, management, and testing of applications to identify defects and security vulnerabilities.

At its core, Mayhem leverages an advanced fuzzer that examines running Linux processes. It combines the proven techniques of instrumentation-guided fuzzing with the innovation of symbolic execution.

Tip

Watch The Science Behind Mayhem for an easy-to-understand breakdown of Mayhem and its instrumentation-guided fuzzing and symbolic execution capabilities.

What is Fuzzing?

Mayhem uses fuzzing to test and secure software. Fuzzing, also known as fuzz testing, is a software testing technique that involves injecting random data into an application to identify bugs and defects associated with program failures or crashes. By continuously running the application with different inputs, fuzzing aims to explore as much of the underlying code as possible and uncover edge cases that could potentially lead to crashes or other issues.

To illustrate how fuzzing works, let's consider a simple example.

Fuzzing a Vulnerable Application

When you first learn to program, you print out Hello World!; when you first learn to fuzz, you try to find bugs and create test suites for the testme application.

Test suites are collections of various test cases that are used to input and run differing parts of a target application. Therefore, if a defect is found for a target application, a resulting test case will be saved that can be used to re-create the defect scenario.

Let's see how this works by taking a look at the source code of our first vulnerable app, testme:

#include <stdio.h>
#include <string.h>

int fuzzme(char *buf)
{
  if(buf[0] == 'b')
    if(buf[1] == 'u')
      if(buf[2] == 'g') {
        abort(); // Defect: Sends a SIGABRT signal
      }
  return 0;
}

int main(int argc, char *argv[])
{
  FILE *f;
  char buf[12];

  if(argc != 2){
    fprintf(stderr, "Must supply a text file\n");
    return -1;
  }
  f = fopen(argv[1], "r");
  if(f == NULL){
    fprintf(stderr, "Could not open %s\n", argv[1]);
    return -1;
  }
  if(fgets(buf, sizeof(buf), f) == NULL){
    fprintf(stderr, "Could not read from %s\n", argv[1]);
    return -1;
  }
  fuzzme(buf);
  return 0;
}

Here we see that the testme application contains two functions:

1. main
2. fuzzme

The main function indicates that when a file is properly supplied as an argument to the testme application, it will invoke the fuzzme function in which the file contents are parsed character by character. If the contents of the given file input spells out "bug", then an abort() function will occur and consequently crash the program by raising a SIGABRT signal.

The use of the abort() function within the testme application will be classified as the following defect when the test case "bug" is given as a file input:

1. Line 9: CWE-20 - Improper input validation defect.

Running the testme application through Mayhem will automatically find the SIGABRT defect, but also create a test suite, or set of inputs, that exercise different code paths.

Note

Code paths within a program that execute due to varying inputs can also be called edges or edges covered in the context of Mayhem's security testing metrics.

For example, imagine a test suite with the four following inputs passed into our testme program:

1. mom would execute the false branch on line 6, returning 0.
2. bob would execute the true branch on line 6 and the false branch on line 7, returning 0.
3. bun would execute the true branches on lines 6-7 and the false branch on line 8, returning 0.
4. bug would execute the true branches on lines 6-8, returning the SIGABRT defect.

The set of inputs, {mom, bob, bun, bug}, execute every branch of the program, and as a result, makes an excellent test suite!

Your packaged target application will then be uploaded to the Mayhem server and tested for security vulnerabilities! Generated test cases can then be synced from the Mayhem server back to the client machine.

How Does Mayhem Work?

Once the runtime environment is provided, Mayhem will run your application and automatically perform the following tasks:

1. Exploitability Factors: Checks to sees if the application is hardened or protected against exploitation.
2. Fuzzing and Symbolic Execution: Tests the target application with different input permutations and monitors for any abnormal behaviors/defects.
3. Triage and Diagnosis: Validate discovered defects and provide coverage analysis.
4. Automated Regression Testing: Validate current results against previous results with saved test suites.

Exploitability Factors

When Mayhem tests a target application, it checks to see if your compiled application is protected or "hardened". A hardened application is one in which its vulnerabilities–-both known and unknown–-are harder to exploit. The developer can harden an application by choosing whether to enable certain compile-time flags.

The current best practices or exploitability factors for hardening an application include:

• ASLR: Address Space Layout Randomization (ASLR) makes it hard for an attacker to predict what memory will look like at runtime, making it harder to reliably exploit a system. ASLR is an operating system feature that is only available if the application was compiled as a “Position Independent Executable” (PIE).
• FORTIFY_SOURCE: The FORTIFY_SOURCE macro provides lightweight support for detecting buffer overflows in various functions that perform operations on memory and strings.
• StackGuard: StackGuard inserts a stack canary into the stack during function entry and checks their value prior to function exit, terminating execution if the value has been corrupted. StackGuard is used to detect stack-based buffer overflow exploits.
• DEP: Data Execution Prevention (DEP) is a set of technologies that perform additional checks on memory to help prevent malicious code from running on your system.

An application will be significantly more difficult to exploit by an attacker if all four hardening techniques are enabled.

Fuzzing and Symbolic Execution

Traditionally, fuzzing was just about feeding random input into an application to test for abnormal behavior—similar to a million monkeys randomly typing on a keyboard. In fact, back in 1998, a professor at the University of Wisconsin named Bart Miller noticed that 33% of UNIX applications would crash if given random input—just like what a monkey would type!

Traditional fuzzing is like a monkey typing on a keyboard.

Mayhem is smart, efficient, and automatically tests your apps with the latest fuzzing and symbolic execution technology.

Mayhem comes with both instrumentation-guided fuzzing and symbolic execution. The two techniques have different performance curves, but are complementary and provide the most advanced form of Behavior Testing.

Instrumentation-guided fuzzing is much more intelligent – and efficient – than randomly generating inputs. The key idea is to add instrumentation that analyzes a program as it executes, and use that knowledge to intelligently pick a new input.

Symbolic execution is even more advanced. Symbolic execution is like program verification, but with a twist. Instead of verifying a program is secure, we verify it is insecure by producing a Proof of Defect. Symbolic execution models each program execution in formal logic, and then uses a mathematical solver to explore each behavior of the program.

Note

Check out the Mayhem Technology documentation to get a comprehensive understanding of Mayhem's fuzzing capabilities.

Triage and Diagnosis

If a program crashes, you know something is wrong, but likely don’t know why. Mayhem helps provide you more insight by automatically performing triage and diagnosis to give you more information about each defect.

For each defect, Mayhem will provide you:

• A backtrace. The backtrace identifies which functions were called, helping security practitioners and developers more quickly identify the root cause of a problem.
• The register and memory state at time of crash. This information can be used to provide context regarding the crash to see if the conditions are the same as that of another crash.

Automated Regression Testing

When Mayhem tests a target application, Mayhem generates test cases where each test case represents unique code coverage for the underlying target. Therefore when a defect is found, a unique test case represents the particular defect and can be re-run to reproduce the error.

As a user, you may want to know when defects are fixed. Thus, Mayhem saves these test cases to then automatically replay the stored test suite and let you know the number of defects fixed compared to previous runs. This is automated regression testing.

What Do I Need to Get Started with Mayhem?

Determining Program Compatibility with Mayhem

First, determine if your application is compatibile with Mayhem. Mayhem can analyze compiled binaries written in languages like C/C++, Go, Rust, Java, and Python that read from a file, standard input, or from the network via a TCP or UDP socket. Mayhem also handles user-land (containerized) Linux applications. Mayhem supports binaries that run on the x86, x64, ARM, and MIPS architectures.

Tip

Check out the Support Matrix for a complete breakdown of Mayhem program compatibility.

Here are some helpful tips for quickly determining if your application is compatible:

• Does the Linux file <app> command say that your application is a Linux ELF file? Mayhem requires the application to be an ELF file.
• Is the application a user-land application? If you can dockerize the application, it is a user-land application. If the application requires special hardware, then you will need to write a test driver to remove the hardware dependencies.
• Is a GUI the only way to interact with the application? Mayhem does not analyze GUIs currently.

Packaging Targets

Before Mayhem can run and detect defects associated with your program, it requires the complete runtime environment for the application. This includes the executable itself, related libraries, environment variables, and configuration settings. Mayhem supports two different methods for providing the runtime environment:

• Docker: Docker is the recommended method for ingesting applications into Mayhem and is a light-weight mechanism for packaging the complete runtime of an application and its dependencies. Given a Docker image, Mayhem will spin up and tear down containers as needed to run the application in the containerized environment.
• Mayhem packages: If Docker is not an option, you can package a Linux application using the mayhem package command to statically analyze your app and gather as many dependencies as it can infer.

Executing Runs via Mayhem UI or Mayhem CLI

Users can execute their runs using either the Mayhem UI or Mayhem CLI.

The Mayhem UI is an intuitive web interface that allows users to test their applications, or targets, and analyze the results. Using the Mayhem UI, users can test containerized applications by creating a Mayhem Run and specifying a Docker image as a source. Mayhem can ingest local Docker images uploaded to Mayhem's internal Docker registry, or Docker images publicly available via the Docker Hub registry.

The Mayhem CLI is a command line interface that users can download and install on their local machines to execute Mayhem operations from their terminals. Users looking to use the Mayhem CLI will need to authenticate their installed Mayhem CLI with a deployed Mayhem server using a generated API token prior to Mayhem CLI usage.

Info

See Generating API Tokens for more info on generating an API token for Mayhem CLI authentication.

✏️ Summary and Recap

In this lesson we learned why you would use Mayhem, what Mayhem is, and the underlying concepts of how Mayhem works and what you needed to get started.

---

I learned how to...

1. Define why you should use Mayhem to test your code.

• Applications now more than ever need to be secure. Without proper application security testing, vulnerabilities in an application's code can open up doors to hackers that can target the underlying security flaws and compromise the integrity of the application and its data.
• Traditional application security testing required both time and manual effort, driving up overhead costs for enterprises and their development teams. However, with Mayhem, users can deliver secure and reliable software with better precision while using less time, effort, and ultimately, costs.

2. Define what Mayhem is.

• Mayhem is an application security testing platform for developers and security practitioners alike to collaborate and easily organize, manage, and test their applications for defects and security vulnerabilities.
• At its core, Mayhem uses an advanced fuzzer that analyzes a running Linux process and combines the tried-and-true methods of instrumentation-guided fuzzing with the ingenuity of symbolic execution.

3. Understand how fuzzing works.

• Fuzzing, or fuzz testing, is a software testing technique that injects random data as inputs to an application to monitor and detect bugs/defects related to program failures or crashes. The application will re-run continuously, each with differing inputs, to attempt to cover as much of the underlying code as possible and seek out edge cases that may result in a crash.

4. Explain how Mayhem tests targets.

• Exploitability Factors: Checks to sees if the application is hardened or protected against exploitation.
• Fuzzing and Symbolic Execution: Tests the target application with different input permutations and monitors for any abnormal behaviors/defects.
• Triage and Diagnosis: Validate discovered defects and provide coverage analysis.
• Automated Regression Testing: Validate current results against previous results with saved test suites.

5. Explain how to get started with Mayhem.

• Determine if your application is compatible with Mayhem. Mayhem can analyze compiled binaries written in languages like C/C++, Go, Rust, and Java that read from a file, standard input, or from the network via a TCP or UDP socket. Mayhem also handles user-land (containerized) Linux applications. Mayhem supports binaries that run on the x86, x64, ARM, and MIPS architectures.
• Before Mayhem can run and detect defects associated with your program, it requires the complete runtime environment for the application:
  • Docker: Docker is the recommended method for ingesting applications into Mayhem and is a light-weight mechanism for packaging the complete runtime of an application and its dependencies.
  • Mayhem packages: If Docker is not an option, you can package a Linux application using the mayhem package command to statically analyze your app and gather as many dependencies as it can infer.