Application Security Testing (AST) Tools

What is...

Fuzz testing?

Fuzz testing, or fuzzing, is a dynamic application security testing technique for negative testing. Fuzzing aims to detect known, unknown, and zero-day vulnerabilities. Fuzzers send malformed inputs to targets. Their objective is to trigger bad behaviors, such as crashes, infinite loops, and/or memory leaks. These anomalous behaviors are often a sign of an underlying vulnerability. Refer to this webinar for more information.

Random fuzz testing?

Random fuzzing is the act of sending random inputs to an application. There is no systematic method to the generation of these test cases, and they do not resemble a valid input. As a result, most inputs do not penetrate the application, leading to low code coverage. For a random fuzzer to reach the same level of coverage as its more effective counterparts, it requires significantly more time and effort from security experts. Refer to this blog for more information.

Template fuzz testing?

Template fuzzers utilize manually supplied custom inputs and modify them to include anomalies, and we also include generational and protocol fuzzers in this subcategory. As a group, they are more effective than random fuzzers, because they resemble valid inputs. The probability for these test cases to penetrate an application is higher than a random fuzzer’s test cases. However, there are several drawbacks. Template fuzzers randomly include anomalies without an understanding of common error-detection techniques. As a result, their test cases are often blocked. Second, templates have limitations. Not only is the test case quality limited to the valid input template provided, but they also do not fuzz beyond the scope of the template. A template fuzzer can be effective for fuzzing a single function, given that it was provided an expertly crafted template. However, scaling template fuzzing requires expertise. Generational fuzzers are a bit better and understand the inner workings of its input type. These tests are written to resemble a valid input, while evading common error-detection techniques. While vendor-maintained generational fuzzers require significantly less expertise than template fuzzers for testing, they still require expertise for operation and results interpretation. Most commercial fuzzers available today are template/generational fuzzers.

Guided fuzz testing?

Guided fuzzers are intelligent, containing the capability to monitor and leverage the target’s behavior to autonomously generate new, custom test cases on-the-fly. These fuzzers have scoring capabilities that measure the effectiveness of the test cases it sends. High-scoring test cases influence the new set of test cases generated and sent. Unlike the aforementioned fuzzer types, guided fuzzers only require a way to monitor its target and sent inputs to them. They are not supplied with starter test cases. In recent years, modern fuzzers have grown in sophistication and wield robust automation capabilities. Guided fuzzers relieve significant strains introduced from manual test case creation. However, if users desire deep fuzzing analysis, guided fuzzers require expert assistance. Mayhem, from ForAllSecure, provides automated guided fuzz testing.

A fuzz test harness?

Fuzz testing requires the use of an entry-point executable (EPE). Many files can natively take input from a command line, but many more can not. So in order to test these you will need to create your own EPE, or fuzz test harness. For example, with a Dynamic Link Library there is not a EPE, so you must have a testing harness to pass input from the command line into the DLL function we want to test.

A seed test suite?

A seed test suite is a set of valid test case inputs that serve as a starting point for fuzzing a target.

Why fuzz test?

Fuzz testing should be a part of every SDLC. It looks at the runtime behavior of the code. It provides more code coverage than SAST or SCA. Refer to this infographic for more information.

How effective is fuzz testing?

Google finds 80% of bugs in its Chrome browser with fuzz testing, while the remaining 20% is found with other techniques or in production. Taking an adversarial approach to your software is a proven and an effective method for addressing the "low hanging fruit" or the weaknesses that are most attractive to malicious actors. While fuzzing is effective, it's also dependent on your tool of choice and the expert behind the tool. Refer to this webinar for more information.

What are the different types of fuzzing techniques?

There are three types of fuzz testing: random, template, and guided.

What problems does fuzz testing help me solve?

Fuzzers send malformed inputs to targets. Their objective is to trigger bad behaviors, such as crashes, infinite loops, and/or memory leaks. These anomalous behaviors are often a sign of an underlying vulnerability.

If fuzzing has been around for over three decades, why isn’t it more known?

When developers are creating software they are generally not looking at all possible ways their software can behave badly. Unfortunately, there are infinite ways to do something wrong. Although the effectiveness of fuzz testing has been well-documented, fuzzing, as a whole, has been criticized for its shallow analysis and inability to penetrate through the peripheral layers of an application. Despite criticism, fuzz testing is a proven method and a recommended practice for organizations who build their own software and rely on supplier software for business productivity. Newer guided fuzzing solutions offer scoring capabilities that measure the effectiveness of the test cases it sends. Guided fuzzers do rely on sample inputs, or a test suite, for initial guidance to explore a program however, thereafter, it monitors and leverages its target’s behavioral feedback to generate new, customized test cases on-the-fly. These newly generated test cases aim to incrementally test new sections of code, checking the security of each new region it successfully penetrates and improving usability.

What types of vulnerabilities can fuzz testing find?

Perhaps the most famous example of fuzz testing is the discovery of the buffer overflow defect in certain versions of OpenSSL, better known as Heartbleed. An updated list of recent vulnerabilities Mayhem uncovered can be found here: https://go.forallsecure.com/vulnerabilities-lab

What are the benefits of fuzz testing?

1. Ability to find unknown vulnerabilities
2. High accuracy, low false positives
3. True automation with the SDLC
4. Security testing done at machine speed and scale
5. Shift-Left DAST

Who benefits from fuzz testing?

Everyone. Seriously, the end users benefit from having secure applications that won’t leak their personal information or crash in life critical situations. Organizations benefit in that defects are easier and therefore less costly to fix in development. And government agencies benefit in that third-party applications can be rigorously tested for unknown vulnerabilities before deployed in mission critical systems.

What types of organizations benefit most from fuzz testing?

Organizations that develop their own code benefit the most because they have the most control over the source code. However, fuzz testing can also be applied to third-party binaries and therefore bring benefit to those who procure third-party applications.