Skip to content

How long Should I Run API Testing?

Unlike other testing methods, like unit and integration tests, the definition for being 'done' for a fuzz test is, well, fuzzy. In general, the longer you run mapi, the more of your endpoints, and the code behind them, will be covered.

Your run duration depends on a few properties of your target API. For example:

  • Latency between CLI and API - local APIs will typically have lower latency
  • API Configuration - is your API configured with multiple threads/workers to handle high concurrency?
  • Number of endpoints - is your API a few or hundreds of endpoints?

The following will help guide you toward the right choice for testing your target API.

Concurrent Requests

Mayhem will automatically run in concurrent execution mode as long as you are running on a machine with more than one CPU core. The higher the concurrency, the more requests can be sent in parallel to your target API. Mayhem defaults its concurrency level to "4", or the number of CPU cores -- whichever is less.

You can increase the number of parallel fuzzers by passing the --concurrency flag to mapi run command.

For example, to run with 8 parallel threads, pass --concurrency 8 to mapi run:

mapi run \
     <target_name> \
     auto \
     <path_to_api_specification> \
     --url "https://example.com" \
     --concurrency 8

Automatic run duration

We recommend running Mayhem with automatic duration if it is your first time using the tool, or aren't sure how long to run your scans. When running with auto duration, mapi will attempt to make a request to every endpoint at least 100 times.

You can run with automatic duration by passing the auto string to the mapi run command. For example:

mapi run \
     <target_name> \
     auto \
     <path_to_api_specification> \
     --url "https://example.com"

When should I use an automatic run duration?

  • It is your first time trying Mayhem
  • You want mapi to automatically adjust to your API as it changes size

Tweaking the minimum request count

Info

When running with auto duration, mapi will attempt to make a request to every endpoint at least 100 times.

When running in auto mode, you can change the threshold of the minimum number of endpoint request with the --min-request-count flag. For instance, you may wish to increase the minimum request threshold in order to exercise your API more than the default of 100.

For example:

mapi run \
     <target_name> \
     auto \
     <path_to_api_specification> \
     --min-request-count 500
     --url "https://example.com"

Fixed Time run duration

mapi will continuously scan your target API until the duration has been reached when running with a fixed duration. The duration time is specified as part of the mapi run command.

For example:

mapi run \
     <target_name> \
     5min \
     <path_to_api_specification> \
     --url "https://example.com"

The duration syntax is fairly flexible, allowing callers to specify time in seconds, minutes or hours.

 - Run for 30 seconds                 [30s or 30sec]
 - Run for 90 minutes                 [90m or 90min]
 - Run for 1 hour                     [1h  or 1hr]
 - Run for 2 hours and 20 minutes     [2h20m]

When should I use a fixed time run duration?

A fixed duration scan is ideal if you want to run Mayhem, but are limited in your computing time/resources. For instance, if you are running mapi as part of your continuous build pipeline, you may not want a step running that takes an unpredictable amount of time. By setting a fixed duration, you can ensure that mapi you are explicit about how long the mapi step will take.