Mayhem Code Testing Release Notes

Release 2.6.0

Minor release, including new features, bug fixes, and improvements:

New Features

• ThreadSanitizer is now supported and appropriate defect types are now reported. Adds support for detecting occurrences of CWE-362, CWE-401, CWE-667, CWE-828, CWE-833 when ThreadSanitizer is enabled.
• UI now reports optimized and total test suite size, in addition to the number of test case reports processed.

Improvements

• Symbolic execution performance improvements, including better handling of large binaries, improved test case prioritization, and better handling of restarts.
• Test suite optimization performance improvements, and reduced run start up time for libFuzzer targets.
• Improved performance on libFuzzer targets when run in combination with symbolic execution.
• Better diagnostics when we are unable to start a run due to the initial test suite causing crashes or timeouts in the target program.

Bug Fixes

• Fix inaccurate statistics being reported from runs with symbolic execution enabled.
• Fix a null pointer dereference causing run failure in certain conditions when MFUZZ_COMPAT_LEVEL was set.
• Fix an issue preventing line coverage generation for certain golang targets.

Release 2.5.0

Minor release, including new features, bug fixes, and improvements:

New Features

• Added unified One Platform web interface for Mayhem code and API testing.

Improvements

• Restructured Mayhem Docs to include code and API testing content. Also improved navigation and general aesthetics.
• Improved error message when unable to pull a missing Docker image.

Bug Fixes

• mayhem init now prompts before overwriting existing output Mayhemfile. To always overwrite, use mayhem -y init.

Release 2.4.3

Patch release, including bug fixes:

Bug Fixes

Fixes an issue where non-admin users cannot start runs due to insufficient permissions.

Release 2.4.2

Patch release, including bug fixes:

Bug Fixes

Fixes a file descriptor leak in the Symbolic Executor.

Release 2.4.1

Patch release, including bug fixes:

Bug Fixes

Fixed aggregated metrics for number of runs per target in Mayhem UI project overview page.

Release 2.4.0

Minor release, including new features, bug fixes, and improvements:

New Features

• Added smoketests for libfuzzer targets which generate no coverage.
• Added a new coverage-driven symbolic execution scheduler that significantly improves coverage.

Bug Fixes

• Ensured Mayhem run overrides always use the latest Docker image referenced in the last Mayhem run.
• Fixed symbolic executor memory capping that was killing processed before they were reaching the cap.

Improvements

• Reduced wait frequency to avoid exhausting GitHub API quotas in multi-target builds.
• Tweaked symbolic executor default timeout to improve testcases throughput.
• Improved handling of instructions for which the symbolic executor doesn't have semantics.
• Tweaked symbolic executor semantics cache to significantly improve performance on small targets.

Release 2.3.4

Patch release, including bug fixes:

Bug Fixes

• Fixed support for images on Docker Hub with no v1 manifest present / added support for v2 image manifests, oci manifests, and manifest indices.
• Updated Mayhem macOS CLI certificates.

Release 2.3.3

Patch release, including bug fixes:

Bug Fixes

• Fixed termination event handling and exit code detection.
• Fixed an issue with corrupted images not loading properly in Mayhem Docs.
• Fixed an issue with out-of-sync instructions for the tutorial Java Base Executable target.
• Fixed lcov stats and added error logs for any >100% coverage scenarios.

Release 2.3.2

Patch release, including bug fixes:

Bug Fixes

• Removed project/runs counts from home/project/runs pages in the web UI.
• Fixed issue with the run filter buttons on the web UI home page.

Release 2.3.1

Patch release, including bug fixes:

Bug Fixes

• Fixed UI icons not displaying properly on Mayhem Run page.

Release 2.3

Minor release, including new features and improvements:

New Features

• Automatically enhance starting testsuites with corpora.
• Introduce a backlog pruning project setting (default is disabled). This feature is useful when you have have rapid updates on your targets and want Mayhem to always be running on your latest commit. When backlog pruning is enabled and a new run is started, all other pending runs for that target will be evicted.

Improvements

• Performance and scheduling improvements for symbolic executor engine.

Release 2.2.2

Patch release, including bug fixes:

Bug Fixes

• Skip Ghidra analysis if the target's executable segments exceeds 16MB.

Release 2.2.1

Patch release, including bug fixes:

Bug Fixes

• Fixes issue of unexpected number of test cases in downloaded corpus.

Release 2.2

Minor release, including new features:

Cool New Features

• Added Mayhem deployments on AWS infrastructure using terraform and EKS.
• Added compatibility for targets compiled with the latest AFL++ toolchain (extended forkserver protocol).

Release 2.1

Minor release, including new features, bug fixes, and improvements:

Cool New Features

• Improved libFuzzer performance.
• Support for AFL and libfuzzer instrumented targets on non-x86 architectures.
• Slow test cases are now being reported and users can view slow test cases in the run page UI.
• Added a service to execute Mayhem runs on idle targets to maximize usage of cluster resources/cores.

Bug Fixes

• Fixes issue correctly collecting block coverage from targets which spawn multiple processes.
• Fixes issue correctly detecting targets instrumented with UBSAN only.

Improvements

• Added better exception handling to automatic Mayhem Private Registry cleanup.

Release 2.0

Major release, including new features and improvements:

Cool New Features

• Users can use GitHub credentials to log into Mayhem via GitHub OAUTH.
• Administrators can now delete users along with their data.
• Added documentation for integrating Mayhem in a CI pipeline using the mCode GitHub Action.

Improvements

• Added new aggregate statistics for the organization overview/projects page within the Mayhem UI:
  • Number of projects in organization
  • Number of targets
  • Number of users
  • Total tests run
  • Total defects
  • Total test cases
• Added support for Python Atheris 2.X.
• Enhancements made to SARIF reports to enable usage with GitHub actions / CodeQL and the security scanning tab.
• Adjusted Mayhemfile/CLI override parameters:
  • Modified project and target fields to be optional (defaults to values myproject and mytarget when not specified, respectively).
  • Modified version field to be optional (defaults to the current mCode version for the deployed Mayhem instance).
  • Renamed baseimage to image.
  • Renamed namespace to owner.
  • Renamed memory_limit_in_mb to memory_limit.
  • Renamed chdir to cwd.
  • Renamed target_input to filepath.
  • Renamed network.is_client to network.client.
  • Renamed tests to testsuite.
  • Renamed corpus to testsuite.
  • Removed disable_se.
  • Removed disable_mayhem_fuzz.

Release 1.17.3

Patch release, including new bug fixes:

• Improved epoll handling for networking targets.

Release 1.17.2

Patch release, including new bug fixes:

• Fixed typo in docs C/C++ libFuzzer exercise.

Release 1.17.1

Patch release, including new bug fixes:

• Fixed failed Mayhemfile validations for ASAN and network binaries.

Release 1.17.0

Minor release, including new bug fixes:

Bug Fixes

• Added mixed case support (lower and uppercase) for user account names.
• Added semantic versioning and proper tracking across Mayhem components.

Release 1.16.2

Patch release, including new bug fixes:

Bug Fixes

• Fixes issue preventing runs from starting for Docker Hub targets in the UI.

Release 1.16.1

Patch release, including new features and improvements:

Cool New Features

• Added asciinema feature to documentation for terminal recordings.

Improvements

• Documentation URLs have been standardized to lowercase and dashes for spaces.
• Added an SVG for the Mayhem Documentation logo.

Release 1.16.0

Minor release, including new features and bug fixes:

Cool New Features

• Get a jump start with a new project in Mayhem by using the mayhem init command to automatically generate a Mayhemfile for a given Docker image.
• Mayhem can now be installed on RHEL8 systems!
• Admins can now set quotas for both the number of active runs users may have and the durations for runs; this can be set on a system-wide or user-specific basis.
• Admins can now invite users to their Mayhem instance simply by entering their email address. Invites can be re-sent and revoked as needed.

Release 1.15.2

Patch release, including new bug fixes:

Bug Fixes

• Added several HyperV daemon packages to support the installation of Mayhem on HyperV managed virtual machines.

Release 1.15.1

Patch release, including new bug fixes:

Bug Fixes

• Auto-remove previous configuration files from past Mayhem installations for on-prem deployments.
• Fixed a Docker re-tag issue that resulted in an additional forallsecure/tutorial image for on-prem deployments.
• Correctly compiled the honggfuzz target in forallsecure/tutorial to be a persistent honggfuzz binary.

Release 1.15.0

Minor release, including improvements and bug fixes:

Improvements

• Allow admin users to invite new members to your instance (requires email to be enabled).
• Removed 2 billion (2^32) test case limitation in terms of storage.

Bug Fixes

• Improved smoke testing stage accuracy for behavior testing tasks.
• Correctly log and present Java fuzzing performance statistics.
• Improved error handling and messaging for unsupported coverage targets.

Release 1.14.0

Minor release, including new features and improvements:

Cool New Features

• Added support in for using environment variables in Mayhemfiles, allowing for better customization and control of your Mayhem runs!
• Expands supported report format offerings by adding SARIF as a report output option for Mayhem runs.
• Concurrent runs are now allowed per target (parameterizable from the project settings page).

Improvements

• honggfuzz versions 1.9 through 2.4 are now supported by Mayhem.

Known Issues

• You may be unable to start runs with older versions of the Mayhem CLI - please update to the latest version!
• Due to some changes with how test cases/corpora are handled, it is possible you may not be able to re-run some older runs. If you encounter this, the workaround is to download the corpus of the run and start a new run with that corpus. You can download the corpus for a run via the “Download Corpus” link on that run's page or by running mayhem sync.

Release 1.13.2

Patch release, including:

• Improved handling for ioctl parameters in network daemons.

Release 1.13.1

Patch release, including:

• Rebuild forallsecure/debian-buster base image.

Release 1.13.0

Patch release, including new features and bug fixes:

Cool New Features

• Support for PowerPC targets (alpha).
• Added the ability for Mayhem to intelligently recover from certain classes of Mayhemfile configuration errors (beta).

Improvements

• Streamlined the Start New Run flow in the UI, allowing for quicker ingestion of new targets into Mayhem.
• New guides added for how to fuzz targets from various programming languages (C/C++, Golang, Rust, Java, Python, Ada).
• Enable users to download run defect reports as PDFs.

Bug Fixes

• Enable auto-upgrade for certificates on bare-metal installations.

Release 1.12.4

Patch release, including:

• Improved handling for memory leaks reported through advanced triage.

Release 1.12.3

Patch release, including:

• Mayhemfile update for Java standalone target tutorial.

Release 1.12.2

Patch release, including:

• Remove unnecessary task from UI-initiated runs.

Release 1.12.1

Patch release, including:

• Raise backend timeout to enable uploading larger artifacts.
• Fix to handle normal termination requests.

Release 1.12.0

Minor release, including new features and bug fixes:

Improvements

• Updated Reporting Dashboard, including new time ranges for better over view of your progress with Mayhem.
• Brand new Mayhem tutorials, better focused to help beginner/advanced/expert users.
• Tutorial added for fuzzing Python applications!
• Tutorial added for fuzzing Java applications!

Bug Fixes

• Fixed corpus download links on the Run page.
• Fix to allow Mayhemfile editing in the UI when running a non-Docker target.

Release 1.11.2

Patch release, including:

• Update tutorial image to include the latest mayhem CLI.

Release 1.11.1

Patch release, including:

• Removes Corpus Minimization progress meter, to prevent instances of it flashing on the Run page.

Release 1.11.0

Minor release, including new features and bug fixes:

Cool New Features

• Support for Corpus Minimization, to optimize performance of the test suites generated by Mayhem (in pre-alpha).
• Select which tasks you want during a Mayhem run through the Mayhemfile tasks option and the UI (in beta).
• Enhanced libFuzzer support to allow for testing Java and Python targets (in alpha).
• API updates to match the latest OpenAPI specification (OAS3) to allow for easier integration with Mayhem's API.
• Runs now show "next steps" to provide further guidance on what to do next with your target (especially if a run fails).

Improvements

• Test case processing improvements to better support bigger test suites.
• Reworked how lists of runs are displayed in the Mayhem UI, to make it easier to find active and pending runs.
• Code coverage statistics are now shown in the Project Overview and Runs listings, allowing for easier discovery of coverage for your targets.

Bug Fixes

• Downloading very large testsuites (tens of GB) will no longer result in disk issues as long as the CLI is updated to 1.11.

Release 1.10.1

Patch release, including:

• Improve honggfuzz crash handling for 2.1 targets.

Release 1.10.0

Minor release, including new features and improvements:

Cool New Features

• Brand new guided workflow for starting runs on Docker targets through the Mayhem UI.
• Edit your Mayhemfile directly through the Mayhem UI when selecting to run a target again and view the Mayhemfile from the Run page.
• Adds block, function, and line coverage metrics to the Run page to more easily show how much of your code Mayhem has explored during Behavior Testing.
• Adds support for running commands specified in a Docker image's entrypoint prior to fuzzing, making it easier to get your Docker targets tested with Mayhem.
• Adds support for finding the proper command path of your Docker images, allowing for easier Docker target testing.

Improvements

• Exploitability Factors, formerly called Security Indicators, updated with more helpful descriptions and prominent placement on the Run page.
• Continued improvements for test case processing, helping to ensure more results, faster!
• Expanded functionality of the Mayhem Docker registry browser, to show metadata for different images as well as allow deletion of images through the Mayhem UI.
• Better support for large test case files.

Release 1.9.4

Patch release, including:

• Resolve upgrade issue for 1.5.2 deployments.

Release 1.9.3

Patch release, including:

• Fixed bug where users could not set a new password after requesting to reset their password.

Release 1.9.2

Patch release, including:

• Changed autorefresh frequency in frontend to speed up the refresh rate.
• Fixed filtering for defects list on the Run page.
• Fixed filtered defect counts on the Run page.

Release 1.9.1

Patch release, including:

• Change mayhem-upgrade for on-premise deployments to wait until migration completes before continuing with the rest of the upgrade process.

Release 1.9.0

Minor release, including new features and bug fixes:

Cool New Features

• Docker Registry Browser (in beta) - now you can browse the list of images in the Mayhem Docker registry, and you can even start runs for them directly from the UI!
• Mayhemfile Editor (in beta) - configure a new run easily with the interactive Mayhemfile creator/editor.

Improvements

• Speed improvements for processing of test cases, now you should see much faster regression tests and new test cases discovered much more quickly!
• Documentation has been reorganized and expanded, to better assist in your Mayhem adventures.

Release 1.8.3

Patch release, including fix for:

• Enabling coverage tasks to run even when behavior testing fails / is stopped

Release 1.8.2

Patch release, including fixes for:

• Improved modeling of ioctl system call for networking targets.
• Allow testsuites that exceed 2GB in size.

Release 1.8.1

Patch release, including new features and bug fixes:

New Features

• Allowed Domains for enrollment. Now Mayhem Admins can configure which domains they want to allow their users to sign up/login with.
• Google Chat support for webhooks. Webhooks can now be sent to Google Chat rooms.

Bug Fixes

• Mayhem Tour fix to not repeatedly show for accounts created with Google/Okta.
• Run Again functionality fixed for some of the run listings.
• Contextual help link fixes.

Release 1.8.0

Minor release, including new features and bug fixes:

Cool New Features

• Mayhem Tour onboarding experience for helping to get users up and running with Mayhem. Includes two workflows: one for users that will be testing their applications with Mayhem, and one for users reviewing the results of Mayhem's testing.
• Fully revamped Run page to provide quicker access to the results of a run in Mayhem, including the list of defects found in the run and the list of tasks that comprised the run.
• Downloadable code coverage reports for a run (beta). Includes files for block coverage (drcov file), function coverage (json file), and line coverage (lcov file), to better help you understand which parts of your applications Mayhem has explored and analyzed.

Improvements

• Scheduling changes for supporting the coverage collection task.
• New and improved Mayhem Documentation look and feel, to better help you unleash Mayhem.
• Indicator and help for runs with slow tests per second. Is Mayhem not generating enough test cases or finding enough defects for you? Now we can help!

Release 1.7.2

Patch release, including fixes for:

• Fixes search in documentation and tutorials.
• Improved support for upgrading from version 1.5 to 1.7.

Release 1.7.1

Patch release, including fix for:

• Ensures the dropdown menu for runs in the UI does not get cut off.

Release 1.7.0

Minor release, including new features and bug fixes:

Cool New Features

• New reporting dashboard release in Alpha for Mayhem administrators to see data the last day/week/month. As an admin, you can access the dashboard from the top right menu.

Improvements

• Improved support for multi-node on-prem installations.
• Support for MIPS target is now available for fuzzing (MIPS32: 4K through Release 6 and MIPS64: R4000 through Release 6).
• Removal of post-processing as an explicit phase from the UI - all post-processing related information is now available from the behavior testing phase.
• New "Send Test Message" functionality for webhooks to allow users to test out their webhook integration outside the context of a run.
• Improved performance when listing regression testing reports.
• Introduced garbage collection for unused and deleted files so that space is reclaimed in the backend.
• Standalone password-reset available for Mayhem admins.
• Updated look/feel for the UI.

Bug Fixes

• Including organization projects in telemetry measurements.
• Scheduling fix to ensure analyze workers are always able to use worker nodes to be deployed.
• Instructions fix for installing the CLI on MacOS.
• Numerous fixes in the documentation and tutorials.
• Improved handling of email case sensitivity for password reset flows.

Release 1.6.3

Patch release, including fix for:

• Improved rust panic detection for better defect categorization.

Release 1.6.2

Patch release, including fix for:

• Ensuring that the 'Content-Type' field for custom webhooks includes the charset.

Release 1.6.1

Patch release, including fix for:

• Ensure metrics image is available for scaling on-prem deployments.

Release 1.6.0

Minor release, including new features and bug fixes:

Cool New Features

• Remote Access is now available. Were you stuck with an issue and were unsure how to let a ForAllSecure engineer assist you to resolve your issue? Remote control is now available for administrators that want to enable it on their instance. You can find it under the "Admin Settings" page.
• Admin audit logging is now available. All administrator actions that modify instance settings are now available for download for administrators. A complete CSV can be downloaded under the "Admin Settings" page.
• Webhooks are now available for organizations / individual projects and available to users under the organization / project settings page.

Improvements

• Improved partitioning support for on-prem deployments. The automated installer has been augmented to handle more disk setups and utilize disk more effectively.
• Improved upgrade support for on-prem deployments. Single-command upgrades for Mayhem installations are now possible.
• Users that are pending verification are now shown in the admin Users listing.
• The "Resource Manager" page was revamped and related information was moved under the "Run Queue" page for simpler navigation.
• A "show password" functionality was added to the UI to see the password you are typing on devices where mistyping is likely.
• API performance improvements.

Bug Fixes

• Improved defect classification for Golang targets.
• Better handling for "unknown" defect types identified by UBSAN targets.
• Improved stack trace identification on multi-threaded targets.
• Stale token handling for multi-node on-prem deployments.
• Fixes to contextual help links.
• Fixes to documentation search functionality.

Release 1.5.2

Patch release, including fixes for:

• Fixing rollups webhooks to report on all subscribed events.
• Permissions change to allow in-place upgrades for on-prem deployments.
• Enabling webhook subscription for user namespaces under admin settings.
• Enabling on-prem installations with non-routable nameservers or gateways.

Release 1.5.1

Patch release, including fix for:

• Unresolved image references to internal repositories removed to unblock on-premise installations.

Release 1.5.0

Minor release, including new features and bug fixes:

Cool New Features

• Webhook support for 5 event types. Do you want to know when a run starts/completes in Mayhem or when a crash/defect was found? If you have administrator access, you can enable them for organizations or individual projects at the top right under "Admin Settings".
• Search support for organizations/teams/users/projects/targets. Were you having trouble finding your project? Search bars were added throughout the web interface to ensure you find what you are looking for faster.

Improvements

• Mac CLI startup time optimizations. The 1.5.0 Mac CLI start up time was improved and may be up to 20x faster. If you are using Mac, try it out!
• The mayhem stop command was extended to take run patterns to allow quickly stopping multiple runs at once. If you are using multibranch pipelines this may come in handy.
• A help widget was added to all screens in the web interface to provide links to documentation when you are not sure what you are looking at. You can find it at the bottom right in the shape of a question mark "?".
• Event log download. Are you having trouble looking at all events that occurred during your run? A "download as .csv" button was added to the event log screen to allow you to fetch all events in bulk.
• Inactive users are now shown with a special marker throughout the UI.
• Reference integration solution examples for CICD integration with Travis is now provided alongside the documentation.
• Password strength checking configuration. Do you find that the default password requirements are too stringent? You can now turn the extra restrictions off from the Admin Settings page.
• Testcases are now named by their sha256 checksum, instead of sequentially increasing numbers. This allows users to uniquely identify test cases and also confirm they are looking at the same one on the console.
• Support for custom certificates and network settings for on-prem installations was added.

Bug Fixes

• The CLI can now handle docker being installed but not running so that it can appropriately emit an error about being unable to perform docker commands.
• Multiple updates and fixes in the documentation to reflect the newest changes.
• Changes in systemd configuration for bare-metal installations to ensure better handling of temporary service unavailability.
• Better handling for bare-metal partitioning scheme to utilize more of the available disk without mixing disks of different types (HDD vs SSD).
• Partial workarounds for a known kernel bug that slows down fuzzing on targets with too frequent restarts. Users will see affected workers restarting in the event log if this issue is detected. This is automatic and no user action is necessary.
• Improved stacktrace collection for UBSAN targets.
• Improved handling of the $MAYHEM_DOCKER_REGISTRY variable internally to handle cases where Mayhem runs are started through a proxy.
• The expiration period for authentication tokens for the docker registry is now customizable. If you are experiencing failures while pushing very large docker images, let your administrator know so that they can increase the default timeout of 10 minutes.

Known Issues

• Test case identification has changed from numbers to checksums. All API endpoints are still supported, but may be deprecated in the future. If you are using test case numbers to identify test cases we recommend switching to sha256 checksums.
• Current workarounds for the kernel bug mentioned in MH-6942 and MH-7002 may be insufficient in completely mitigating the issue. When the issue manifests, the observed behavior for users is that one or more runs may not start or appear "stuck". We recommend trying to stop the run when that happens and try to restart it. If that doesn't resolve it to contact your administrator to reboot the affected node.
• A performance issue was identified that makes runs fail when too many are started all at once. A default limit of 5 for runs that can be preparing concurrently was introduced until a better resolution is identified.

Release 1.4.0

Minor release, including new features and bug fixes:

Cool New Features

• New installer ISO available for one-click on-prem installations.
• Backup/restore functionality added for on-prem installations to allow users to checkpoint their data.
• Email verification enabled for accounts on deployments with SMTP-setup availability. If you do want access to the email service, please contact your deployment's administrator to enable it for you!
• Email password resets are now available for deployments that enable email verification and sending.

Improvements

• Standalone (non-LibFuzzer) ASAN instrumented targets supported.
• Include inline function information in stack traces when available.
• Reference integration solution examples for CICD integration with Jenkins provided alongside the documentation.
• Several UI improvements for a more compact project/target/run view. Viewing API tokens now requires an explicit "click to show" action.

Bug Fixes

• Fix handling of targets which read input via the readv() syscall.
• Improved error reporting when an invalid dictionary is supplied.
• Fix reproducer command for targets accepting input via stdin.
• Fixed run failure on uninstrumented targets using certain SSE instructions.
• Fixed permissions issue accessing /dev/stdin, /dev/stdout, and /dev/stderr from target.

Known Issues

• On-prem installations that run on Virtualized hardware will not work with NAT networking. To work around the issue, the user can switch to using bridged networking for all virtual machines used as nodes.
• When docker is available on a client machine but docker is not running, "mayhem login" is expected to fail while trying to login the user to the internal docker registry. To work around the issue, the user can start docker on their local machine.
• By default the internal docker registry has a 10-minute expiration period for authentication tokens, which may prevent pushing large containers over slower connections ("docker push" generates an "authentication required" error). There is no known work around at the moment besides attempting to shrink the pushed image or using a faster connection to the internal registry.
• On recent macOS installations (10.15+) the mayhem CLI may run very slowly (even minutes per command) due to a fresh macOS update (see: https://sigpipe.macromates.com/2020/macos-catalina-slow-by-design/). The only known workaround at the moment is to navigate to "Settings -> Privacy -> Developer Tools" and enable the "Terminal" item (this step may require installing a recent version of xcode).

Release 1.3.3

Patch release, changes to allow external integration with custom certificates:

Changes

• [MH-6715]: Enable the API to access internal docker registry images when custom user certificates are provided.
• [MH-6716]: Enable the backend to access API data when custom user certificates are used.

Release 1.3.2

Patch release, okta default authorization server changes:

Changes

• okta integration now uses default authorization server by default (rather than API Access Management) as described here: https://support.okta.com/help/s/article/400-error-on-authentication-to-Okta-using-OIDC

Release 1.3.1

Patch release, resolving GKE upgrades from 1.2.x:

Bug Fixes

• Includes a fix from stable/prometheus helm charts to support changing server from "Rollingupdate" strategy to "Recreate".
• Configuration change to allow more than 200 instances of target processes to run in a single node.

Release 1.3

Minor release, including new features and bug fixes:

Cool New Features

• For documentation on new features, navigate to the top right of the UI to open the documentation section.
• Mayhem can now have multiple users designated in the admin role. See all instance users under the top right of the UI under Users (Admin only).
• Organizations and Teams can now be created to better manage project permissions for groups of users.
• New authentication types have been added to allow for single sign-on (Google and Okta). Talk to your administrator to enable them for your instance. Find toggles for controlling them in the admin settings screen of Mayhem.
• Password strength requirements were strengthened to require a minimum password length of 11 and to not allow for weak passwords.

Improvements

• Mayhem security and privacy improvements in the entire stack.
• The Mayhemfile reference in the documentation can now be copied and pasted as a Mayhemfile template.
• Updated the navigation for organizations, members, and teams to show the hierarchy.
• Added namespace example to the Mayhemfile reference documentation.
• Updated the password strength requirements.
• Avatars added to projects and organizations.
• A "How To Share Projects" guide has been added to the product documentation.
• The Mayhem CLI now packaged as a signed and notarized installer for macOS.
• Admin users can now deactivate other users from the admin Users view under the top right of the UI.

Bug Fixes

• When starting a run, the docker_pullable field is inferred from the Mayhemfile if it is not specified in the API request.
• When a run is stopped that has a set duration, the elapsed time is now shown correctly.
• Defect titles fixed to show consistently in the UI.
• Fixed Honggfuzz edge coverage to reflect the actual coverage number.
• The project settings page in the UI was updated to show user first and last names.

Known Issues

• Occasionally, regression testing may not pick up all the test cases from the previous run. If this happens, stop the ongoing run and restart it. The missing test cases should get picked up by the new run.
• As a result of security hardening, the /api/v1/testsuite/{testsuite_id} endpoint is now only accessible to Mayhem Admins. Other users of the API should transition to using /api/v1/namespace/{namespace}/project/{project_slug}/target/{target_slug}/testsuite/{testsuite_id}, which properly verifies project and target access permissions. This endpoint may be deprecated in a future release and we recommend against using it.
• Versions of the Mayhem CLI, mayhem, earlier than 1.3 can no longer download testsuites from Mayhem clusters on version 1.3 or later, due to the above change. Please update your Mayhem CLI by choosing "Download Mayhem CLI" from the user menu at the top right of any page and following the instructions.
• Projects are exclusively private. Changing the project visibility setting to "authenticated" will not affect the project visibility.
• Frontend URLs in 1.3 were redesigned and direct links to specific pages may no longer work. Users may have to re-navigate from the landing page to the desired page.

Release 1.2.1

Patch release, including new features and several bug fixes.

Cool New Features

• Mac OS users are now able to install the Mayhem CLI (Available for Linux and macOS). This new link is found on the Mayhem UI page under the Tips/Getting Started section, as well as in the dropdown menu in the top right of the screen.
• A Support Matrix was created and added to the documentation to determine what targets are supported. The matrix can be found in the documentation repository under the Support Matrix section.

Improvements

• Updated the sizing guide to include the latest engineering metrics.
• Improved component coverage.

Bug Fixes

• Fixes issue wherein the base image field was required when dragging and dropping a Mayhemfile through the UI. This field is no longer required, but optional.
• Tutorial images have been updated to match release.
• Improved performance for downloading docker base images.
• The edge count for runs that only contain libfuzzer targets has been corrected.
• Fixed scheduler issue that could make certain runs get stuck in the post-processing phase.

Known Issues

• If post_processing is already complete, and another request to end the same phase is sent, thea new output testsuite overwriting the old one would be generated.
• On some targets, the Backtrace field for a crashing test case may appear truncated as if frames are missing. As a workaround, increase the target's network timeout in the Mayhemfile: cmds[].network.timeout and restart the target's run.
• Depending on system configuration and OS version, the Mac version of the mayhem client may cause a security warning to appear on the first run that reads 'mayhem cannot be opened because the developer cannot be verified.' When this dialog appears, choose 'Cancel.' Then, choose Apple Menu > System Preferences from the top left of your screen and navigate to the "Security & Privacy" preference pane. (Its icon looks like a house with a combination lock.) In this pane, choose the "General" tab and look for a paragraph stating 'mayhem was blocked from use because it is not from an identified developer.' Click the corresponding 'Open Anyway' button. Now, run the mayhem client again. A similar warning dialog will appear but this time an 'Open' option will be available; click it. From now on, mayhem is saved as an exception in the security settings on this computer and can be used normally. Reference: https://support.apple.com/en-us/HT202491
• When using the Mayhem Login command is used to log in, a certificate is presented with a Common Name or Subject Alternative Name different from the URL used to access, the login operation will fail with an SSL Error indicating 'certificate verify failed.' Fix: access using the proper URL or contact your administrator to install a valid certificate.

Release 1.2.0

Minor release, including new features and several bug fixes.

Features

• The way in which paginated items are displayed in the UI was redesigned. A standard pagination system has replaced the infinite scroll to allow users to jump page numbers to improve navigation and share results more easily.

Bug Fixes

• The bare-metal installer was upgraded to support CentOS 7.7.
• Project/Target/Run deletion is now asynchronous to handle targets with tens of thousands of test cases more efficiently.
• Defect report listing in the API was optimized to return faster on targets with thousands of defects.
• API timeout was introduced to prevent starvation when very slow routes were hit. As a result of this safeguard, an error will be returned if the API is unable to return on time.
• Fixes to allow fuzzing of arm32v7 and arm64v8 targets.
• Fixed documentation to include max_length field in the Mayhemfile (available for libfuzzer targets).
• Fixed API documentation to properly show the X-Mayhem-Token header as required.
• Fixed identification of certain stack-overflow errors detected by libfuzzer targets.
• Fixed issue where the license usage page would stop updating when the cluster was offline for extended periods of time.
• Fixed UI handling of missing pages.
• Fixes in the UI to improve the user experience for runs that are in progress (less flashing, better error handling).
• Fixed mismatch between the run queue state and the number of active workers for each run.
• Fixed docker login functionality to work properly with mayhem login on docker versions before 1.13.
• Fixed presentation of defects that originate from advanced triage vs regular triage to allow the user to disambiguate them.
• Fixed crash deduplication issue affecting Rust targets.
• Fixes to handle golang server fuzzing better.
• Fixed CLI issue that was causing mayhem commands to fail non-deterministically when the file upload timeout was hit.

Known Issues

• The system will give a "Searching for Available workers" warning when there are available workers in the resource manager page. The workaround is to stop and restart the run. If it continues happening, contact FAS Support.
• There is a discrepancy between "test suite size" and "test case report number" in the behavior testing view while running against a target, but the number matches when the run is complete.
• An experimental feature was removed from the client and the Mayhem API. This feature used the maximize_resources field in the POST data to create a new run. To preserve compatibility with older clients and programs targeting the existing API, POSTs to this endpoint will silently ignore the maximize_resources field, if present.
• On some targets, the Backtrace field for a crashing testcase may appear truncated as if frames are missing. As a workaround, increase the target's network timeout in the Mayhemfile: cmds[].network.timeout and restart the target's run.

Release 1.1.0

Minor release, including new features and several bug fixes:

Features

• Improved mayhem login experience. The Mayhem CLI will now: 1) prompt users for accepting self-signed certificates when logging to a remote Mayhem server, and 2) automatically log in users to the internal docker registry.
• Resource manager page added to the UI so that users can track the current work queue of their Mayhem installation as well as how many resources are available vs in use. The resource manager is available for users from the top-level bar next to the "Projects" link.
• Usage page added for admin users so that they can track their installation usage across time. The usage page is accessible under the top-right dropdown menu under "Usage".
• Target-feature auto-detection for fuzzers, and language-specific settings. Users no longer are required to specify the type of their binary in their Mayhemfiles (e.g., libfuzzer, asan, etc) and will be auto-detected. The users can still override the autodetection by providing their own values.
• Enable portability of Mayhemfiles across installations. The new Mayhemfiles support a $MAYHEM_DOCKER_REGISTRY variable in the baseimage field that references Mayhem's internal docker registry. The default baseimage for Mayhemfiles has been changed to $MAYHEM_DOCKER_REGISTRY/forallsecure/debian-buster.
• New coverage measurement backend for faster test case processing.
• New fuzzing backends with alternate fuzzing strategies and increased performance on file-based inputs.
• The projects, targets and runs views now all have delete options.
• The tutorial docker image is now packaged along with the Mayhem installation so that users with limited network connectivity can more easily run through the documentation.

Bug Fixes

• Better support for Golang/Rust targets. The fuzzing backend has been extended to better autodetect Golang/Rust targets and also identify and bucketize crashes more effectively.
• API stability/performance Issues. Over 20 optimizations and bug fixes were completed in the API to handle usage under heavy load that should lead to a better user experience overall.

Known Issues

• Targets with a very large number of test cases (over 100K) may have runs fail to start because the download can time out.
• Runs may fail with the following critical error "Unknown fuzzing error: At-risk data found ...". If this occurs the run can safely be stopped and restarted.
• Stripped golang binaries are not correctly detected as golang.
• It may take mayhem-fuzz several minutes to start up for targets that have a large number of testcases (1K or higher) or targets that run slowly.
• Using a docker installed through snap on Ubuntu with a protection mechanism called apparmor might cause a problem adding untrusted server certificates. The user will be prompted to docker log in manually.

Release 1.0.2

Patch release, including new features and bug fixes.

Features

• Crash detection for binaries written in memory safe languages like Rust.

Bug Fix

• Analysis improvements for handling networking targets like Rust binaries using epoll/select/ppoll.
• Key-Value storage for Mayhem extensions was fixed to allow updates by external tools.

Release 1.0.1

Patch release, including new features and Bug fixes:

New Features

• New tutorials page, with examples demonstrating the docker workflow as well as the "Advanced Triage" feature. The tutorials are now accessible from the dropdown at the top-right of the UI.

Bug Fixes

• Multiple scheduler fixes to handle re-scheduling phases with temporary failures and marking all failed phases appropriately.
• Advanced triage now handles ubuntu:19.04 base images.
• Advanced triage now handles images with libc6-dbg preinstalled.
• Advanced triage smoke testing now handles non-absolute paths.
• Advanced triage smoke testing now handles chdir in Mayhemfiles.
• API performance improvement to handle very high numbers of defects.
• API now handles defects with arbitrarily long backtraces.
• Analysis no longer fails when the corpus directory contains folders.
• Analysis can now handle new SSE instructions that appear in Rust targets.
• UI is now less aggressive with queries made to the API to improve responsiveness with multiple tabs.
• mayhem validate now handles libfuzzer targets with the dictionary parameter.
• Improvements in error and info messages emitted by the CLI during validate, run and show.
• Disk limits are now explicitly listed for SEs to configure upon installation.

Release 1.0.0

Major release, including new features and bug fixes:

Features

• Advanced Triage. Introduces a new analysis mode called "Advanced Triage" that runs valgrind against the target to find additional defects. Advanced Triage defects are tied to their CWE number. Several new screens are added to the UI for Advanced triage as well. Advanced triage is disabled by default. To enable, set your Mayhemfile version to '1.0' and add a top level field: "advanced_triage: true" Limitations: Advanced Triage is only supported for x86 and x64 targets that use glibc. Other architectures and C Libraries are not supported. Additionally, the "libc6-dbg" package must be installed in the target's base image for Advanced Triage to work. Golang programs will also not work with Advanced Triage.
• Faster testcase upload and download. Upload and download performance for targets with large numbers of testcases has been significantly improved.

Bug Fixes

• "mayhem validate" now works correctly for targets with a baseimage whose entrypoint is something other than "bash"

Known issues

• CLI upgrade required. Users must upgrade to version 1.0.0 of the mayhem CLI. Versions before 1.0.0 will no longer work for starting runs.
• Users must wait for a run to complete (or be stopped) in order for the "mayhem download" command to download testcase for the run.
• Queuing up multiple runs at the same time will result in subsequent runs not picking up the testcases generated by the previous run.
• When the output testsuite of a run is not available, the test cases downloaded by mayhem download will include the original file names given by the user.

Release 0.8.1

Patch release, including bug fixes:

• Removes unused copy of internal registry that was triggering a non-deterministic failure mode when upgrading deployments.
• Triager update for fixing the issue that was affecting test case processing for custom base images with no libffi library being present.
• Honggfuzz test case stats reporting has been fixed so that the "Tests Run" is properly updated.

Release 0.8.0

Minor release, including new features and bug fixes:

Features

• Brand new project overview page. The new page provides users with the ability to view and navigate between targets, runs and defects within a project, as well as view aggregates from the latest runs.
• Support for targets that receive input from standard input (/dev/stdin). Users can now submit targets reading from stdin simply by omitting the "@@" symbol from the command line used for targets reading from files.
• Support for honggfuzz-based targets. Users can now submit honggfuzz-compiled targets using the standard flow and specifying honggfuzz: true in their Mayhemfile (and asan: true - assuming the sanitizer is enabled).

Bug Fixes

• Added "Preparing" state in the phase status to show to the user when a run phase was picked up and started preparing (downloading base image and test cases, smoke testing target etc) as opposed to providing no feedback.
• Resolved task unavailability issue where RabbitMQ was consuming 100% of one CPU core.
• Resolved issue where tasks were getting stuck in behavior analysis when the time remaining for a run was 0 seconds.
• Resolved issue where behavior analysis tasks were killed and restarted by the scheduler when celery was temporarily unresponsive.
• Improved smoke testing support for golang targets that perform special signal handling while launching.

Release 0.7.0

Minor release, including new features and bug fixes:

Features

• Client-side package validation was added to enable users to test their targest locally before starting runs. The mayhem validate path/to/package command is now available for all users. To run the complete set of validation checks, access to docker is required. When docker is not available validation is still possible, but limited to static checks which are now run automatically on every mayhem run invocation.

• Server-side package validation was enabled along with actionable messages to detect target errors when starting fuzzing runs. Critical errors are now exposed in the UI in the phase where they happened.

• Telemetry data collection can be activated (off by default) on a per-deployment basis to collect usage statistics. Enabling it requires the installation to be able to make outbound connections to our data collection server.

Bug Fixes

• Improved time handling in the UI to eliminate issue with clock skews between the browser and the cluster server that was showing negative time durations on the frontend.
• Improved phase reporting in the UI when run phases are scheduled to better communicate to the user what is happening in the backend.
• Resolved race condition in the analysis backend that was preventing running multithreaded servers.
• Resolved race condition in the analysis backend that was preventing test cases from reaching the post-processing phase.
• Resolved multiple scheduler bugs that were preventing certain phases from getting started or running to completion.

Release 0.6.2

Patch release, includes:

• Updates to CLI/API to allow for parameterizing timeouts for API requests using the --timeout option. Especially useful for slow networks.
• Updates to default scheduling prefetching policy that would block certain runs from starting when other ones were in progress.
• Changes default duration for autogenerated self-signed certificates to 2 years (the default openssl one is 30 days).
• Re-enables htpasswd authentication for deployments where we desire to guard authentication by an htdigest prompt.
• Documentation changes to reflect the process used during target packaging.

Release 0.6.1

Patch release, includes:

• Updates to installer for handling older versions of bash during installation.
• Updates to Mayhemfile documentation for fields newly introduced by 0.6.0.

Release 0.6.0

Changes

• Added support for packaging targets as docker images:

  • Users can specify what uid/gid their target runs.
  • Files can be placed in arbitrary directories with absolute paths (rather than everything under /root/).
  • The root file system can be writeable (so /var/log or /etc/foo/bar.conf is writeable) whereas it used to be read-only.
  • Users can specify docker images as base layers for the container.

• Mayhem's symbolic execution was updated to a new version:

  • Migrated to the latest Binary Analysis Platform (1.6).
  • Improved stability by removing dependencies on postgres.
  • Improved stability by reducing disk footprint of the test case generator.

• Improved UI/UX:

  • New Home page - shows your projects (along with the latest run from each), active runs, and tips for getting started.
  • Projects page - shows list of projects along with the latest run from each.
  • Active Runs dropdown - always-accessible list of runs you have currently going with Mayhem.
  • Runs page - shows full list of runs performed by Mayhem.
  • An Event log is available for each phase/run which provides users with feedback when errors occur.
  • Improved user management.
  • Performance improvements for starting up runs faster with the new scheduling architecture.
  • Removed max filesize limit that was blocking uploading files over 100MB.

• New service: private docker registry

  • The 0.6 release bundles a docker registry with the standard deployment and enables users that do not have easy access to a registry to be able to store their images. Mayhem users should be able to authenticate to the registry, push/pull images, and start runs based on those images.

• Known limitations:

  • Max machine size of 16-20 cores. Due to resource contention in the Linux kernel, fuzzing degredation occurs when fuzzing across too many cores. A workaround for best performance is to ensure Mayhem is deployed on machines with fewer than 20 cores, or larger machines are partitioned into Virtual Machines of this size.
  • Images stored in dockerhub that have a timestamp before June 2016 are using a manifest our system does not yet support and pulling those images directly will fail. The user can work around this issue by pulling and re-pushing the image to the registry and using the updated reference.

Release 0.5.0

UI Bug Fixes

• Test cases show time as 00:00 on the post-processing tab
• Test case graph time axis stops at the dynamic analysis phase
• Projects are not sorted alphabetically and limited to 10 on the namespace page
• Losing connection to server results in repeated red error messages
• Crashing testcase doesn't have an assembly
• When analysis stops, the UI shows the phase still going

Analysis Bug Fixes

• No underscores allowed in Mayhemfile
• Mayhem run crashes when a symlink points to a nonexistent file

Release 0.4.0

Changes

• Project-based Workflow for Testing with Mayhem: The Mayhem fuzzing workflow will be target-centric and enable users to easily correlate between runs of the same target.
• Configuring projects will be done using our new and improved configuration file that is automatically generated as part of a target packaging (and can be easily extended to support specific target requirements).
• Test suites can be downloaded for a each build of a project.
• Support for continued fuzzing and integration with CI/CD workflows.
• Crash de-duplication support (identify unique defects from a list of crashes).