Invalid Response Spec¶
Overview¶
The response from the API was not described in the provided spec.
This is commonly attributed to drift between the API implementation and a manually maintained specification. This may also be a sign of unintentional information exposure, leading to a potential security issue.
Consider an API that is returning fields that are not part of the specification. Those fields might contain sensitive information that the API was not supposed to return, such as a some Personally Identifiable Information. In this situation, you will need to update the API so that this information is no longer returned in order to keep your information safe.
If your specs are purely informational, this type of issue is on par with out-of-date documentation. For projects which depend on API specs for client code generation, automated testing and other tooling, this type of issue can lead to downstream problems, though usually far from catastrophic.
For what it's worth, Mayhem for API itself doesn't require a perfect, comprehensive spec to test a given API. We're able to do spec validation as a side-effect of our primary testing, so we produce these warnings in case you find them useful. You don't need to fix them just to keep using the tool!
Recommendation¶
Make sure that each specified endpoint's "responses" field is in sync with what the code actually produces: * HTTP status codes that the endpoint returns are specified * Content-Type that the endpoint returns are specified
Examples¶
Perhaps the simplest case is when the spec for a path doesn't include one of the possible http status codes that path may return. For example, given this trivial yaml OpenAPI spec:
openapi: "3.0.0"
info:
title: "status"
version: "0"
paths:
/status:
get:
responses:
'204':
description: everything is fine!
If Mayhem for API ever sees any non-204 response from this endpoint, it will observe that the spec is invalid.
Note that OpenAPI supports a "default" response that may come in handy!