Got: The GET method cannot be used with a body

[timdp]

Describe the bug

• Node.js version: 12
• OS & version: Ubuntu

TypeError {
  message: 'The `GET` method cannot be used with a body',
}
Object.exports.normalizeRequestArguments (node_modules/got/dist/source/normalize-arguments.js:262:19)
get (node_modules/got/dist/source/request-as-event-emitter.js:39:55)
node_modules/got/dist/source/request-as-event-emitter.js:244:19

I ran into this when upgrading to [email protected].

I realize this is up for debate, but to my best knowledge, a lot of HTTP servers _do_ support GET requests with a body. Elasticsearch, for example, heavily uses this. (They also provide an escape hatch with POST though.)

I don't think got should be the limiting factor, given that the underlying modules allow it.

Checklist

• [x] I have read the documentation.
• [x] I have tried my code with the latest version of Node.js and Got.

[szmarczak]

Got follows the RFC

[szmarczak]

Also this is a duplicate issue

[sindresorhus]

@szmarczak Can you link to the RFC and the duplicate issue? We should probably document this more clearly in the readme.

[timdp]

I searched for the exact error message but I guess it changed in v10.

The only reason why I encountered the issue is because body: null is no longer supported. Hence, it's not that I need to supply a body with a GET request per se. I just thought I'd flag it for others.

I don't think a client library needs to be this strict necessarily, especially if servers support it. There could also be a warning or an opt-in-flag.

But for the sake of completeness, the HTTP/2 RFC does state that GETs don't have a body: https://tools.ietf.org/html/rfc7540#section-8.1.3

[szmarczak]

@sindresorhus I can't find the issue, but I remember there was one 100%.

[szmarczak]

I guess we should look at the closed issues and pick the most common ones. Then add these to the FAQ.

[sindresorhus]

I don't think got should be the limiting factor, given that the underlying modules allow it.

It's a balance. Node.js core is generally fast and loose and doesn't protect you against mistakes. We want to make something human-friendly. GET with a body is almost always a mistake. If enough people need this, we could consider an option to allow it, but it definitely doesn't make sense to allow it as default behavior.

[timdp]

I'm not sure I agree that it's almost certainly a mistake, but I definitely support your reasoning. Thanks for taking the time to discuss this.

[jakesjews]

My team has to integrate with some third party APIs which unfortunately require a body with a GET. Is it possible to work around this or are we stuck on version 9?

[szmarczak]

@jakesjews I don't think this will be an option. As @timdp has already mentioned, HTTP2 doesn't support GET with a payload anyway.

[timdp]

Nuance: the RFC doesn't support it. The RFC also wants you to keep the header size under 2,000 bytes but I've seen URLs way longer than that. Should got also disallow those?

[kelp404]

I just call this function then get TypeError: The 'GET' method cannot be used with a body.
ipStack() is work on v9.6.0, but it throw the error on v10.2.1.

exports.ipStack = ip => {
  const query = {access_key: config.ipStackToken};
  return got.get(
    `http://api.ipstack.com/${ip}?${queryString.stringify(query)}`,
    {
      json: true,
      timeout: config.vendorApiTimeout
    }
  ).then(response => response.body);
};

[0] Mongoose:      72ms IPs.findOne({ ip: '128.199.133.240' }, {})
[0] TypeError: The `GET` method cannot be used with a body
[0]     at Object.exports.normalizeRequestArguments (/Volumes/Data/kelp404/meetpet/node_modules/got/dist/source/normalize-arguments.js:285:19)
[0]     at get (/Volumes/Data/kelp404/meetpet/node_modules/got/dist/source/request-as-event-emitter.js:42:55)
[0]     at /Volumes/Data/kelp404/meetpet/node_modules/got/dist/source/request-as-event-emitter.js:257:19
[0] (node:36085) UnhandledPromiseRejectionWarning: TypeError: The `GET` method cannot be used with a body
[0]     at Object.exports.normalizeRequestArguments (/Volumes/Data/kelp404/meetpet/node_modules/got/dist/source/normalize-arguments.js:285:19)
[0]     at get (/Volumes/Data/kelp404/meetpet/node_modules/got/dist/source/request-as-event-emitter.js:42:55)
[0]     at /Volumes/Data/kelp404/meetpet/node_modules/got/dist/source/request-as-event-emitter.js:257:19
[0] (node:36085) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 8)

[nordluf]

Breaking backward compatibility without any option to return known behavior is not the best way, in my mind. Simple option, but I would not need to migrate the whole codebase.

[szmarczak]

Please read the docs before complaining.

[nordluf]

I did, unfortunately, I cannot find any ability to get around that behavior. Based on https://github.com/sindresorhus/got/blame/master/readme.md#L188 and https://github.com/sindresorhus/got/blob/master/source/normalize-arguments.ts#L348 it looks impossible. What do I miss?

[szmarczak]

@nordluf Unfortunately there's nothing you can do but to switch to POST instead. We understand your frustration, but this is done to make the behavior more reliable according to the RFC.

@kelp404's example has got some inconsistencies: the json option works differently than in v9.6.0, it no longer converts response to JSON. got(...).json() should be used instead (in this case).

[nolde]

Elastic Search requires the use of body with GET.
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html

Just an example where people get creative with their usages of HTTP. I think that a blanket restriction like that is quite weird, when it has been supported for so long by HTTP/1.1.

[timdp]

Elasticsearch will actually happily accept POST as well because Elastic are aware of this discrepancy.

[nolde]

@timdp , Elasticsearch statement about it is the following (emphasis mine):

The HTTP libraries of certain languages (notably JavaScript) don’t allow GET requests to have a request body. In fact, some users are suprised that GET requests are ever allowed to have a body.

The truth is that RFC 7231—the RFC that deals with HTTP semantics and content—​does not define what should happen to a GET request with a body! As a result, some HTTP servers allow it, and some—​especially caching proxies—​don’t.

The authors of Elasticsearch prefer using GET for a search request because they feel that it describes the action—​retrieving information—​better than the POST verb. However, because GET with a request body is not universally supported, the search API also accepts POST requests

That does not really matter, though, as Elasticserach is just an example. But looking into the docs, RFC 7237 says that "_[it] is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG)._"

It points to RFC7231, "_Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content_", which contains this excerpt about GET method (emphasis mine):

The GET method requests transfer of a current selected representation
for the target resource. GET is the primary mechanism of information
retrieval and the focus of almost all performance optimizations.
Hence, when people speak of retrieving some identifiable information
via HTTP, they are generally referring to making a GET request.

It is tempting to think of resource identifiers as remote file system
pathnames and of representations as being a copy of the contents of
such files. In fact, that is how many resources are implemented (see
Section 9.1 for related security considerations). However, there are
no such limitations in practice. The HTTP interface for a resource
is just as likely to be implemented as a tree of content objects, a
programmatic view on various database records, or a gateway to other
information systems. Even when the URI mapping mechanism is tied to
a file system, an origin server might be configured to execute the
files with the request as input and send the output as the
representation rather than transfer the files directly. Regardless,
only the origin server needs to know how each of its resource
identifiers corresponds to an implementation and how each
implementation manages to select and send a current representation of
the target resource in a response to GET.

A client can alter the semantics of GET to be a "range request",
requesting transfer of only some part(s) of the selected
representation, by sending a Range header field in the request
([RFC7233]).

A payload within a GET request message has no defined semantics;
sending a payload body on a GET request might cause some existing
implementations to reject the request.

The response to a GET request is cacheable; a cache MAY use it to
satisfy subsequent GET and HEAD requests unless otherwise indicated
by the Cache-Control header field (Section 5.2 of [RFC7234]).

There is nothing — in the current documentation — that determines that GET bodies should be prohibited, specially when coming from a client library making the request. Therefore, the current behaviour has been determined by got, not by any real established standards. I understand the recommendation not to use it (and I agree with it), but blanket prohibition makes it unusable when interacting with systems that chose unorthodox GET semantics.

I would support a change in this behaviour, if possible.

[timdp]

There is nothing — in the current documentation — that determines that GET bodies should be prohibited

Not true, I’m afraid. As mentioned above, the more recent HTTP 2 RFC states:

An HTTP GET request includes request header fields and no payload body

[nolde]

So we should assume got is only compliant, or mainly targets, HTTP/2 ?

On the documentation, that is not specified and, on the comparison table, it actually mentions _HTTP/2 support_, which implies that it is not the main target, but an extra feature.

[szmarczak]

which implies that it is not the main target, but an extra feature.

That's correct. HTTP/2 support is still in the works. We don't allow sending body for GETs because that's undefined behavior. Some servers may reject the request, some may not.

[nordluf]

Doesn't that make sense to allow a user to decide if his server capable to serve the body in GET requests if HTTP/1.1 is in use?

[szmarczak]

That's an anti pattern

[nordluf]

Which one is anti-pattern: use body in GET or allow to setup library behavior?

What is you suggestion to do instead? I'm against making POST request to retrieve information. As of RFC (https://tools.ietf.org/html/rfc7231#section-4.3.1)
The GET method requests transfer of a current selected representation for the target resource. GET is the primary mechanism of information retrieval and the focus of almost all performance optimizations. Hence, when people speak of retrieving some identifiable information via HTTP, they are generally referring to making a GET request.

POST semantics is - create or publish something.

[szmarczak]

use body in GET

that one

[szmarczak]

I'm going to lock this issue, the decision has been already made long ago.

[sindresorhus]

There is now an option to allow GET with a body. However, we still strongly recommend against using it unless you really have no other choice. https://github.com/sindresorhus/got/releases/tag/v10.6.0