Got: Add `maxRedirects` option

Created on 22 Jul 2019 · 9Comments · Source: sindresorhus/got

What problem are you trying to solve?

I am trying to follow more than 10 redirects.

Describe the feature

The configurable property, that will override the hardcoded 10.

enhancement ✭ help wanted ✭

Source

petrpatek

All 9 comments

How many redirects are you trying to follow and in what crazy situation would it be more than 10?

sindresorhus on 22 Jul 2019

👍1

I am using it for a web-scraping and in some cases like for example on TripAdvisor, they have more redirects than ten. I think it might be connected that the request is redirected through a lot of bot check servers. In comparison, Chrome browser uses 20 redirects. I guess it is an edge use case that most of got users don't need, but on the other hand, it will be more configurable.

petrpatek on 23 Jul 2019

Will 20 be enough for you?

sindresorhus on 23 Jul 2019

👍1

The point of the limitation is to prevent infinite loops, not valid use-cases. I would also prefer not to make it configurable as everything that is configurable adds extra overhead for the maintainers and also for users having to read the docs.

sindresorhus on 23 Jul 2019

👍1

20 would be great. I will create a PR for it. Thank you very much 👍

petrpatek on 23 Jul 2019

I would much rather have a default low, e.g. 2 redirects and maxRedirects configuration.

This is simply because 99% of websites will never make more than 1-2 redirects at a time.

We are using Got to aggregate data from thousands of websites. If one of them started to redirect in loop (e.g. broken session) and we had 50 jobs running, then 20x50 could easily result in some service disruption.

That said, we already don't use followRedirect at all for exactly this reason (even 10 is high).