Netlify-cms: Timeout uploading large files

[FTWinston]

Describe the bug
When uploading a large media file to a new post, the upload repeatedly times out after 60s.
As it's a big file (~50MB), I'd expect the upload to take longer than 60s, but the CMSs repeatedly aborts and retries after 60s.
It looks like this is intentional, perhaps caused by #3649?

Console output below:

Pausing requests for 4 seconds due to fetch failures: Request timed out after 60 seconds
cms.js:118591 Done pausing requests
cms.js:118589 Pausing requests for 9 seconds due to fetch failures: Request timed out after 60 seconds
cms.js:118591 Done pausing requests
cms.js:118589 Pausing requests for 16 seconds due to fetch failures: Request timed out after 60 seconds
react_devtools_backend.js:2450 leave called too many times.
cms.js:118591 Done pausing requests
cms.js:118589 Pausing requests for 25 seconds due to fetch failures: Request timed out after 60 seconds
react_devtools_backend.js:2450 leave called too many times.
cms.js:118591 Done pausing requests
react_devtools_backend.js:2450 API_ERROR: Request timed out after 60 seconds
    at new t (cms.js:53893:25)
    at i._createSuperInternal (cms.js:796:283)
    at new i (cms.js:53886:27)
    at E.handleRequestError (cms.js:54745:17)
    at E._callee69$ (cms.js:54804:61)
    at tryCatch (cms.js:223192:40)
    at Generator.invoke [as _invoke] (cms.js:223421:22)
    at Generator.prototype.<computed> [as throw] (cms.js:223244:21)
    at asyncGeneratorStep (cms.js:223885:24)
    at _throw (cms.js:223911:9)

cms.js:223888 Uncaught (in promise) {type: "ENTRY_PERSIST_FAILURE", error: "Failed to persist entry", payload: {…}}

To Reproduce

1. Go to the CMS, add a new content entry, of a type with a "file" field.
2. Click "choose a file", then "upload" and pick a large file.
3. Save the entry, wait a few minutes.
4. See error.

Expected behavior
Saving should eventually succeed.

Applicable Versions:

• netlify-cms-app: 2.13.1
• netlify-cms-core: 2.33.1
• netlify-cms: 2.10.66
• Git provider: GitHub
• OS: Windows 10
• Browser version chrome 86

CMS configuration
config.yml

Additional context
If this timeout were configurable, or could be disabled when uploading files, that would seem to me to prevent this problem.

Alternatively, if this is simply a much larger file than I ought to be storing in the CMS, then by all means tell me that.

Thanks in advance!

[erezrokah]

Thanks @FTWinston, you are correct it is a good idea to have an increased default timeout for POST requests and also make it configurable.

[FTWinston]

Either of those sound good to me. Both sound great!

On Sun, 8 Nov 2020, 11:20 Erez Rokah, [e-mail] wrote:

Thanks @FTWinston https://github.com/FTWinston, maybe it would be a
good idea to have an increased default timeout for POST requests and also
make it configurable.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/netlify/netlify-cms/issues/4538#issuecomment-723563209,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAOG32MGOEJNTRRUEBHHDD3SOZ5INANCNFSM4TNVBKPQ
.

[KancerEzeroglu]

I'm going to work on this issue 🎉

[KancerEzeroglu]

Hi,

I changed the timeout to 120 in the unsentRequest.js file.

https://github.com/netlify/netlify-cms/blob/dd080347b09606d651bb154a9a4870d704f0d48e/packages/netlify-cms-lib-util/src/unsentRequest.js#L13

Then I start getting the error below:

It's coming directly from GitHub and I couldn't find the root cause.
How should we proceed?

[TristanSchorn]

@KancerEzeroglu I found this which seems to imply this is an HTTP post buffer size limitation. Is it possible to reduce the buffer Netlify CMS uses?

[KancerEzeroglu]

Hi,

I played with timeouts on our side but I still got errors from Github API, for me, it seems there are some limitations on the Github site.
I couldn't find any option to set timeout and/or buffer size in the APIs.

As an improvement, we can show a more user-friendly error instead of logging an error to the console.

@erezrokah wdyt?