Prisma-client-js: Timeouts in Photon for high parallel usage

[timsuchanek]

When using Photon with about 100 concurrent requests per second, got, out library used for querying the query-engine returns timeouts:

 Error:
 Invalid `photon.()` invocation in /app/src/scrapers/link-to-article/worker.ts:127:6



 Reason: {
   "name": "HTTPError",
   "host": "localhost:46685",
   "hostname": "localhost",
   "method": "POST",
   "path": "/",
   "protocol": "http:",
   "url": "http://localhost:46685/",
   "gotOptions": {
     "path": "/",
     "protocol": "http:",
     "slashes": true,
     "auth": null,
     "host": "localhost:46685",
     "port": "46685",
     "hostname": "localhost",
     "hash": null,
     "search": null,
     "query": null,
     "pathname": "/",
     "href": "http://localhost:46685/",
     "retry": {
       "methods": {},
       "statusCodes": {},
       "errorCodes": {}
     },
     "headers": {
       "user-agent": "got/9.6.0 (https://github.com/sindresorhus/got)",
       "content-type": "application/json",
       "accept": "application/json",
       "accept-encoding": "gzip, deflate",
       "content-length": 4298
     },
     "hooks": {
       "beforeRequest": [],
       "beforeRedirect": [],
       "beforeRetry": [],
       "afterResponse": [],
       "beforeError": [],
       "init": []
     },
     "decompress": true,
     "throwHttpErrors": true,
     "followRedirect": true,
     "stream": false,
     "form": false,
     "json": true,
     "cache": false,
     "useElectronNet": false,
     "body": "{\"query\":\"mutation {\\n  createOneArticle(data: {\\n    wordCount: 635\\n    link: {\\n      connect: {\\n        id: 833953\\n      }\\n    }\\n    text: \\\"Credit to Chris Hallbeck @ Maximumble.comI remember being in graduate school and being very excited to begin my practicums. My studies and skills being put to the test with REAL LIFE clients! I envisioned sitting in my chair, as the client laid.\\\"\\n    byline: \\\"Steve Oh\\\"\\n    excerpt: \\\"I remember being in graduate school and being very excited to begin my practicums. My studies and skills being put to the test with REAL…\\\"\\n    siteName: \\\"Medium\\\"\\n    title: \\\"“No I’m not”. The use of denial and preservation\\\"\\n  }) {\\n    id\\n    wordCount\\n    text\\n    excerpt\\n    byline\\n    title\\n    siteName\\n  }\\n}\",\"variables\":{}}",
     "method": "POST"
   },
   "statusCode": 408,
   "statusMessage": "Request Timeout",
   "headers": {
     "content-length": "0",
     "date": "Mon, 28 Oct 2019 07:40:14 GMT"
   },
   "body": ""
 }

     at PhotonFetcher.<anonymous> (/app/node_modules/@generated/photon/index.js:47:27)
     at Generator.throw (<anonymous>)
     at rejected (/app/node_modules/@generated/photon/index.js:6:65)
     at runMicrotasks (<anonymous>)
     at processTicksAndRejections (internal/process/task_queues.js:93:5)

We need to invest in a testing setup with full-stack performance testing (including Node.js and Rust) to prevent this.

[divyenduz]

Internal note: candidate from bug triaging meeting

[pantharshit00]

Might be related to https://github.com/prisma/photonjs/issues/305

[pantharshit00]

I can confirm this

[divyenduz]

@pantharshit00 Can you please share your project? From the screenshot, I don't know if you are using queries or mutations? What the schema and query is like.

[divyenduz]

I can reproduce this, this repo has the reproduction steps: https://github.com/divyendu-test/p2-perf

[timsuchanek]

Update: This is not a bug, but in this case we're just hitting the database limits. The default timeout in the rust engine is 5s, which is being reached here.
We can improve the error message, but in general the user either needs to increase the timeout or scale up the database machine.

We'll add a &timeout=X where X is the timeout in milliseconds to the connection string to configure databases.

[pimeys]

I suggest following the postgres manual here with the timeout parameters:

connectTimeout = int

The timeout value used for socket connect operations. If connecting to the server takes longer than this value, the connection is broken. The timeout is specified in seconds and a value of zero means that it is disabled.

socketTimeout = int

The timeout value used for socket read operations. If reading from the server takes longer than this value, the connection is closed. This can be used as both a brute force global query timeout and a method of detecting network problems. The timeout is specified in seconds and a value of zero means that it is disabled.

[pimeys]

https://jdbc.postgresql.org/documentation/head/connect.html

[pimeys]

We can get both timeouts for PostgreSQL and MySQL. SQLite doesn't naturally have connect_timeout, but we can get some kind of socket_timeout when using https://sqlite.org/c3ref/busy_timeout.html

[pimeys]

• connect_timeout is the time we can wait for a new connection to be available for that specific client, in seconds!
• socket_timeout marks the time we wait for every single query to finish.
• Both defaults are five seconds.
• SQLite accepts only socket_timeout and is a bit different compared to the other databases. We use the busy_timeout pragma and the timeout is for any other queries that are waiting for a long-running write to finish. So one write can lock the database, and everybody else reading or writing will get a timeout if the one write locks the database for a longer time.

[pimeys]

https://github.com/prisma/quaint/pull/66

[pimeys]

Latest alpha query engine should have this feature.

[janpio]

No further work like Prisma Client JS adaptions or documentation necessary?

[pimeys]

Docs yeah. Do we use the same ticket for those?

[janpio]

Yes, normal issue that has a bit of information so Niko has everything or it links to where he can get stuff - and we then apply a kind/docs label.

[pimeys]

* `connect_timeout` is the time we can wait for a new connection to be available for that specific client, in seconds!

* `socket_timeout` marks the time we wait for every single query to finish.

* Both defaults are five seconds.

* SQLite accepts only `socket_timeout` and is a bit different compared to the other databases. We use the `busy_timeout` pragma and the timeout is for any other queries that are waiting for a long-running write to finish. So one write can lock the database, and everybody else reading or writing will get a timeout if the one write locks the database for a longer time.

So this is the relevant bit for @nikolasburk

[janpio]

Process prefers new issues, so I created https://github.com/prisma/prisma-client-js/issues/433 Thanks anyway!