Prisma-client-js: Unknown Error with Prisma Preview 16-17.2

[CaptainChemist]

I have a Prisma 2 app that will look up every second for Triggers that match the current time. I will repeatedly look up Triggers and last night when I was running it, I got the following error:

Oops, an unknown error occured! This is on us, you did nothing wrong.
It occured in the `photon.()` invocation in /app/conduit-api/src/types/rootTypes.ts:41:56
Reason: called `Option::unwrap()` on a `None` value in src/libcore/option.rs:378:21
Please create an issue in the photonjs repo with your `schema.prisma` and the Photon method you tried to use 🙏:
https://github.com/prisma/photonjs/issues/new

My resolver is:

      resolve: async (_, { time }, ctx) => {
        const foundEvents = await ctx.photon.triggers.findMany({
          where: { type: 'EVENT', hasExecuted: false },
        });

        if (time) {
          const lowerTime = new Date(time);
          lowerTime.setMilliseconds(0);
          const upperTime = new Date(time);
          upperTime.setMilliseconds(999);
          const lowerTimeMinutes = lowerTime.getMinutes();
          lowerTime.setMinutes(lowerTimeMinutes - 1);
          const foundTimes = await ctx.photon.triggers.findMany({
            where: {
              AND: [
                { time: { gte: lowerTime } },
                { time: { lte: upperTime } },
                { hasExecuted: false },
              ],
            },
          });
          return [...foundTimes, ...foundEvents];
        }
        return [...foundEvents];
      },

The relevant part of my schema is:

model Trigger {
    id          String      @default(cuid()) @id @unique
    createdAt   DateTime    @default(now())
    updatedAt   DateTime    @updatedAt
    type        TriggerType
    time        DateTime?
    hasExecuted Boolean     @default(false)
}

Unfortunately, after that error occurred, it did not gracefully recover because then every time I tried to run an updateTrigger mutation (just a standard nexus based mutation) it would error out. This same error occurs actually when I run any arbitrary command against the backend. Thanks so much for looking into this!

Invalid `photon.()` invocation in /app/conduit-api/node_modules/nexus-prisma/src/builder.ts:236:18
undefined target=\"exit\" timestamp=\"2019-11-21T02:04:19.354Z\" fields={\"message\":\"255\"}","stack":"Error: 
Invalid `photon.()` invocation in /app/conduit-api/node_modules/nexus-prisma/src/builder.ts:236:18
undefined target=\"exit\" timestamp=\"2019-11-21T02:04:19.354Z\" fields={\"message\":\"255\"}
at PhotonFetcher.<anonymous> (/app/conduit-api/node_modules/@generated/photon/index.js:51:27)
at Generator.throw (<anonymous>)
at rejected (/app/conduit-api/node_modules/@generated/photon/index.js:6:65)
at processTicksAndRejections (internal/process/task_queues.js:85:5)

[pantharshit00]

Can you please send me the stack trace ran in debug mode? To enable debug mode run export DEBUG=* before running the backend.

[CaptainChemist]

Sure- I will try to replicate this in debug mode. Thanks!

[CaptainChemist]

I was able to get the same kind of error to occur again with the tracing turned on. It happened this time on a different model, but the error is the same. I hope this is helpful!

2019-11-21T19:00:31.813Z photon mutation {
backend         |   createOneExecutionLog(data: {
backend         |     trigger: "{\"id\":\"ck38z9pxj04881pnz2xr57t0k\",\"type\":\"TIME\",\"time\":\"2019-11-21T18:58:38.000Z\",\"stage\":{\"id\":\"ck38z9pw704351pnzfd0kl506\",\"actions\":[{\"id\":\"ck38z9pw904361pnzqi15r6z8\",\"type\":\"MUTATION\",\"name\":\"setMode\",\"endpoint\":\"http://backend:4000\",\"order\":0,\"variables\":{\"echoMode\":null,\"setMode\":{\"mode\":\"STANDBY\",\"parameter1\":null,\"parameter2\":null,\"targetSOC\":null},\"writeInputs\":null}}]}}"
backend         |     action: "{\"id\":\"ck38z9pw904361pnzqi15r6z8\",\"type\":\"MUTATION\",\"name\":\"setMode\",\"endpoint\":\"http://backend:4000\",\"order\":0,\"variables\":{\"echoMode\":null,\"setMode\":{\"mode\":\"STANDBY\",\"parameter1\":null,\"parameter2\":null,\"targetSOC\":null},\"writeInputs\":null}}"
backend         |     stage: "{\"id\":\"ck38z9pw704351pnzfd0kl506\",\"actions\":[{\"id\":\"ck38z9pw904361pnzqi15r6z8\",\"type\":\"MUTATION\",\"name\":\"setMode\",\"endpoint\":\"http://backend:4000\",\"order\":0,\"variables\":{\"echoMode\":null,\"setMode\":{\"mode\":\"STANDBY\",\"parameter1\":null,\"parameter2\":null,\"targetSOC\":null},\"writeInputs\":null}}]}"
backend         |     result: "{\"setMode\":[{\"_fc\":6,\"_address\":3,\"_value\":5,\"_code\":null}]}"
backend         |     success: true
backend         |   }) {
backend         |     id
backend         |     createdAt
backend         |     updatedAt
backend         |     trigger
backend         |     action
backend         |     stage
backend         |     result
backend         |     success
backend         |   }
backend         | }
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router dispatching POST /
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router query  : /
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router expressInit  : /
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router mounted_app  : /
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router dispatching POST /
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router query  : /
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router expressInit  : /
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router responseTime  : /
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router corsMiddleware  : /
backend         | Thu, 21 Nov 2019 19:00:31 GMT body-parser:json content-type "application/json"
backend         | Thu, 21 Nov 2019 19:00:31 GMT body-parser:json content-encoding "identity"
backend         | Thu, 21 Nov 2019 19:00:31 GMT body-parser:json read body
backend         | Thu, 21 Nov 2019 19:00:31 GMT body-parser:json parse body
backend         | Thu, 21 Nov 2019 19:00:31 GMT body-parser:json parse json
backend         | 2019-11-21T19:00:31.845Z photon Request:
backend         | 2019-11-21T19:00:31.845Z photon mutation {
backend         |   createOneExecutionLog(data: {
backend         |     trigger: "{\"id\":\"ck38z9pxj04881pnz2xr57t0k\",\"type\":\"TIME\",\"time\":\"2019-11-21T18:58:38.000Z\",\"stage\":{\"id\":\"ck38z9pw704351pnzfd0kl506\",\"actions\":[{\"id\":\"ck38z9pw904361pnzqi15r6z8\",\"type\":\"MUTATION\",\"name\":\"setMode\",\"endpoint\":\"http://backend:4000\",\"order\":0,\"variables\":{\"echoMode\":null,\"setMode\":{\"mode\":\"STANDBY\",\"parameter1\":null,\"parameter2\":null,\"targetSOC\":null},\"writeInputs\":null}}]}}"
backend         |     action: "{\"id\":\"ck38z9pw904361pnzqi15r6z8\",\"type\":\"MUTATION\",\"name\":\"setMode\",\"endpoint\":\"http://backend:4000\",\"order\":0,\"variables\":{\"echoMode\":null,\"setMode\":{\"mode\":\"STANDBY\",\"parameter1\":null,\"parameter2\":null,\"targetSOC\":null},\"writeInputs\":null}}"
backend         |     stage: "{\"id\":\"ck38z9pw704351pnzfd0kl506\",\"actions\":[{\"id\":\"ck38z9pw904361pnzqi15r6z8\",\"type\":\"MUTATION\",\"name\":\"setMode\",\"endpoint\":\"http://backend:4000\",\"order\":0,\"variables\":{\"echoMode\":null,\"setMode\":{\"mode\":\"STANDBY\",\"parameter1\":null,\"parameter2\":null,\"targetSOC\":null},\"writeInputs\":null}}]}"
backend         |     result: "{\"setMode\":[{\"_fc\":6,\"_address\":3,\"_value\":5,\"_code\":null}]}"
backend         |     success: true
backend         |   }) {
backend         |     id
backend         |     createdAt
backend         |     updatedAt
backend         |     trigger
backend         |     action
backend         |     stage
backend         |     result
backend         |     success
backend         |   }
backend         | }
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router dispatching POST /
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router query  : /
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router expressInit  : /
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router mounted_app  : /
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router dispatching POST /
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router query  : /
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router expressInit  : /
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router responseTime  : /
backend         | Thu, 21 Nov 2019 19:00:31 GMT express:router corsMiddleware  : /
backend         | Thu, 21 Nov 2019 19:00:31 GMT body-parser:json content-type "application/json"
backend         | Thu, 21 Nov 2019 19:00:31 GMT body-parser:json content-encoding "identity"
backend         | Thu, 21 Nov 2019 19:00:31 GMT body-parser:json read body
backend         | Thu, 21 Nov 2019 19:00:31 GMT body-parser:json parse body
backend         | Thu, 21 Nov 2019 19:00:31 GMT body-parser:json parse json
backend         | 2019-11-21T19:00:31.898Z photon Request:
backend         | 2019-11-21T19:00:31.898Z photon mutation {
backend         |   createOneExecutionLog(data: {
backend         |     trigger: "{\"id\":\"ck38z9pxj04881pnz2xr57t0k\",\"type\":\"TIME\",\"time\":\"2019-11-21T18:58:38.000Z\",\"stage\":{\"id\":\"ck38z9pw704351pnzfd0kl506\",\"actions\":[{\"id\":\"ck38z9pw904361pnzqi15r6z8\",\"type\":\"MUTATION\",\"name\":\"setMode\",\"endpoint\":\"http://backend:4000\",\"order\":0,\"variables\":{\"echoMode\":null,\"setMode\":{\"mode\":\"STANDBY\",\"parameter1\":null,\"parameter2\":null,\"targetSOC\":null},\"writeInputs\":null}}]}}"
backend         |     action: "{\"id\":\"ck38z9pw904361pnzqi15r6z8\",\"type\":\"MUTATION\",\"name\":\"setMode\",\"endpoint\":\"http://backend:4000\",\"order\":0,\"variables\":{\"echoMode\":null,\"setMode\":{\"mode\":\"STANDBY\",\"parameter1\":null,\"parameter2\":null,\"targetSOC\":null},\"writeInputs\":null}}"
backend         |     stage: "{\"id\":\"ck38z9pw704351pnzfd0kl506\",\"actions\":[{\"id\":\"ck38z9pw904361pnzqi15r6z8\",\"type\":\"MUTATION\",\"name\":\"setMode\",\"endpoint\":\"http://backend:4000\",\"order\":0,\"variables\":{\"echoMode\":null,\"setMode\":{\"mode\":\"STANDBY\",\"parameter1\":null,\"parameter2\":null,\"targetSOC\":null},\"writeInputs\":null}}]}"
backend         |     result: "{\"setMode\":[{\"_fc\":6,\"_address\":3,\"_value\":5,\"_code\":null}]}"
backend         |     success: true
backend         |   }) {
backend         |     id
backend         |     createdAt
backend         |     updatedAt
backend         |     trigger
backend         |     action
backend         |     stage
backend         |     result
backend         |     success
backend         |   }
backend         | }
backend         | {"level":50,"time":1574362831901,"pid":39,"hostname":"5eaa862413c8","container":"backend","requestId":"ck392z7iq0c8c13nzhq4ydkod","operation":"mutation","name":"createOneExecutionLog","args":"{\"data\":{\"trigger\":\"{\\\"id\\\":\\\"ck38z9pxa04761pnzmow65wfo\\\",\\\"type\\\":\\\"TIME\\\",\\\"time\\\":\\\"2019-11-21T18:34:38.000Z\\\",\\\"stage\\\":{\\\"id\\\":\\\"ck38z9pw704351pnzfd0kl506\\\",\\\"actions\\\":[{\\\"id\\\":\\\"ck38z9pw904361pnzqi15r6z8\\\",\\\"type\\\":\\\"MUTATION\\\",\\\"name\\\":\\\"setMode\\\",\\\"endpoint\\\":\\\"http://backend:4000\\\",\\\"order\\\":0,\\\"variables\\\":{\\\"echoMode\\\":null,\\\"setMode\\\":{\\\"mode\\\":\\\"STANDBY\\\",\\\"parameter1\\\":null,\\\"parameter2\\\":null,\\\"targetSOC\\\":null},\\\"writeInputs\\\":null}}]}}\",\"action\":\"{\\\"id\\\":\\\"ck38z9pw904361pnzqi15r6z8\\\",\\\"type\\\":\\\"MUTATION\\\",\\\"name\\\":\\\"setMode\\\",\\\"endpoint\\\":\\\"http://backend:4000\\\",\\\"order\\\":0,\\\"variables\\\":{\\\"echoMode\\\":null,\\\"setMode\\\":{\\\"mode\\\":\\\"STANDBY\\\",\\\"parameter1\\\":null,\\\"parameter2\\\":null,\\\"targetSOC\\\":null},\\\"writeInputs\\\":null}}\",\"stage\":\"{\\\"id\\\":\\\"ck38z9pw704351pnzfd0kl506\\\",\\\"actions\\\":[{\\\"id\\\":\\\"ck38z9pw904361pnzqi15r6z8\\\",\\\"type\\\":\\\"MUTATION\\\",\\\"name\\\":\\\"setMode\\\",\\\"endpoint\\\":\\\"http://backend:4000\\\",\\\"order\\\":0,\\\"variables\\\":{\\\"echoMode\\\":null,\\\"setMode\\\":{\\\"mode\\\":\\\"STANDBY\\\",\\\"parameter1\\\":null,\\\"parameter2\\\":null,\\\"targetSOC\\\":null},\\\"writeInputs\\\":null}}]}\",\"result\":\"{\\\"setMode\\\":[{\\\"_fc\\\":6,\\\"_address\\\":3,\\\"_value\\\":5,\\\"_code\\\":null}]}\",\"success\":true}}","err":{"type":"Error","message":"\nInvalid `photon.()` invocation in /app/backend/node_modules/nexus-prisma/src/builder.ts:236:18\n\n\n\nundefined target=\"exit\" timestamp=\"2019-11-21T19:00:06.364Z\" fields={\"message\":\"255\"}","stack":"Error: \nInvalid `photon.()` invocation in /app/backend/node_modules/nexus-prisma/src/builder.ts:236:18\n\n\n\nundefined target=\"exit\" timestamp=\"2019-11-21T19:00:06.364Z\" fields={\"message\":\"255\"}\n    at PhotonFetcher.<anonymous> (/app/backend/node_modules/@generated/photon/index.js:51:27)\n    at Generator.throw (<anonymous>)\n    at rejected (/app/backend/node_modules/@generated/photon/index.js:6:65)\n    at processTicksAndRejections (internal/process/task_queues.js:85:5)"},"v":1}
backend         | Error: 
backend         | Invalid `photon.()` invocation in /app/backend/node_modules/nexus-prisma/src/builder.ts:236:18
backend         | 
backend         | 
backend         | 
backend         | undefined target="exit" timestamp="2019-11-21T19:00:06.364Z" fields={"message":"255"}
backend         |     at PhotonFetcher.<anonymous> (/app/backend/node_modules/@generated/photon/index.js:51:27)
backend         |     at Generator.throw (<anonymous>)
backend         |     at rejected (/app/backend/node_modules/@generated/photon/index.js:6:65)
backend         |     at processTicksAndRejections (internal/process/task_queues.js:85:5)
backend         | Error: 
backend         | Invalid `photon.()` invocation in /app/backend/node_modules/nexus-prisma/src/builder.ts:236:18
backend         | 
backend         | 
backend         | 
backend         | undefined target="exit" timestamp="2019-11-21T19:00:06.364Z" fields={"message":"255"}
backend         |     at PhotonFetcher.<anonymous> (/app/backend/node_modules/@generated/photon/index.js:51:27)
backend         |     at Generator.throw (<anonymous>)
backend         |     at rejected (/app/backend/node_modules/@generated/photon/index.js:6:65)
backend         |     at processTicksAndRejections (internal/process/task_queues.js:85:5)

[momakes3]

undefined target="exit" timestamp="2019-11-21T19:00:06.364Z" fields={"message":"255"}

This error gives no information on what caused the issue and I've seen it a couple of times already 😄

[malekjaroslav]

This happens often to me as well

[pantharshit00]

Can you all please retest this with preview017(npm install -g prisma2)?

If anyone has a simpler reproduction for this, please share that as that would help me a ton 🙏

[martonlanga]

Happens with preview017 as well for me

[martonlanga]

I got this error while running npm run seed. I was using an .env variable in schema.prisma for the db url, which wasn't resolved during build (npm i: postinstall -> generate). So I replaced the db url with the actual string, ran npm i and now npm run seed worked.

So I guess you could reproduce @pantharshit00 this by defining an empty string as the db url in schema.prisma.

[pantharshit00]

Yes, .env not getting read is a known issue. https://github.com/prisma/prisma2/issues/956

@CaptainChemist Can you please confirm that you are also using environment variables in the schema.prisma file?

[pantharshit00]

@martonlanga

Thanks again. I can reproduce this by generating the client when I have an empty string the url property of datasource.

I think we should have a better error here. So I am going to mark this as an confirmed bug.

[malekjaroslav]

@pantharshit00 getting this error now more often. I turned on the DEBUG mode and discovered some kind of PORT conflict.

[BE] 2019-11-30T07:54:46.099Z engine connect ECONNREFUSED 127.0.0.1:57310
[BE] Error: 
[BE] Invalid `photon.()` invocation in /Users/macbookpro/Sites/farely/farely/backend/build/webpack:/src/resolvers/auth.ts:60:41
[BE] 
[BE] 
[BE] 
[BE] undefined target="exit" timestamp="2019-11-30T07:54:42.995Z" fields={"message":"1"}
[BE]     at PhotonFetcher.<anonymous> (/Users/macbookpro/Sites/farely/farely/backend/node_modules/@prisma/photon/index.js:54:27)
[BE]     at Generator.throw (<anonymous>)
[BE]     at rejected (/Users/macbookpro/Sites/farely/farely/backend/node_modules/@prisma/photon/index.js:6:65)

[CaptainChemist]

@pantharshit00 I can definitely confirm that I'm providing the proper envs. The error that I received happened after hours of sending photon calls once per minute, so it unfortunately seems like more of a transient thing. Unfortunately, once it produces an error it will create an error in every subsequent photon call until I restart the docker container.

I'm trying this again now with prisma v17.2 to see if it works better and report back. Thanks again!

[alexichepura]

Hey all, I was trying to debug this:
Error: undefined target="exit" timestamp="2019-12-02T19:06:35.149Z" fields={"message":"255"}

We use docker. The issue appeared after few hundreds requests i've made to the server.
Here is the difference in the node processes working VS broken.

Note /app/node_modules/@prisma/photon/runtime/query-engine-debian-openssl-1.1.x is running before load test, but stopped after.

To quickly test i've used https://www.npmjs.com/package/loadtest
loadtest -c 10 --rps 200 https://yoursite.com

[CaptainChemist]

@alexichepura fascinating- your observations explain why every photon call fails for me after it fails the first time. It must be that the engine dies. That's also a great suggestion on using a loadtest to speed along the failure.

[alexichepura]

@CaptainChemist
Thx for the comment) Maybe you can adjust the title to Prisma Preview 16-17.2 so it has more attension?
@pantharshit00 @divyenduz please can this issue be prioritised? We started to work with prisma2 and faced it from the beggining. But now we cannot even show a proper staging for the customers, as it's brokes constantly.

[malekjaroslav]

@CaptainChemist
Thx for the comment) Maybe you can adjust the title to Prisma Preview 16-17.2 so it has more attension?
@pantharshit00 @divyenduz please can this issue be prioritised? We started to work with prisma2 and faced it from the beggining. But now we cannot even show a proper staging for the customers, as it's brokes constantly.

Exactly, we are in the same position...

[divyenduz]

Thank you for raising your concerns @alexichepura @malekjaroslav

I have marked this issue as a candidate and it will be discussed in the upcoming planning meeting that happens on Thursday.

[janpio]

Hey @alexichepura @malekjaroslav @martonlanga @momakes can you please post the _complete_ error message you are getting (best with DEBUG=* being set) please? Also, if you have some code you can share we would greatly appreciate that.

Are you all running in Docker containers?
Can you consistently reproduce this?
(If yes to both, could you privately share the Docker container maybe?)

Right now we do not have a clear picture if this is all the same problem, or multiple different problems that just end in the same bad error message - and to figure that out we need a lot more information than just a "+1, happens to me as well". Would be great if you could help us out.

[pantharshit00]

I tried reproducing this in loadtest with an express server that performs a user creation operation with Apache bench and I was unable to reproduce this.

Also, can you all please confirm whether you are using nexus-prisma or not. ALso, please tell us the version of that as well.

[malekjaroslav]

Happens to me in docker and without docker as well. I am using [email protected]

[pantharshit00]

I opened a separate issue for empty datasource url problem: https://github.com/prisma/photonjs/issues/318

[pantharshit00]

@malekjaroslav Can you please the code in a git repository with us if possible?

[malekjaroslav]

@pantharshit00 I think that is not possible as it is a private project, but I can share my package.json or some parts you need

[malekjaroslav]

I tried to loadtest it and it did not fail neither. It seems like it occurs on different occasions...

[alexichepura]

Ok, i've tried to reproduce it in super minimal repo. But it works
https://github.com/alexichepura/photon-issue-305

So I need add more to it, similar to our real project.

[alexichepura]

@pantharshit00 or I can share the codebase that fails privately with your team. Tomorrow.

[alexichepura]

Ok, it doesn't fail now on the staging and we cannot replicate the issue in any way.
Currently it's preview 17.2 and all fine.
Just thinking if it's related to the binaries? And if them got updated recently..
> Downloading darwin binary for query-engine [====================] 100%

@malekjaroslav did you upgrade? We can close this issue if it works for all.

[malekjaroslav]

Haven’t encountered it yet. We can close it and I would comment on the issue if it happens again.

[pantharshit00]

@CaptainChemist Are you still able to reproduce this with 17.2?

[CaptainChemist]

@pantharshit00 I've been testing this the past few days and I haven't been able to reproduce the problem! I'm down to close this issue. Thanks everyone!

[janpio]

Just thinking if it's related to the binaries? And if them got updated recently..
> Downloading darwin binary for query-engine [====================] 100%

You always get the exact same binaries with a specific version of the CLI. Even if we release new ones, you only get them when you install the new CLI.

[malekjaroslav]

@pantharshit00 @janpio So I am on Preview 18 and still getting this error

"Error in Photon: \nError: undefined target=\"exit\" timestamp=\"2019-12-09T06:32:22.006Z\" fields={\"message\":\"1\"}\n    at NodeEngine.engineReady (/app/node_modules/@prisma/photon/runtime/index.js:3512:23)",

and

Error in Photon: \nError: undefined target=\"exit\" timestamp=\"2019-12-09T06:53:22.592Z\" fields={\"message\":\"255\"}\n    at currentRequestPromise.then.catch.error (/app/node_modules/@prisma/photon/runtime/index.js:3569:27)\n    at process._tickCallback (internal/process/next_tick.js:68:7

I get to this point loadtesting the API like this:
loadtest -c 10 --rps 500 -T 'application/json' -P '{"query":"{chapters{id maxPoints}}"}' https://graphql-api-url.example/

[pantharshit00]

@malekjaroslav Can you please share the resolver code for this? Without any reproduction, I can't help you here since I am unable to reproduce this issue by any means so far.

If you want to keep it private, you can mail it to [email protected] or you can invite me to a private repository

[lvauvillier]

@pantharshit00 I am still having this issue on Preview 18.

This is hard to reproduce, but here is some info about the context (which is different from @malekjaroslav ) :

• I run my project on Google cloud Run
• The error appears after a long period of inactivity (maybe after a cold start of an instance)
• When the error appears, the next queries are all in error.
• I have to redeploy the same image to fix the issue.

[malekjaroslav]

@lvauvillier I am running on Cloud Run as well, but the issue is not inactivity as it happens also under heavy loads, but maybe it has something to do with scaling (cold start)

[lvauvillier]

@malekjaroslav in my case, this is is not heavy load for sure.

This happens in a dev workflow, and the issue appears generally on the first request in the morning after a night of inactivity...

Cold start seems to be a potential root cause.
This issue mention a problem with env vars (https://github.com/prisma/photonjs/issues/318). I'll add a route to return my ENV vars to be sure they are properly set when the issue will happen again.

[RafaelKr]

This error is similar to that one: https://github.com/prisma-labs/nexus-prisma/issues/546

We found out that this error is logged when Photon can't reach the database:

Error: 
Invalid `photon.()` invocation in /path/to/project/packages/server/node_modules/nexus-prisma/src/builder.ts:238:18

  234 const photon = this.getPhoton(ctx)
  235 assertPhotonInContext(photon)
  236 return photon[mappedField.photonAccessor][
  237   mappedField.operation
→ 238 ](args

undefined target="exit" timestamp="2019-12-04T16:44:27.839Z" fields={"message":"1"}
    at PhotonFetcher.<anonymous> (/path/to/project/node_modules/@prisma/photon/index.js:54:27)
    at Generator.throw (<anonymous>)
    at rejected (/path/to/project/node_modules/@prisma/photon/index.js:6:65)

Maybe your errors are also related to that.

I could imagine the following:
High Load = Database not responding before a timeout
Cold start = Connection to database not available (yet)
Long inactivity = The connection was closed and is not correctly reestablished

[malekjaroslav]

@RafaelKr That would make sense. Also it would mean Photon doesn’t retry or at least not properly and rather dies.

[RafaelKr]

@malekjaroslav I just found the following issues: https://github.com/prisma/photonjs/issues/273, https://github.com/prisma/photonjs/issues/274
So it seems my theory about cold start and long inactivity are very likely related to this.

[embiem]

I have to add my voice here as well... having a similar problem. I'm using ZEIT now and when developing locally all API endpoints work fine. Then on a deployment, after some initially working API endpoints, some of the endpoints start running into this issue:

Unhandled rejection: Error: Error in Photon: 
Error: undefined target="exit" timestamp="2019-12-10T14:07:16.007Z" fields={"message":"1"}
    at NodeEngine.engineReady (/var/task/db/node_modules/@prisma/photon/runtime/index.js:3512:23)
    at <anonymous>
    at PhotonFetcher.<anonymous> (/var/task/db/node_modules/@prisma/photon/index.js:72:31)
    at Generator.throw (<anonymous>)
    at rejected (/var/task/db/node_modules/@prisma/photon/index.js:6:65)
    at <anonymous>

This is then consistently happening for these endpoints, while the initially working endpoints still run fine.

I guess it might be related to #274.

[harri121]

@pantharshit00 I was also experiencing this error. I've noticed that this error occurs when connections are kept open, i.e. when photon.disconnect() is not called for every photon.connect().

[pantharshit00]

I am waiting on https://github.com/prisma/photonjs/issues/357.

I don't want to get too deep into this without having much information on what is getting returned from the engine. So I am waiting on a fix that would improve the error message.

[embiem]

I'll update once the fix of #357 arrives in a new version.

[PeterKow]

I had this problem after deploying to now - zeit

from graphql playground:
```{
"data": null,
"errors": [
{
"message": "Error in Photon: \nError: undefined target=\"exit\" timestamp=\"2019-12-20T13:39:00.994Z\" fields={\"message\":\"1\"}\n at NodeEngine.engineReady (/var/task/node_modules/@prisma/photon/runtime/index.js:3533:23)\n at /var/task/node_modules/@prisma/photon/runtime/index.js:3456:21",
"locations": [
{
"line": 2,
"column": 3
}
],
"path": [
"feed"
]
}
]
}

Looks like env on execution is lost so you have to pass it on build AND normal env

my `now.json`
```{
  "version": 2,
  "name": "api",
  "env": {
    "POSTGRES_URL": "@postgres_url"
  },
  "build": {
    "env": {
      "POSTGRES_URL": "@postgres_url"
    }
  },
  "builds": [
    {
      "src": "services/api/src/server.ts",
      "use": "@now/node@canary"
    }
  ],
  "routes": [{ "src": "/.*", "dest": "services/api/src/server.ts" }]
}

and with this, it works 👍

FYI: @embiem ^^

[embiem]

@PeterKow I've actually been doing this. I have POSTGRES_URL defined both in build and for runtime, exactly like you.

Additionally, I don't think this is the cause of the issue, as I shouldn't have gotten any connection to the DB without the env variable.

[embiem]

@pantharshit00 I got the error message of the DB, and it seems like I'm simply hitting the max connections limit on the postgres db:

sorry, too many clients already

(full error message below)

This is what I'm calling to get a photon instance in any lambda functions that need DB access:

let photon = null;

async function getPhoton() {
  if (photon === null) {
    photon = new Photon();
    console.log("Create new Photon instance");
  }

  return photon;
};

AFAIK there's no way to know the connection status, otherwise I'd try to optimize it based on the approach shown here: https://spectrum.chat/zeit/now/now-2-0-connect-to-database-on-every-function-invocation~e25b9e64-6271-4e15-822a-ddde047fa43d

Is prisma2 aiming to supporting usage in serverless environments? Am I missing something?

undefined timestamp="2001-12-29T12:02:21.385Z" target="mobc" fields={"message":"mobc failed to connect: Compat { error: QueryError(Error { kind: Db, cause: Some(DbError { severity: \"FATAL\", 
parsed_severity: Some(Fatal), code: SqlState(\"53300\"), message: \"sorry, too many clients already\", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some(\"proc.c\"), line: Some(363), routine: Some(\"InitProcess\") }) }\n\nstack 
backtrace:\n   0: backtrace::backtrace::trace\n   1: backtrace::capture::Backtrace::new_unresolved\n   2: failure::backtrace::internal::InternalBacktrace::new\n   3: <failure::backtrace::Backtrace as core::default::Default>::default\n   4: quaint::connector::postgres::error::<impl core::convert::From<tokio_postgres::error::Error> for quaint::error::Error>::from\n   5: quaint::connector::postgres::PostgreSql::new::{{closure}}\n   6: <std::future::GenFuture<T> as core::future::future::Future>::poll\n   7: <std::future::GenFuture<T> as core::future::future::Future>::poll\n   8: tokio::task::core::Core<T>::poll\n   9: std::panicking::try::do_call\n  10: __rust_maybe_catch_panic\n             at src\\libpanic_unwind/lib.rs:80\n  11: tokio::task::harness::Harness<T,S>::poll\n  12: tokio::runtime::thread_pool::worker::GenerationGuard::run_task\n  13: tokio::runtime::thread_pool::worker::GenerationGuard::run\n  14: std::thread::local::LocalKey<T>::with\n  15: std::thread::local::LocalKey<T>::with\n  16: std::thread::local::LocalKey<T>::with\n  17: tokio::runtime::thread_pool::worker::Worker::run\n  18: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n  19: std::panicking::try::do_call\n  20: __rust_maybe_catch_panic\n             at src\\libpanic_unwind/lib.rs:80\n  21: tokio::task::harness::Harness<T,S>::poll\n  22: tokio::runtime::blocking::pool::Inner::run2\n  23: std::thread::local::LocalKey<T>::with\n  24: tokio::runtime::time::variant::with_default\n  25: std::sys_common::backtrace::__rust_begin_short_backtrace\n  26: __rust_maybe_catch_panic\n             at src\\libpanic_unwind/lib.rs:80\n  27: core::ops::function::FnOnce::call_once{{vtable.shim}}\n  28: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once\n        
     at /rustc/4560ea788cb760f0a34127156c78e2552949f734\\src\\liballoc/boxed.rs:922\n  29: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once\n             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\\src\\liballoc/boxed.rs:922\n   
   std::sys_common::thread::start_thread\n             at src\\libstd\\sys_common/thread.rs:13\n      std::sys::windows::thread::Thread::new::thread_start\n             at src\\libstd\\sys\\windows/thread.rs:47\n  30: sqlite3GenerateConstraintChecks\n  31: sqlite3GenerateConstraintChecks\n) }","log.target":"mobc","log.module_path":"mobc","log.file":"/root/.cargo/registry/src/github.com-1ecc6299db9ec823/mobc-0.3.1/src/lib.rs","log.line":455}

[RafaelKr]

Hi @embiem, this document should answer your question about serverless environments: https://github.com/prisma/prisma2/blob/master/docs/photon/deployment.md

[embiem]

@RafaelKr thanks a lot, this info helped!

I'm Not sure why I missed this document... sorry.

[divyenduz]

@embiem Since #357 is fixed, can you please try this again and see what the actual error is? Thanks.

This issue looks to contain multiple errors, if you think that your error is different from the main topic of this issue, please create a new GH issue for that.

[pantharshit00]

Closing due to inactivity, please open a new issue if you are still facing this.