Prisma: Unix socket connection not working

Created on 9 Sep 2019  ·  42Comments  ·  Source: prisma/prisma

Tried all possibilities of connections strings to the DB but still unsuccessful deploying to Google Cloud (App Engine/Cloud Run). Connection strings are replaced with placeholders for security.

Step #1 - "builder": $ DEBUG=* RUST_BACKTRACE=full yarn lift:up && yarn build
Step #1 - "builder": $ prisma2 lift up
Step #1 - "builder": info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Step #1 - "builder": info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Step #1 - "builder":
Step #1 - "builder": INFO `yarn_run_gcp_build` had stderr output:
Step #1 - "builder": 2019-09-09T11:44:30.680Z LiftEngine:rpc SENDING RPC CALL { id: 1,
Step #1 - "builder": jsonrpc: '2.0',
Step #1 - "builder": method: 'listMigrations',
Step #1 - "builder": params:
Step #1 - "builder": { projectInfo: '',
Step #1 - "builder": sourceConfig:
Step #1 - "builder": 'datasource db {\n provider = "postgresql"\n url = "postgresql://<usr>:<pass>@<ip>/<db>?schema=public"\n}\n\ngenerator photon {\n provider = "photonjs"\n}\n\ngenerator nexus_prisma {\n provider = "nexus-prisma"\n}\n\nmodel File {\n id String @id @default(cuid())\n name String\n project Project\n url String\n type String\n}\n\nmodel User {\n id String @id @default(cuid())\n authId String @unique\n email String @unique\n projects Project[]\n name String\n profileImg String?\n postVersions PostVersion[]\n faqs Faq[]\n}\n\nmodel Company {\n id String @id @default(cuid())\n name String\n projects Project[]\n}\n\nmodel Project {\n id String @id @default(cuid())\n name String\n company Company\n webUrl String?\n logo String\n logoSmall String\n postCategories PostCategory[]\n faqCategories FaqCategory[]\n posts PostVersion[]\n faqs Faq[]\n admins User[]\n files File[]\n}\n\nmodel Post {\n id String @id @default(cuid())\n draftedVersion PostVersion @relation(name: "draft")\n publishedVersion PostVersion @relation(name: "published")\n}\n\nmodel PostVersion {\n id String @id @default(cuid())\n draftedPost Post? @relation(name: "draft")\n publishedPost Post? @relation(name: "published")\n title String\n text String\n language String\n categories PostCategory[]\n author User\n published Boolean\n project Project\n ogTitle String?\n ogDescription String?\n createdAt DateTime\n updatedAt DateTime @updatedAt\n}\n\nmodel Faq {\n id String @id @default(cuid())\n question String\n answer String\n category FaqCategory?\n published Boolean\n language String\n project Project\n author User\n createdAt DateTime @default(now())\n updatedAt DateTime @updatedAt\n}\n\nmodel PostCategory {\n id String @id @default(cuid())\n name String\n project Project\n posts PostVersion[]\n}\n\nmodel FaqCategory {\n id String @id @default(cuid())\n name String\n project Project\n faqs Faq[]\n}' } }
Step #1 - "builder": 2019-09-09T11:44:33.703Z LiftEngine:stderr thread 'main' panicked at 'Box<Any>', migration-engine/core/src/main.rs:55:23
Step #1 - "builder": 2019-09-09T11:44:33.703Z LiftEngine:stderr stack backtrace:
Step #1 - "builder": 2019-09-09T11:44:33.730Z LiftEngine:stderr 0: backtrace::backtrace::libunwind::trace
Step #1 - "builder": 2019-09-09T11:44:33.730Z LiftEngine:stderr at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/libunwind.rs:88
Step #1 - "builder": 2019-09-09T11:44:33.731Z LiftEngine:stderr 1: backtrace::backtrace::trace_unsynchronized
Step #1 - "builder": 2019-09-09T11:44:33.731Z LiftEngine:stderr at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/mod.rs:66
Step #1 - "builder": 2019-09-09T11:44:33.732Z LiftEngine:stderr 2: std::sys_common::backtrace::_print
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr at src/libstd/sys_common/backtrace.rs:47
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 3: std::sys_common::backtrace::print
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr at src/libstd/sys_common/backtrace.rs:36
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 4: std::panicking::default_hook::{{closure}}
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr at src/libstd/panicking.rs:200
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 5: std::panicking::default_hook
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr at src/libstd/panicking.rs:214
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 6: std::panicking::rust_panic_with_hook
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr at src/libstd/panicking.rs:477
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 7: std::panicking::begin_panic
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 8: migration_engine::main
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 9: std::rt::lang_start::{{closure}}
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 10: std::rt::lang_start_internal::{{closure}}
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr at src/libstd/rt.rs:49
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 11: std::panicking::try::do_call
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr at src/libstd/panicking.rs:296
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr 12: __rust_maybe_catch_panic
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr at src/libpanic_unwind/lib.rs:82
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr 13: std::panicking::try
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr at src/libstd/panicking.rs:275
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr 14: std::panic::catch_unwind
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr at src/libstd/panic.rs:394
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr 15: std::rt::lang_start_internal
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr at src/libstd/rt.rs:48
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr 16: main
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr 17: __libc_start_main
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr 18: _start
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Step #1 - "builder": Error: Error: Error in migration engine: thread 'main' panicked at 'Box<Any>', migration-engine/core/src/main.rs:55:23
Step #1 - "builder": stack backtrace:
Step #1 - "builder": 0: backtrace::backtrace::libunwind::trace
Step #1 - "builder": at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/libunwind.rs:88
Step #1 - "builder": 1: backtrace::backtrace::trace_unsynchronized
Step #1 - "builder": at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/mod.rs:66
Step #1 - "builder": 2: std::sys_common::backtrace::_print
Step #1 - "builder": at src/libstd/sys_common/backtrace.rs:47
Step #1 - "builder": 3: std::sys_common::backtrace::print
Step #1 - "builder": at src/libstd/sys_common/backtrace.rs:36
Step #1 - "builder": 4: std::panicking::default_hook::{{closure}}
Step #1 - "builder": at src/libstd/panicking.rs:200
Step #1 - "builder": 5: std::panicking::default_hook
Step #1 - "builder": at src/libstd/panicking.rs:214
Step #1 - "builder": 6: std::panicking::rust_panic_with_hook
Step #1 - "builder": at src/libstd/panicking.rs:477
Step #1 - "builder": 7: std::panicking::begin_panic
Step #1 - "builder": 8: migration_engine::main
Step #1 - "builder": 9: std::rt::lang_start::{{closure}}
Step #1 - "builder": 10: std::rt::lang_start_internal::{{closure}}
Step #1 - "builder": at src/libstd/rt.rs:49
Step #1 - "builder": 11: std::panicking::try::do_call
Step #1 - "builder": at src/libstd/panicking.rs:296
Step #1 - "builder": 12: __rust_maybe_catch_panic
Step #1 - "builder": at src/libpanic_unwind/lib.rs:82
Step #1 - "builder": 13: std::panicking::try
Step #1 - "builder": at src/libstd/panicking.rs:275
Step #1 - "builder": 14: std::panic::catch_unwind
Step #1 - "builder": at src/libstd/panic.rs:394
Step #1 - "builder": 15: std::rt::lang_start_internal
Step #1 - "builder": at src/libstd/rt.rs:48
Step #1 - "builder": 16: main
Step #1 - "builder": 17: __libc_start_main
Step #1 - "builder": 18: _start
Step #1 - "builder": note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Step #1 - "builder": at ChildProcess.child.on (/workspace/node_modules/prisma2/build/index.js:107250:37)
Step #1 - "builder": at ChildProcess.emit (events.js:198:13)
Step #1 - "builder": at ChildProcess.EventEmitter.emit (domain.js:448:20)
Step #1 - "builder": at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
Step #1 - "builder": error Command failed with exit code 1.
Step #1 - "builder": error Command failed with exit code 1.
Step #1 - "builder":
Step #1 - "builder": ERROR error: `yarn_run_gcp_build` returned code: 1
Step #1 - "builder": INFO yarn_run_gcp_build took 5 seconds
Step #1 - "builder": INFO build process for FTL image took 44 seconds
Step #1 - "builder": INFO full build took 44 seconds
Step #1 - "builder": ERROR `yarn_run_gcp_build` had stderr output:
Step #1 - "builder": 2019-09-09T11:44:30.680Z LiftEngine:rpc SENDING RPC CALL { id: 1,
Step #1 - "builder": jsonrpc: '2.0',
Step #1 - "builder": method: 'listMigrations',
Step #1 - "builder": params:
Step #1 - "builder": { projectInfo: '',
Step #1 - "builder": sourceConfig:
Step #1 - "builder": 'datasource db {\n provider = "postgresql"\n url = "postgresql://<user>:<pass>@<ip>/<db>?schema=public"\n}\n\ngenerator photon {\n provider = "photonjs"\n}\n\ngenerator nexus_prisma {\n provider = "nexus-prisma"\n}\n\nmodel File {\n id String @id @default(cuid())\n name String\n project Project\n url String\n type String\n}\n\nmodel User {\n id String @id @default(cuid())\n authId String @unique\n email String @unique\n projects Project[]\n name String\n profileImg String?\n postVersions PostVersion[]\n faqs Faq[]\n}\n\nmodel Company {\n id String @id @default(cuid())\n name String\n projects Project[]\n}\n\nmodel Project {\n id String @id @default(cuid())\n name String\n company Company\n webUrl String?\n logo String\n logoSmall String\n postCategories PostCategory[]\n faqCategories FaqCategory[]\n posts PostVersion[]\n faqs Faq[]\n admins User[]\n files File[]\n}\n\nmodel Post {\n id String @id @default(cuid())\n draftedVersion PostVersion @relation(name: "draft")\n publishedVersion PostVersion @relation(name: "published")\n}\n\nmodel PostVersion {\n id String @id @default(cuid())\n draftedPost Post? @relation(name: "draft")\n publishedPost Post? @relation(name: "published")\n title String\n text String\n language String\n categories PostCategory[]\n author User\n published Boolean\n project Project\n ogTitle String?\n ogDescription String?\n createdAt DateTime\n updatedAt DateTime @updatedAt\n}\n\nmodel Faq {\n id String @id @default(cuid())\n question String\n answer String\n category FaqCategory?\n published Boolean\n language String\n project Project\n author User\n createdAt DateTime @default(now())\n updatedAt DateTime @updatedAt\n}\n\nmodel PostCategory {\n id String @id @default(cuid())\n name String\n project Project\n posts PostVersion[]\n}\n\nmodel FaqCategory {\n id String @id @default(cuid())\n name String\n project Project\n faqs Faq[]\n}' } }
Step #1 - "builder": 2019-09-09T11:44:33.703Z LiftEngine:stderr thread 'main' panicked at 'Box<Any>', migration-engine/core/src/main.rs:55:23
Step #1 - "builder": 2019-09-09T11:44:33.703Z LiftEngine:stderr stack backtrace:
Step #1 - "builder": 2019-09-09T11:44:33.730Z LiftEngine:stderr 0: backtrace::backtrace::libunwind::trace
Step #1 - "builder": 2019-09-09T11:44:33.730Z LiftEngine:stderr at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/libunwind.rs:88
Step #1 - "builder": 2019-09-09T11:44:33.731Z LiftEngine:stderr 1: backtrace::backtrace::trace_unsynchronized
Step #1 - "builder": 2019-09-09T11:44:33.731Z LiftEngine:stderr at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/mod.rs:66
Step #1 - "builder": 2019-09-09T11:44:33.732Z LiftEngine:stderr 2: std::sys_common::backtrace::_print
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr at src/libstd/sys_common/backtrace.rs:47
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 3: std::sys_common::backtrace::print
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr at src/libstd/sys_common/backtrace.rs:36
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 4: std::panicking::default_hook::{{closure}}
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr at src/libstd/panicking.rs:200
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 5: std::panicking::default_hook
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr at src/libstd/panicking.rs:214
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 6: std::panicking::rust_panic_with_hook
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr at src/libstd/panicking.rs:477
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 7: std::panicking::begin_panic
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 8: migration_engine::main
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 9: std::rt::lang_start::{{closure}}
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 10: std::rt::lang_start_internal::{{closure}}
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr at src/libstd/rt.rs:49
Step #1 - "builder": 2019-09-09T11:44:33.733Z LiftEngine:stderr 11: std::panicking::try::do_call
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr at src/libstd/panicking.rs:296
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr 12: __rust_maybe_catch_panic
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr at src/libpanic_unwind/lib.rs:82
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr 13: std::panicking::try
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr at src/libstd/panicking.rs:275
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr 14: std::panic::catch_unwind
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr at src/libstd/panic.rs:394
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr 15: std::rt::lang_start_internal
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr at src/libstd/rt.rs:48
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr 16: main
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr 17: __libc_start_main
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr 18: _start
Step #1 - "builder": 2019-09-09T11:44:33.734Z LiftEngine:stderr note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Step #1 - "builder": Error: Error: Error in migration engine: thread 'main' panicked at 'Box<Any>', migration-engine/core/src/main.rs:55:23
Step #1 - "builder": stack backtrace:
Step #1 - "builder": 0: backtrace::backtrace::libunwind::trace
Step #1 - "builder": at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/libunwind.rs:88
Step #1 - "builder": 1: backtrace::backtrace::trace_unsynchronized
Step #1 - "builder": at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/mod.rs:66
Step #1 - "builder": 2: std::sys_common::backtrace::_print
Step #1 - "builder": at src/libstd/sys_common/backtrace.rs:47
Step #1 - "builder": 3: std::sys_common::backtrace::print
Step #1 - "builder": at src/libstd/sys_common/backtrace.rs:36
Step #1 - "builder": 4: std::panicking::default_hook::{{closure}}
Step #1 - "builder": at src/libstd/panicking.rs:200
Step #1 - "builder": 5: std::panicking::default_hook
Step #1 - "builder": at src/libstd/panicking.rs:214
Step #1 - "builder": 6: std::panicking::rust_panic_with_hook
Step #1 - "builder": at src/libstd/panicking.rs:477
Step #1 - "builder": 7: std::panicking::begin_panic
Step #1 - "builder": 8: migration_engine::main
Step #1 - "builder": 9: std::rt::lang_start::{{closure}}
Step #1 - "builder": 10: std::rt::lang_start_internal::{{closure}}
Step #1 - "builder": at src/libstd/rt.rs:49
Step #1 - "builder": 11: std::panicking::try::do_call
Step #1 - "builder": at src/libstd/panicking.rs:296
Step #1 - "builder": 12: __rust_maybe_catch_panic
Step #1 - "builder": at src/libpanic_unwind/lib.rs:82
Step #1 - "builder": 13: std::panicking::try
Step #1 - "builder": at src/libstd/panicking.rs:275
Step #1 - "builder": 14: std::panic::catch_unwind
Step #1 - "builder": at src/libstd/panic.rs:394
Step #1 - "builder": 15: std::rt::lang_start_internal
Step #1 - "builder": at src/libstd/rt.rs:48
Step #1 - "builder": 16: main
Step #1 - "builder": 17: __libc_start_main
Step #1 - "builder": 18: _start
Step #1 - "builder": note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Step #1 - "builder": at ChildProcess.child.on (/workspace/node_modules/prisma2/build/index.js:107250:37)
Step #1 - "builder": at ChildProcess.emit (events.js:198:13)
Step #1 - "builder": at ChildProcess.EventEmitter.emit (domain.js:448:20)
Step #1 - "builder": at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
Step #1 - "builder": error Command failed with exit code 1.
Step #1 - "builder": error Command failed with exit code 1.
Step #1 - "builder":
Step #1 - "builder": error: `yarn_run_gcp_build` returned code: 1
Step #1 - "builder": Traceback (most recent call last):
Step #1 - "builder": File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
Step #1 - "builder": "__main__", fname, loader, pkg_name)
Step #1 - "builder": File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
Step #1 - "builder": exec code in run_globals
Step #1 - "builder": File "/usr/local/bin/ftl.par/__main__.py", line 65, in <module>
Step #1 - "builder": File "/usr/local/bin/ftl.par/__main__.py", line 57, in main
Step #1 - "builder": File "/usr/local/bin/ftl.par/__main__/ftl/common/ftl_error.py", line 58, in UserErrorHandler
Step #1 - "builder": IOError: [Errno 2] No such file or directory: '""/output'
Finished Step #1 - "builder"
ERROR
ERROR: build step 1 "gcr.io/gae-runtimes/nodejs10_app_builder:nodejs10_20190602_10_16_0_RC00" failed: exit status 1
kinfeature binary deployment-platform
Source
picture malekjaroslav
👍1

Most helpful comment

This worked for me: postgres://user:password@localhost/mydb?host=/cloudsql/myproject:us-east4:test

picture Sytten on 26 Apr 2020
👍3 🚀2 ❤1

All 42 comments

The error message looks very similar to https://github.com/prisma/prisma2/issues/381 which got recently closed.

Can you please tell me where the database is hosted?

pantharshit00 picture pantharshit00 on 9 Sep 2019

Also, please make sure you are on the latest version of prisma2

pantharshit00 picture pantharshit00 on 9 Sep 2019

This database is hosted on Google Cloud (Cloud SQL). I managed to make it work by connecting to a public IP. Sockets did not work, now trying to use SSL, but when I add it to the db connection string (sslcert, sslkey) it does not accept it. I am using the latest version of Prisma2.

malekjaroslav picture malekjaroslav on 9 Sep 2019

Yes, right now we don't support those options.

Can you open an issue(ie feature request) for that 🙏

pantharshit00 picture pantharshit00 on 9 Sep 2019

@pantharshit00 What options do you mean? Sockets or SSL?

malekjaroslav picture malekjaroslav on 9 Sep 2019

We don't support sslcert and sslkey options right now.

I will need to confirm socket support

pantharshit00 picture pantharshit00 on 9 Sep 2019

But sockets should work, is that correct? If so, what is the recommended connection string? I am unable to make it work

malekjaroslav picture malekjaroslav on 9 Sep 2019

There is one more weird thing happening. Prisma init via sockets works, but migration (prisma2 lift up / prisma2 dev) fails

Screenshot from 2019-09-09 17-27-36

Screenshot from 2019-09-09 17-38-24

malekjaroslav picture malekjaroslav on 9 Sep 2019

@pantharshit00 Do you have some updates on this?

malekjaroslav picture malekjaroslav on 10 Sep 2019

I think sockets are not supported in the Rust implementation right now.

The CLI uses the nodejs driver to do introspection right now.

pantharshit00 picture pantharshit00 on 13 Sep 2019

@pantharshit00 But in the prism2 init flow it looks like is connecting through socket. Is this going to be solved somehow? Sockets are widely used in Google Cloud and other Cloud providers for connecting to databases.

malekjaroslav picture malekjaroslav on 13 Sep 2019

In the docs of Prisma 2 it says to use any supported connection string and Rust's Postgresql implementation supports it https://docs.rs/postgres/0.15.2/postgres/struct.Connection.html#impl

Without sockets and SSL support, there is basically no way to communicate securely between the backend and the database...

malekjaroslav picture malekjaroslav on 13 Sep 2019
👍2

I need to confirm this with the backend team. I don't work on the Rust part of the codebase, it might be a specific issue.

I will try to make a concrete reproduction for this.

pantharshit00 picture pantharshit00 on 13 Sep 2019
👍1

error when start after init
`` ➜ prisma prisma2 lift save Error: Error in migration engine: thread 'tokio-runtime-worker-1' panicked at 'get data_type', src/libcore/option.rs:1034:5 stack backtrace: 0: std::panicking::default_hook::{{closure}} 1: std::panicking::default_hook 2: std::panicking::rust_panic_with_hook 3: std::panicking::continue_panic_fmt 4: rust_begin_unwind 5: core::panicking::panic_fmt 6: core::option::expect_failed 7: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &mut F>::call_once 8: <alloc::vec::Vec<T> as alloc::vec::SpecExtend<T,I>>::from_iter 9: sql_schema_describer::mysql::SqlSchemaDescriber::get_table 10: <core::iter::adapters::Map<I,F> as core::iter::traits::iterator::Iterator>::fold 11: <alloc::vec::Vec<T> as alloc::vec::SpecExtend<T,I>>::from_iter 12: <sql_schema_describer::mysql::SqlSchemaDescriber as sql_schema_describer::SqlSchemaDescriberBackend>::describe 13: <sql_migration_connector::sql_database_migration_inferrer::SqlDatabaseMigrationInferrer as migration_connector::database_migration_inferrer::DatabaseMigrationInferrer<sql_migration_connector::sql_migration::SqlMigration>>::infer 14: <migration_core::api::MigrationApi<C,D> as migration_core::api::GenericApi>::infer_migration_steps 15: migration_core::api::rpc::RpcApi::create_sync_handler 16: tokio_executor::enter::exit 17: tokio_threadpool::blocking::blocking 18: <futures::future::lazy::Lazy<F,R> as futures::future::Future>::poll 19: futures::future::chain::Chain<A,B,C>::poll 20: <futures::future::then::Then<A,B,F> as futures::future::Future>::poll 21: <futures::future::lazy::Lazy<F,R> as futures::future::Future>::poll 22: futures::future::chain::Chain<A,B,C>::poll 23: <futures::future::then::Then<A,B,F> as futures::future::Future>::poll 24: <futures::future::map::Map<A,F> as futures::future::Future>::poll 25: <futures::future::either::Either<A,B> as futures::future::Future>::poll 26: <futures::future::map::Map<A,F> as futures::future::Future>::poll 27: <futures::future::map_err::MapErr<A,F> as futures::future::Future>::poll 28: <futures::stream::and_then::AndThen<S,F,U> as futures::stream::Stream>::poll 29: <futures::stream::forward::Forward<T,U> as futures::future::Future>::poll 30: <futures::future::map::Map<A,F> as futures::future::Future>::poll 31: <futures::future::map_err::MapErr<A,F> as futures::future::Future>::poll 32: futures::task_impl::std::set 33: std::panicking::try::do_call 34: __rust_maybe_catch_panic 35: tokio_threadpool::task::Task::run 36: tokio_threadpool::worker::Worker::run_task 37: tokio_threadpool::worker::Worker::run 38: std::thread::local::LocalKey<T>::with 39: std::thread::local::LocalKey<T>::with 40: tokio_reactor::with_default 41: tokio::runtime::threadpool::builder::Builder::build::{{closure}} 42: std::thread::local::LocalKey<T>::with 43: std::thread::local::LocalKey<T>::with note: Some details are omitted, run withRUST_BACKTRACE=fullfor a verbose backtrace.

AhmedElywa picture AhmedElywa on 14 Sep 2019

@AhmedElywa are you running prisma prisma2 lift save or prisma2 lift save in prisma folder?

malekjaroslav picture malekjaroslav on 16 Sep 2019

@malekjaroslav
the folder name is prisma command prisma2 lift save

AhmedElywa picture AhmedElywa on 16 Sep 2019
👍1

@pantharshit00 Hi, do you have some updates regarding this issue?

malekjaroslav picture malekjaroslav on 18 Sep 2019

We will write a spec for this so that we integrate this in the schema.

We are on it, might take some time as it is a schema change.

pantharshit00 picture pantharshit00 on 18 Sep 2019
🚀3

Great, thanks!

malekjaroslav picture malekjaroslav on 18 Sep 2019

@pantharshit00 or @malekjaroslav Can you please summarize what this issue is about? Is it still reproducible

Is it just about SSL support or something broader?

divyenduz picture divyenduz on 30 Sep 2019

@pantharshit00 or @malekjaroslav Can you please summarize what this issue is about? Is it still reproducible

Is it just about SSL support or something broader?

It is about Unix socket support for connection to the DB. For example when using Google Cloud, the easiest and safest way to connect is through a socket connection. But we were unable to connect via a unix socket.

As SSL is not supported yet, there is no secure way to connect to the DB.

malekjaroslav picture malekjaroslav on 30 Sep 2019
👍2

@pantharshit00 please help me push this one forward :)

divyenduz picture divyenduz on 1 Oct 2019

@malekjaroslav do you mind sharing an example of what gets populated in CLOUD_SQL_CONNECTION_NAME?

I'm looking at the following example from: https://cloud.google.com/sql/docs/mysql/connect-app-engine

let pool;
const createPool = async () => {
  pool = await mysql.createPool({
    user: process.env.DB_USER, // e.g. 'my-db-user'
    password: process.env.DB_PASS, // e.g. 'my-db-password'
    database: process.env.DB_NAME, // e.g. 'my-database'
    // If connecting via unix domain socket, specify the path
    socketPath: `/cloudsql/${process.env.CLOUD_SQL_CONNECTION_NAME}`,
    // If connecting via TCP, enter the IP and port instead
    // host: 'localhost',
    // port: 3306,

    //...
  });
};
createPool();

Updated Proposal: https://github.com/prisma/prisma2/issues/525#issuecomment-549445413

Background:

There seems to be a couple different ways to support this:

Proposed Solution:

Ideally we can unify our unix domain socket syntax, and let the connectors (providers) properly format the connection strings:

I'd prefer the following:

datasource pg {
  provider = "postgresql"
  url = "postgresql:///var/lib/postgres/database.sock"
}

datasource ms {
  provider = "mysql"
  url = "mysql:///var/lib/mysql/database.sock"
}

datasource ms {
  provider = "mysql"
  url = `mysql:///cloudsql/${env("CLOUD_SQL_CONNECTION_NAME")}`
}

Open Questions

Will there be an issue with the ///?

The postgres documentation states:

The host component is interpreted as described for the parameter host. In particular, a Unix-domain socket connection is chosen if the host part is either empty or starts with a slash, otherwise a TCP/IP connection is initiated.

Note, however, that the slash is a reserved character in the hierarchical part of the URI. So, to specify a non-standard Unix-domain socket directory, either omit the host specification in the URI and specify the host as a parameter, or percent-encode the path in the host component of the URI:

postgresql:///dbname?host=/var/lib/postgresql
postgresql://%2Fvar%2Flib%2Fpostgresql/dbname

I'm not following why %2F is necessary, can someone explain this to me?

matthewmueller picture matthewmueller on 21 Oct 2019

@matthewmueller : If i understand your suggestion correctly that would mean a deviation from the strategy we chose for our datasource URLs. From the spec:

If a given database already defines a connection URL standard the datasource URL is based on this standard. Additional parameters required by the Prisma tool suite are added on top of the existing standard.

Your suggestion would not follow this approach for MySQL, right? (see https://dev.mysql.com/doc/refman/8.0/en/connecting-using-uri-or-key-value-pairs.html)
If we want to go down that route we should make have a framework for when we go with a custom format and when not. Users need to understand when they can consult the docs of the given database and when not.

mavilein picture mavilein on 21 Oct 2019

Thanks for your feedback @mavilein.

If I'm understanding your suggestion properly, in our example case of Unix Domain Sockets for MySQL, our connection string should look like this:

datasource ms {
  provider = "mysql"
  url = "root@localhost?socket=(/tmp/mysql.sock)"
}

Source: https://dev.mysql.com/doc/refman/8.0/en/connecting-using-uri-or-key-value-pairs.html

Please correct if I'm misunderstanding ☺️

I'm operating under the assumption that the connection string format is already a wild west, so we're taming it by providing a consistent syntax and managing the variability on our end.

Using the Unix Domain Sockets as an example and Google Cloud SQL as the provider, the URL wouldn't work as is. You would need to connect with this connection string:

mysql+pymysql://<db_user>:<db_pass>@/<db_name>?unix_socket=/cloudsql/<cloud_sql_instance_name>

Source: https://cloud.google.com/sql/docs/mysql/connect-app-engine

I definitely see your point about not being to use the SQL documentation to figure out the connection string, but we've already kind of invalidated that resource with PSL.

That being said, I'd be on board for a system that follows the language's connection string and also reformats for these popular cloud platforms, it just seems like more engineering work.

matthewmueller picture matthewmueller on 22 Oct 2019

Please correct if I'm misunderstanding ☺️

You understood it correctly.

I'm operating under the assumption that the connection string format is already a wild west, so we're taming it by providing a consistent syntax and managing the variability on our end.

That is an approach that we explicitly moved away from.

Using the Unix Domain Sockets as an example and Google Cloud SQL as the provider, the URL wouldn't work as is. You would need to connect with this connection string:

I don't think this is true. This example looks like as if it is applying to the Python driver only.

mavilein picture mavilein on 28 Oct 2019

@malekjaroslav what is your current workaround?
I'm in the same configuration: Cloud SQL + Cloud Run.

I can only make it working through public IP, no SSL and accepting requests from 0.0.0.0/0, that is... bad.

lvauvillier picture lvauvillier on 3 Nov 2019

@lvauvillier, well this is my workaround as well, but SSL should be supported by now.

More details here: https://github.com/prisma/prisma2/issues/568#issuecomment-539043245

malekjaroslav picture malekjaroslav on 4 Nov 2019

Update to the proposal above.

⚠ This isn't implemented yet. The first step is to define the syntax

After speaking with @mavilein, we'll use the connection string that the database provides, rather than try to come up with a universal URL string syntax. 

datasource pg {
  provider = "postgresql"
  url = "postgresql:///dbname?host=/var/lib/postgresql"
}

datasource ms {
  provider = "mysql"
  url = "root@localhost?socket=(/tmp/mysql.sock)"
}

datasource mongo {
  provider = "mongo"
  url = "mongodb://%2Ftmp%2Fmongodb-27017.sock"
}

So to get this working on GCE, it'd look something like this:

datasource ms {
  provider = "mysql"
  url = "${env("DB_USER")}:${env("DB_PASS")}@localhost/${env("DB_NAME")}?socket=(/cloudsql/${env("CLOUD_SQL_CONNECTION_NAME")})"
}
matthewmueller picture matthewmueller on 4 Nov 2019
👍3

I was reading the source code and it could be quite easy to implement:

Here we get the host from the Uri. But in a case of a socket the uri would look like postgres://user:pass@/var/run/asdfg.sock, and host_str returns localhost which is wrong.

The postgres crate states:

host - The host to connect to. On Unix platforms, if the host starts with a / character it is treated as the path to the directory containing Unix domain sockets. Otherwise, it is treated as a hostname. Multiple hosts can be specified, separated by commas. Each host will be tried in turn when connecting. Required if connecting with the connect method.

Which means if we give it the path it should use a socket instead of TCP.

For mysql

https://docs.rs/mysql_async/0.21.0-alpha.4/mysql_async/struct.OptsBuilder.html#method.socket

We can do the following: If Url cannot see a host in the uri, we try to look for path which should give us the socket.

How this works now is that you do not need to define any hostname and it will be automatically set to localhost. I'd like to enforce the hostname and if you don't write it, we'll try to get the path and use a socket.

pimeys picture pimeys on 8 Nov 2019

Implementation, please read the tests:

https://github.com/prisma/quaint/pull/27/files#diff-a34a59925e5160fbec39a56d4fa6b3c6R362-R368
https://github.com/prisma/quaint/pull/27/files#diff-0a8a3c3d21eb76727df625b9399a69c4R491-R504

pimeys picture pimeys on 8 Nov 2019

Ok, this is now merged to the engines master, some notes:

  • MySQL connection strings look like this: mysql://user:password@localhost/database?socket=/var/run/mysqld/mysqld.sock, where you must have the host written but if socket parameter is set, it'll try to connect using a unix socket. This needs to point to the correct sock file.
  • PostgreSQL is a bit different: postgresql://user:password@/database?host=/var/run/postgresql/. Here if the host parameter is set, we'll try to connect using it instead of the host part of the URI. This way we can point it to a socket, but compared to MySQL we pass the directory containing the file .s.PGSQL.5432=. If host or the host param is not set, we connect to localhost.

Works only in the Rust core. I tried to run prisma2 lift save using the uris above and got an error Invalid URL: postgresql://root:prisma@/prisma?host=/var/run/postgresql/. Seems like our TypeScript code is doing some parsing and needs to be checked out by @timsuchanek.

pimeys picture pimeys on 11 Nov 2019
👍5

Thanks a lot for reporting 🙏
This issue is fixed in the latest alpha version of prisma2.
You can try it out with npm i -g prisma2@alpha.

In case it’s not fixed for you - please let us know and we’ll reopen this issue!

timsuchanek picture timsuchanek on 11 Dec 2019
👍2

@timsuchanek I'm not sure if this has fixed it for me. I'm using a typical docker compose for postgres to repro locally, that listens on unix socket: /var/run/postgresql/.s.PGSQL.5432. 

version: "3.7"

services:
  db:
    image: postgres:12-alpine
    restart: always
    environment:
      POSTGRES_PASSWORD: mypassword
    volumes:
      - ./postgres-data:/var/lib/postgresql/data
    ports:
      - 5432:5432`

I am on alpha version: 2.0.0-alpha.538. I believe this is the last alpha version before the breaking changes introduced. I would upgrade but I also use nexus-prisma, so I'm waiting on the changes coming in there.

I'm in a similar situation where I would like to use this with Google Cloud Run and Cloud SQL in GCP.

When I try to make a gql query I get this error: 

Invalid `photon.()` invocation in
/home/dana/Projects/flags/flag-core/flag-api/node_modules/nexus-prisma/src/builder.ts:238:18



Error parsing connection string: empty host
    at PhotonFetcher.request (/home/dana/Projects/flags/flag-core/flag-api/node_modules/@prisma/photon/index.js:62:23)

With the connection string (replace dbname)

POSTGRESQL_URL=postgresql://postgres:mypassword@/<dbname>?host=/var/run/postgresql/

I am able to connect to this just fine using other connection methods with a connection string.

daaasbu picture daaasbu on 23 Jan 2020
👍1

This is at least fixed in the backend side, here's the test: https://github.com/prisma/quaint/blob/master/src/connector/postgres.rs#L421-L426

Is the javascript client using some weird URL parser that doesn't understand the socket without a host?

pimeys picture pimeys on 23 Jan 2020

I remember @timsuchanek fixing it, but is the alpha.538 too old to contain the fix?

pimeys picture pimeys on 23 Jan 2020

@daaasbu Can you please open a new issue about this? This particular issue tracks the progress of implementation of this feature which is done now. If you encounter any bugs please open a new issue. Also, please test it out with the latest version once :pray:

pantharshit00 picture pantharshit00 on 23 Jan 2020
👍2

This is a pretty important one - this is the documented path for most of GCP and especially important for any serverless deployment, as documented here GCP cloud run <==> SQL connection guide Has anybody managed to figure out a working connection string?

I'm currently trying:
DATABASE_URL="postgresql://USER:[email protected]/ying-staging?host=/cloudsql/DATABASE_NAME/.s.PGSQL.5432"

nargetdev picture nargetdev on 25 Apr 2020

This worked for me: postgres://user:password@localhost/mydb?host=/cloudsql/myproject:us-east4:test

Sytten picture Sytten on 26 Apr 2020
👍3 🚀2 ❤1

@nargetdev If I remember correctly, you should not append .s.PGSQL.5432 to the socket string. That will automatically get appended by the driver

Use postgresql://USER:[email protected]/ying-staging?host=/cloudsql/DATABASE_NAME

pantharshit00 picture pantharshit00 on 27 Apr 2020
👍1

Thank you very much @Sytten, that worked for me as well!!

papb picture papb on 17 Jul 2020

@Sytten solution works like charm, it is very hard to find a solution for this case, I almost gave up

tylim88 picture tylim88 on 19 Nov 2020
👍1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

nikolasburk picture nikolasburk  ·  57Comments
timsuchanek picture timsuchanek  ·  26Comments
marktani picture marktani  ·  31Comments
iherger picture iherger  ·  26Comments
malekjaroslav picture malekjaroslav  ·  53Comments