Neon: Possibility to run neon on Google cloud functions

[opensourcegeek]

Just wondering if it's possible to write some code in rust lang and use it via nodejs in Google Cloud Functions It would be great if it is possible, as costing is based on milliseconds of usage for example.

[jedireza]

According to the Cloud Functions page:

Cloud Functions are written in JavaScript and execute in a standard Node.js runtime environment. We don’t assume anything proprietary all the way down to the operating system, which means your functions will just work—including native libraries you bring to the platform.

Via: https://cloud.google.com/functions/

Currently Rust and Cargo are dependencies for Neon modules to be built. Those would probably have to be present in the Google Cloud Function environment for Neon modules to build.

---

Out of curiosity I looked up how AWS Lambda handles native node modules:

Native modules are similarly installed and deployed, but you’ll need to build them against the Amazon Linux libraries. You’ll need to either ensure that the libraries and their transitive dependencies are statically compiled or use rpath-style linking

Via: https://aws.amazon.com/blogs/compute/nodejs-packages-in-lambda/

So with AWS Lambda, you're pre-compiling for their environment and uploading a zip file.

[opensourcegeek]

Interesting, I'd have to look more into it. We use GCP already, so curious to see what I could do with cloud functions. Some of the data transformations we do with data takes slightly more than a minute, which runs approx. every minute and there's few hundred thousands of them in a month. It will be interesting to see how much compute time is required when I use Rust, especially if I could cut down cost with Rust it will help it's adoption in other areas at my work.

[softprops]

+1

[softprops]

Any motion on this front?

[muditsaurabh]

Any update?

[tarcieri]

I don't suppose anyone has gotten this to work with or without Neo? This is about all I can find in regard to Rust + Cloud Functions

https://github.com/Byron/google-apis-rs/tree/master/gen/cloudfunctions1

I guess the next question:

We don’t assume anything proprietary all the way down to the operating system, which means your functions will just work—including native libraries you bring to the platform.

What kinds of native libraries can you "bring to the platform"? There's not a whole lot of information here:

https://cloud.google.com/functions/docs/deploying/
https://cloud.google.com/functions/docs/deploying/filesystem

There are options for "ZIP upload" and "ZIP from Cloud Storage" so it seems at least you can provide a package containing several files.

The first link says:

When deploying, Cloud Functions will look for a file named index.js or function.js. If you've provided a package.json file that contains a "main" entry, then Cloud Functions will look for the specified file instead.

So if nothing else it seems to support package.json files. Would it make sense to do something like create an npm module containing rustup that installs cargo?

[opensourcegeek]

I haven't checked out what's possible in production GCP functions yet, however trying to run it in emulator (https://github.com/GoogleCloudPlatform/cloud-functions-emulator) works.

sudo functions call hello
Warning: You're using Node.js v8.9.3 but Google Cloud Functions only supports v6.11.5.
ExecutionId: f019z600-dzxf-48ea-ad06-6e0734189f1e
Result: hello node

I just used neon hello world example, and created a separate index.js at project root and hooked it to lib/index.js that neon creates.

Also, the author of this article has got it working on AWS lambda (https://dev.to/kayis/rust-on-aws-lambda-with-neon--cloud9--4el7) too.

[kjvalencik]

Take it with a grain of salt, because I haven't actually tested it, but you should be able to build against the Google Cloud nodejs docker image (gcr.io/google-appengine/nodejs). You will likely need ensure that the compiled binary is included in the zip that is submitted and remove the neon build from postinstall in the package.json.

[neoeinstein]

FYI, I have an interest in playing around with this, so I'm going to take some time and see if I can get this working.

[kanekv]

I've tried to compile a native module and run it on google cloud functions - and I'm getting this error:

Detailed stack trace: Error: /user_code/index.node: undefined symbol: _ZdlPvm

@neoeinstein did you manage to get around it?