Node-oracledb: AWS Lambda + node-oracledb

[cleitonjar]

Hello,
I'm testing the AWS Lambda (Amazon) with nodejs + node-oracle, but gave a error:
libclntsh.so.12.1: cannot open shared object file: No such file or directory.
AWS lambda with java 8 + jdbc is perfect, because doesn't use the libs.so.
Someone know how to resolve this problem?
I opened here this problem for information.

[cjbj]

In my understanding you would need a statically linked node-oracledb, and there isn't a static Instant Client available.

You might consider https://cloud.oracle.com/acc

[cleitonjar]

Hi,
I found the problem in my aws lambda.
1-) Libraries should be at the root in the /lib. (The aws lambda found the libraries).
2-) Added the library libaio.so.1.0.1.

Now, I have other problem:
I installed in nodejs 4.3 (npm install [email protected] and 1.9.3).
I installed in nodejs 5.10/11 (npm install [email protected] and 1.9.3).
What is the version correct of the oracledb and nodejs?
{
"errorMessage": "Module version mismatch. Expected 46, got 47.",
"errorType": "Error",
"stackTrace": [
"Object.Module._extensions..node (module.js:434:18)",
"Module.load (module.js:343:32)",
"Function.Module._load (module.js:300:12)",
"Module.require (module.js:353:17)",
"require (internal/module.js:12:17)",
"Object. (/var/task/node_modules/oracledb/lib/oracledb.js:32:19)"....

[sagiegurari]

From quick glance, looks like npm install ran with nodejs 5, and when you run your app, you are running with nodejs 4.
nodejs 4 can't load native compiled code of node 5 because of different v8.
use same nodejs for both npm install and node run.

[cjbj]

Wow, I thought it was locked down. Anyway, I'm sure people would like to see the complete list of steps you followed.

(PS. Use the latest node-oracledb, which currently is 1.9.3. And follow @sagiegurari's advice about node versions.)

[cleitonjar]

@sagiegurari Yes, You are correct!
@cjbj Yes, oracledb 1.9.3 with node v4.3.0.

The aws lambda uses linux (aim x64) and node-v4.3.
Step by step local machine:
1-) download node-v4.3.0-linux-x64.tar.gz and configure your env PATH inside ~/.profile.
2-) download oracle instant client 12 (basic and sdk).
3-) unzip files inside your project with folder name 'lib' (the aws lambda to read implicit).
4-) add libaio.so.1.0.1 of linux to 'lib' and create symbolic link libaio.so.1.
5-) npm install oracledb.
6-) success

WORKING!!!

Thank you guys!

[cartolari]

Basically you need to set LD_LIBRARY_PATH environment variable, but when the lambda runs your code it's too late to do it (unless you start a new process with it set).
Apparently you just need to put the Oracle Instant Client in the bundled zip you send to AWS Lambda in a folder called lib, according to this post in AWS Lambda Forum, it should work.

[nalbion]

I'm getting a different error from Lambda when it tries to import('oracledb'):

 /var/task/node_modules/oracledb/build/Release/oracledb.node: ELF file's phentsize not the expected size
 /var/task/node_modules/oracledb/lib/oracledb.js:32:19

[tomdavidson]

@cleitonjar thanks for opening this issue.

I copied my system's libaio.so.1.0.1 to the source bundle but I am also missing libstdc++ - even after added it to lib/ as I did libaio:

"/usr/local/lib64/node-v4.3.x/lib/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /var/task/node_modules/oracledb/build/Release/oracledb.node)"

curious why it would be it would be different if we all using the same amis Im looking forward to reading other's experiences.

[nalbion]

@tomdavidson try this: https://github.com/nalbion/node-oracledb-for-lambda

[tomdavidson]

@nalbion Outstanding! thank you.
I also want to enforce encryption from the client side ( #471 ). With node-oracledb I can just config the instantclient with a .ora file. Do you guidance specific to your lambda fork or more generalized for node-oracledb?

[nalbion]

@tomdavidson if it works with this module it will work with mine, I've just provided the precompiled libs

[cjbj]

I'd be very interested if someone blogged all about this. @nalbion @tomdavidson ??

[a-panickar]

I tried all these suggestions but I would still get the following ORACLE errors when connecting to a database inside the VPC using AWS Lambda:

ORA-24408 could not generate unique server group name
ORA-24454: client host name is not set

It appears, there's a problem with the name resolution and the work around is to use HOSTALIASES since /etc/hosts cant be updated with lambda.

The following post has details:

https://stackoverflow.com/questions/39201869/aws-python-lambda-with-oracle-oid-generation-failed

[cjbj]

@abhilash80 Thanks for pointing out the use of HOSTALIASES.

You must be using 12.2 client - that's the first report of the new 'clearer' error message I've seen; it took me a while to champion that change :)

I will continue to have discussions with the Oracle Net team about GUID generation improvements, but host information is currently required.

[a-panickar]

Yes I am using 12.2 client, I tried multiple options to see what works for resolving this issue, including trying node v4 and oracle client 12.1 and I got different errors but all seem related:

ORA-24408 could not generate unique server group name
ORA-24454: client host name is not set
ORA-21561: OID generation failed

[cjbj]

Yes they are related. The error number & text changed in 12.2.

[varmeEqualsthis]

adding HOSTALIASES fixed the problem? If yes, could you help with the details on adding "HOSTALIASES" to lambda function?

[a-panickar]

@varmeEqualsthis

Sure...

Inside your lambda function before you create the connection pool have a line of code which reads the hostname dynamically and writes it to the HOSTALIASES file, something like below for a node.js app:

const os = require('os');
const fs = require('fs-extra')

fs.writeFile("/tmp/HOSTALIASES", os.hostname() + ' localhost \r\n')
.then(() => {
...Create connection pool here...
})

[varmeEqualsthis]

Thank you @abhilash80 . I was able to get it working as explained here and it worked. Appreciate your help!

[emadOmara]

Hi, I have a question on the size of the client , as it will make the package so big.
what is the best practise to overcome the size issue ?

we are using :

AWS Lambda
Node JS
we are tring : node-oracledb-for-lambda

[cjbj]

@emadOmara use Instant Client Basic Light, and remove all the libraries you don't need. You will need to keep:

libclntsh.so.12.1
libclntshcore.so.12.1
libipc1.so
libmql1.so
libnnz12.so
libociicus.so
libons.so

[emadOmara]

@cjbj Thanks for you reply , when I tried it on my windows machine I could connect but when I put the package on Lambda it gives erros , is this because the npm install for the oracledb ran on windows machine and not linux ? if so will it works if I use the command on any linux distribution ?

[cjbj]

@emadOmara You need to run 'npm install oracledb' on each platform - there is a different node-oracledb binary for each. Prebuilt binaries are only available for Linux x64, Windows x64 and macOS. Source code can be compiled for others.

Instant Client needs the appropriate system libraries. Commonly libaio (sometimes known as libaio1) needs to be explicitly installed.

[emadOmara]

@cjbj Thanks a million , it's working for us now , one last question what if we have multiple functions if each one will connect to the DB this will be a big size can we have the lib with the instant client some where and all lambda functions share it ? or is it better to have one lambda function as if it's a connection factory and all other functions to call it ?

[cjbj]

@emadOmara that's a good question for a Lambda expert. Let us know what you find.

[nalbion]

@emadOmara great idea! In my apps I don't use connection pooling, but thankfully it does not get too much traffic.

What you could do (this is all theoretical, untested) is to have a DAO Lambda function and call it (either synchronously or asynchronously) from your other Lambda functions. I noticed the other day that you can configure the maximum concurrency of you DAO Lambda function. I'm not sure how performance would be affected, but it would probably help with scaling

[GaryGSC]

@emadOmara I found this blog post to be pretty useful when learning how to reuse connections (as much as possible) in a Lambda. It doesn't really address your architecture question though.

[cjbj]

I'm not a Lambda user, but I can see how DRCP may play a role. DRCP is useful when you have multiple mid-tier processes accessing the database.

[azertyDev]

@cleitonjar
Can I find out how you connected the Oracle together with the Lambda through the code !?

[emadOmara]

@azertyDev
For us , we created a generic Lambda function that takes query and parameters and execute it then return results to the caller using : "oracledb-for-lambda"

[krishna-thotakura]

@cleitonjar thank you ! this approach works with lambda node 10.x runtime also.

Note: when creating the zip file, use zip --symlinks on top of the usual chmod 755 and 644 checks on the contents of the zip.

[julian-saunders]

@varmeEqualsthis

Sure...

Inside your lambda function before you create the connection pool have a line of code which reads the hostname dynamically and writes it to the HOSTALIASES file, something like below for a node.js app:

const os = require('os');
const fs = require('fs-extra')

fs.writeFile("/tmp/HOSTALIASES", os.hostname() + ' localhost \r\n')
.then(() => {
...Create connection pool here...
})

This worked if we also added the following Lambda environment variable:

HOSTALIASES = /tmp/HOSTALIASES

[eswarijayakumar]

Try this https://geekyturtles.com/blogs/connect-oracle-db-from-lambda-using-node-js-select-and-insert-operations/