Krypto-trading-bot: Error when trying to run bot

Created on 12 Jul 2017 · 13Comments · Source: ctubio/Krypto-trading-bot

I am trying to run the bot on CentOs 7 but I am getting the following error on starting the bot:

Error: libquickfix.so.16: cannot open shared object file: No such file or directory
at Object.Module._extensions..node (module.js:598:18)
at Module.load (module.js:503:32)
at tryModuleLoad (module.js:466:12)
at Function.Module._load (module.js:458:3)
at Module.require (module.js:513:17)
at require (internal/module.js:11:18)
at Object. (/home/user/K/node_modules/node-quickfix-wrap/index.js:1:78)
at Module._compile (module.js:569:30)
at Object.Module._extensions..js (module.js:580:10)
at Module.load (module.js:503:32)

I have installed "npm i node-quickfix-wrap" but still getting the error.

question

Source

dexter369

All 13 comments

i bet the error message does not lie

can you verify?

 $ ls -la /usr/local/lib/libquickfix.so*
lrwxrwxrwx 1 root root  21 Jul  9 23:55 /usr/local/lib/libquickfix.so -> libquickfix.so.16.0.1*
lrwxrwxrwx 1 root root  21 Jul  9 23:55 /usr/local/lib/libquickfix.so.16 -> libquickfix.so.16.0.1*
-rwxr-xr-x 1 root root 17M Jul  9 23:55 /usr/local/lib/libquickfix.so.16.0.1*

quickfix is supposed to be installed on install, i bet² the interesting error is in the installation log (you pasted the execution log); feel free to paste the installation log

ctubio on 12 Jul 2017

$ ls -la /usr/local/lib/libquickfix.so*
lrwxrwxrwx 1 root root 21 Jul 11 22:57 /usr/local/lib/libquickfix.so -> libquickfix.so.16.0.1
lrwxrwxrwx 1 root root 21 Jul 11 22:57 /usr/local/lib/libquickfix.so.16 -> libquickfix.so.16.0.1
-rwxr-xr-x 1 root root 21187432 Jul 11 22:57 /usr/local/lib/libquickfix.so.16.0.1

So it seems to be there.
Will paste installation log in a bit.

dexter369 on 12 Jul 2017

Installation:


[user@ubuntu-2gb-sfo2-04 K5]$ npm start

> [email protected] start /home/user/K5
> npm run test-compile && forever start --uid $npm_package_config_config -a -l /dev/null K.js $npm_package_config_config && exec npm run test-stunnel


> [email protected] test-compile /home/user/K5
> test -d app || npm $(test -d node_modules && echo run post)install


> [email protected] preinstall /home/user/K5
> sudo rm -rf /usr/local/etc/stunnel && sudo mkdir -p /usr/local/etc/stunnel/ && exec npm run install-packages

[sudo] password for user:

> [email protected] install-packages /home/user/K5
> test -n "$(command -v apt-get)" && sudo apt-get -y install g++ build-essential automake autoconf libtool libxml2 libxml2-dev zlib1g-dev libsqlite3-dev libssl-dev openssl stunnel python wget gzip || (test -n "$(command -v yum)" && sudo yum -y install gcc-c++ automake autoconf libtool libxml2 libxml2-devel zlib-devel sqlite-devel openssl openssl-devel zlib-devel stunnel python wget gzip) || (test -n "$(command -v brew)" && (xcode-select --install || :) && (brew install automake autoconf libxml2 sqlite openssl zlib libuv stunnel python wget gzip || brew upgrade || :)) && (sudo mkdir -p /data/db/ && sudo chown `id -u` /data/db) && make && openssl s_client -showcerts -connect fix.gdax.com:4198 < /dev/null | openssl x509 -outform PEM > fix.gdax.com.pem && sudo mv fix.gdax.com.pem /usr/local/etc/stunnel/ && exec npm run test-stunnel

Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.sjc02.svwh.net
 * extras: mirror.san.fastserv.com
 * updates: repos-lax.psychz.net
Package gcc-c++-4.8.5-11.el7.x86_64 already installed and latest version
Package automake-1.13.4-3.el7.noarch already installed and latest version
Package autoconf-2.69-11.el7.noarch already installed and latest version
Package libtool-2.4.2-22.el7_3.x86_64 already installed and latest version
Package libxml2-2.9.1-6.el7_2.3.x86_64 already installed and latest version
Package libxml2-devel-2.9.1-6.el7_2.3.x86_64 already installed and latest version
Package zlib-devel-1.2.7-17.el7.x86_64 already installed and latest version
Package sqlite-devel-3.7.17-8.el7.x86_64 already installed and latest version
Package 1:openssl-1.0.1e-60.el7_3.1.x86_64 already installed and latest version
Package 1:openssl-devel-1.0.1e-60.el7_3.1.x86_64 already installed and latest version
Package zlib-devel-1.2.7-17.el7.x86_64 already installed and latest version
Package stunnel-4.56-6.el7.x86_64 already installed and latest version
Package python-2.7.5-48.el7.x86_64 already installed and latest version
Package wget-1.14-13.el7.x86_64 already installed and latest version
Package gzip-1.5-8.el7.x86_64 already installed and latest version
Nothing to do
mkdir -p build app/server/lib
make quickfix
make[1]: Entering directory `/home/user/K5'
(test -f /usr/local/lib/libquickfix.so || test -f /usr/local/lib/libquickfix.dylib) || ( \
        curl -L https://github.com/quickfix/quickfix/archive/v.1.14.4.tar.gz | tar xz -C build  \
        && cd build/quickfix-v.1.14.4 && ./bootstrap && ./configure && make                     \
        && sudo make install && sudo cp config.h /usr/local/include/quickfix/                  \
        && (test -f /sbin/ldconfig && sudo ldconfig || :)                                      \
)
make[1]: Leaving directory `/home/user/K5'
make uws
make[1]: Entering directory `/home/user/K5'
test -d build/uWebSockets-0.14.3 || curl -L https://github.com/uNetworking/uWebSockets/archive/v0.14.3.tar.gz | tar xz -C build
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   132    0   132    0     0    348      0 --:--:-- --:--:-- --:--:--   349
  0     0    0  465k    0     0   643k      0 --:--:-- --:--:-- --:--:-- 4749k
make[1]: Leaving directory `/home/user/K5'
NODEv=v7.1.0 ABIv=51 make node `(uname -s)`
make[1]: Entering directory `/home/user/K5'
test -d build/node-v7.1.0 || curl https://nodejs.org/dist/v7.1.0/node-v7.1.0-headers.tar.gz | tar xz -C build
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  471k  100  471k    0     0   556k      0 --:--:-- --:--:-- --:--:--  556k
g++ -std=c++11 -DUSE_LIBUV -O3 -shared -fPIC -Ibuild/node-v7.1.0/include/node -Ibuild/uWebSockets-0.14.3/src              build/uWebSockets-0.14.3/src/Extensions.cpp build/uWebSockets-0.14.3/src/Group.cpp      build/uWebSockets-0.14.3/src/Networking.cpp build/uWebSockets-0.14.3/src/Hub.cpp        build/uWebSockets-0.14.3/src/Node.cpp build/uWebSockets-0.14.3/src/WebSocket.cpp  build/uWebSockets-0.14.3/src/HTTPSocket.cpp build/uWebSockets-0.14.3/src/Socket.cpp     build/uWebSockets-0.14.3/src/Epoll.cpp src/lib/K.cc -static-libstdc++ -static-libgcc -s -o app/server/lib/K.linux.51.node -lsqlite3
make[1]: Leaving directory `/home/user/K5'
NODEv=v8.1.2 ABIv=57 make node `(uname -s)`
make[1]: Entering directory `/home/user/K5'
test -d build/node-v8.1.2 || curl https://nodejs.org/dist/v8.1.2/node-v8.1.2-headers.tar.gz | tar xz -C build
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  479k  100  479k    0     0   538k      0 --:--:-- --:--:-- --:--:--  537k
g++ -std=c++11 -DUSE_LIBUV -O3 -shared -fPIC -Ibuild/node-v8.1.2/include/node -Ibuild/uWebSockets-0.14.3/src              build/uWebSockets-0.14.3/src/Extensions.cpp build/uWebSockets-0.14.3/src/Group.cpp      build/uWebSockets-0.14.3/src/Networking.cpp build/uWebSockets-0.14.3/src/Hub.cpp        build/uWebSockets-0.14.3/src/Node.cpp build/uWebSockets-0.14.3/src/WebSocket.cpp  build/uWebSockets-0.14.3/src/HTTPSocket.cpp build/uWebSockets-0.14.3/src/Socket.cpp     build/uWebSockets-0.14.3/src/Epoll.cpp src/lib/K.cc -static-libstdc++ -static-libgcc -s -o app/server/lib/K.linux.57.node -lsqlite3
make[1]: Leaving directory `/home/user/K5'
for K in app/server/lib/K*node; do chmod +x $K; done
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA
verify return:1
depth=0 C = US, ST = California, L = San Francisco, O = "Coinbase, Inc.", CN = *.gdax.com
verify return:1
DONE

> [email protected] test-stunnel /home/user/K5
> test -z "$SKIP_STUNNEL$(ps axu | grep stunnel | grep -v grep)" && exec stunnel dist/K-stunnel.conf || :


> [email protected] install /home/user/K5/node_modules/uws
> node-gyp rebuild > build_log.txt 2>&1 || exit 0


> [email protected] install /home/user/K5/node_modules/utf-8-validate
> prebuild-install || node-gyp rebuild

prebuild-install info begin Prebuild-install version 2.2.0
prebuild-install info looking for local prebuild @ prebuilds/utf-8-validate-v3.0.3-node-v57-linux-x64.tar.gz
prebuild-install info looking for cached prebuild @ /home/user/.npm/_prebuilds/https-github.com-websockets-utf-8-validate-releases-download-v3.0.3-utf-8-validate-v3.0.3-node-v57-linux-x64.tar.gz
prebuild-install info found cached prebuild
prebuild-install info unpacking @ /home/user/.npm/_prebuilds/https-github.com-websockets-utf-8-validate-releases-download-v3.0.3-utf-8-validate-v3.0.3-node-v57-linux-x64.tar.gz
prebuild-install info unpack resolved to /home/user/K5/node_modules/utf-8-validate/build/Release/validation.node
prebuild-install info unpack required /home/user/K5/node_modules/utf-8-validate/build/Release/validation.node successfully
prebuild-install info install Prebuild successfully installed!

> [email protected] install /home/user/K5/node_modules/node-quickfix-wrap
> node-gyp rebuild

make: Entering directory `/home/user/K5/node_modules/node-quickfix-wrap/build'
  CXX(target) Release/obj.target/node_quickfix/src/FixSession.o
  CXX(target) Release/obj.target/node_quickfix/src/FixLoginProvider.o
  CXX(target) Release/obj.target/node_quickfix/src/FixLoginResponse.o
  CXX(target) Release/obj.target/node_quickfix/src/FixAcceptor.o
  CXX(target) Release/obj.target/node_quickfix/src/FixAcceptorStartWorker.o
  CXX(target) Release/obj.target/node_quickfix/src/FixAcceptorStopWorker.o
  CXX(target) Release/obj.target/node_quickfix/src/FixApplication.o
  CXX(target) Release/obj.target/node_quickfix/src/FixConnection.o
  CXX(target) Release/obj.target/node_quickfix/src/FixInitiator.o
  CXX(target) Release/obj.target/node_quickfix/src/FixInitiatorStartWorker.o
  CXX(target) Release/obj.target/node_quickfix/src/FixInitiatorStopWorker.o
  CXX(target) Release/obj.target/node_quickfix/src/FixSendWorker.o
  CXX(target) Release/obj.target/node_quickfix/src/node_quickfix.o
  SOLINK(target) Release/obj.target/node_quickfix.node
  COPY Release/node_quickfix.node
make: Leaving directory `/home/user/K5/node_modules/node-quickfix-wrap/build'

> [email protected] install /home/user/K5/node_modules/bufferutil
> prebuild-install || node-gyp rebuild

prebuild-install info begin Prebuild-install version 2.2.0
prebuild-install info looking for local prebuild @ prebuilds/bufferutil-v3.0.2-node-v57-linux-x64.tar.gz
prebuild-install info looking for cached prebuild @ /home/user/.npm/_prebuilds/https-github.com-websockets-bufferutil-releases-download-v3.0.2-bufferutil-v3.0.2-node-v57-linux-x64.tar.gz
prebuild-install info found cached prebuild
prebuild-install info unpacking @ /home/user/.npm/_prebuilds/https-github.com-websockets-bufferutil-releases-download-v3.0.2-bufferutil-v3.0.2-node-v57-linux-x64.tar.gz
prebuild-install info unpack resolved to /home/user/K5/node_modules/bufferutil/build/Release/bufferutil.node
prebuild-install info unpack required /home/user/K5/node_modules/bufferutil/build/Release/bufferutil.node successfully
prebuild-install info install Prebuild successfully installed!

> [email protected] postinstall /home/user/K5
> npm run compile-server && npm run compile-client && npm run compile-public && npm run compile-bundle


> [email protected] compile-server /home/user/K5
> tsc --alwaysStrict -t ES6 -m commonjs --outDir app src/server/*.ts src/server/*/*.ts src/share/*.ts


> [email protected] compile-client /home/user/K5
> tsc --alwaysStrict --experimentalDecorators -t ES6 -m commonjs --outDir app/pub/js src/client/*.ts src/share/*.ts


> [email protected] compile-public /home/user/K5
> cp -R src/pub/* app/pub/


> [email protected] compile-bundle /home/user/K5
> browserify -t [ babelify --presets [ babili es2016 ] ] app/pub/js/client/main.js app/pub/js/lib/*.js | uglifyjs | gzip > app/pub/js/client/bundle.min.js

added 607 packages in 170.528s
warn:    --minUptime not set. Defaulting to: 1000ms
warn:    --spinSleepTime not set. Your script will exit if it does not stay up for at least 1000ms
info:    Forever processing file: K.js

> [email protected] test-stunnel /home/user/K5
> test -z "$SKIP_STUNNEL$(ps axu | grep stunnel | grep -v grep)" && exec stunnel dist/K-stunnel.conf || :

Still same error:


[user@ubuntu-2gb-sfo2-04 K5]$ node K.js K.json
K.linux.57
module.js:598
  return process.dlopen(module, path._makeLong(filename));
                 ^

Error: libquickfix.so.16: cannot open shared object file: No such file or directory
    at Object.Module._extensions..node (module.js:598:18)
    at Module.load (module.js:503:32)
    at tryModuleLoad (module.js:466:12)
    at Function.Module._load (module.js:458:3)
    at Module.require (module.js:513:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/home/user/K5/node_modules/node-quickfix-wrap/index.js:1:78)
    at Module._compile (module.js:569:30)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:503:32)

dexter369 on 12 Jul 2017

err.. can you try sudo ldconfig and see if magically fixes your problem? (may not output nothing but link the new lib silently)
thanks'¡

otherwise i will suggest to recompile quickfix witth sudo rm /usr/local/lib/libquickfix.so* && npm install

ctubio on 12 Jul 2017

Thanks tried both and reached back to same error.

So I tried fresh Centos 7 install and got to the error like:
/usr/bin/ld: cannot find -lstdc++
So I executed:
sudo yum install libstdc++-devel.i686
Which took me to next error :
forever command not found
So I executed:
npm install forever
Now I am getting the error:

Service [gdax_stream]: Failed to initialize SSL context
str_stats: 10 block(s), 1153 data byte(s), 580 control byte(s)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] test-stunnel: `test -z "$SKIP_STUNNEL$(ps axu | grep stunnel | grep -v grep)" && exec stunnel dist/K-stunnel.conf || :`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] test-stunnel script.

I have even set Bitfinex in config but getting Gdax related error.
Is there something obvious I am missing leading to so many issues?

dexter369 on 12 Jul 2017

this error means that the SSL cert is missing, but you have downloaded it as the log says

it should be at /usr/local/etc/stunnel/fix.gdax.com.pem

to retry again openssl s_client -showcerts -connect fix.gdax.com:4198 < /dev/null | openssl x509 -outform PEM > fix.gdax.com.pem && sudo mv fix.gdax.com.pem /usr/local/etc/stunnel/ && npm run test-stunnel

if you dont want to use gdax, you can skip this stunnel setup by setting environment variable SKIP_STUNNEL to some value, for example SKIP_STUNNEL=1 npm start (later you can maybe keep hunting this error without hurry [when i used centos i always need to sudo yum group install "Development Tools" not sure if this is the missing step here?])

ctubio on 12 Jul 2017

Actually the problem seems to be:
/usr/bin/ld: cannot find -lstdc++
This is not fixed by
sudo yum install libstdc++-devel
But if I run npm start again it skips the error and the rest of the instructions :
[email protected] install-packages: test -n "$(command -v apt-get)" && sudo apt-get -y install g++ build-essential automake autoconf libtool libxml2 libxml2-dev zlib1g-dev libsqlite3-dev libssl-dev openssl stunnel python wget gzip || (test -n "$(command -v yum)" && sudo yum -y install gcc-c++ automake autoconf libtool libxml2 libxml2-devel zlib-devel sqlite-devel openssl openssl-devel zlib-devel stunnel python wget gzip) || (test -n "$(command -v brew)" && (xcode-select --install || :) && (brew install automake autoconf libxml2 sqlite openssl zlib libuv stunnel python wget gzip || brew upgrade || :)) && (sudo mkdir -p /data/db/ && sudo chown id -u /data/db) && make && openssl s_client -showcerts -connect fix.gdax.com:4198 < /dev/null | openssl x509 -outform PEM > fix.gdax.com.pem && sudo mv fix.gdax.com.pem /usr/local/etc/stunnel/ && exec npm run test-stunnel

So it goes to error forever missing, but I think forever should be installed automatically during install and is not installed as those instructions got skipped due to previous error.

Any idea how to fix this or any other OS where I have better chance to install:

build/uWebSockets-0.14.3/src/HTTPSocket.cpp build/uWebSockets-0.14.3/src/Socket.cpp     build/uWebSockets-0.14.3/src/Epoll.cpp src/lib/K.cc -static-libstdc++ -static-libgcc -s -o app/server/lib/K.linux.51.node -lsqlite3
/usr/bin/ld: cannot find -lstdc++
collect2: error: ld returned 1 exit status
make[1]: *** [Linux] Error 1
make[1]: Leaving directory `/home/demo/K'
make: *** [K] Error 2
npm ERR! code ELIFECYCLE
npm ERR! errno 2

Also sudo yum group install "Development Tools" did not solve the issue.

dexter369 on 12 Jul 2017

what about sudo yum -y install glibc-static libstdc++-static?

otherwise if you can move to Debian 8 or latest ubuntu, i just installed one today in 7 minutes

(the install fails repeating cos thinks it already have quickfix :( the only solution is to sudo rm /usr/local/lib/libquickfix.so* app build node_modules && npm install in case you wanna keep trying centOS)

ctubio on 12 Jul 2017

sudo yum libstdc++-static worked but finally reached back to the first error:

K.linux.57
module.js:598
  return process.dlopen(module, path._makeLong(filename));
                 ^

Error: libquickfix.so.16: cannot open shared object file: No such file or directory
    at Object.Module._extensions..node (module.js:598:18)
    at Module.load (module.js:503:32)
    at tryModuleLoad (module.js:466:12)
    at Function.Module._load (module.js:458:3)
    at Module.require (module.js:513:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/home/demo/K9/node_modules/node-quickfix-wrap/index.js:1:78)
    at Module._compile (module.js:569:30)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:503:32)

Thanks anyway, will try Debian 8 and Ubuntu

dexter369 on 12 Jul 2017

👍1

in ubuntu to instal node quick i did:

sudo apt-get install npm
sudo npm i -g n
sudo n latest

then restart bash session and check if node and nodejs are same version, otherwise remove one and symlink to the other so both are equal; then git clone and easy install

ctubio on 12 Jul 2017

Thanks finally got it installed and running on Ubuntu without any issue.

dexter369 on 13 Jul 2017

🎉1

Hey I had the same error when running it...
For me it was fixed with setting the right library path for it or actually setting a lib path

-> you need to find your own path though i guess

LD_LIBRARY_PATH=/usr/local/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH

mattygug on 13 Jul 2017

I have a similar path issue it seems on AWS Ubuntu 16:

root@ip-10-20-1-168:/home/ubuntu/Krypto-trading-bot# node K.js
K.linux.57
/home/ubuntu/Krypto-trading-bot/app/server/main.js:22
throw new Error(e);
^

Error: Error: Cannot find module './lib/K.linux.57'
at bindings (/home/ubuntu/Krypto-trading-bot/app/server/main.js:22:19)
at Object. (/home/ubuntu/Krypto-trading-bot/app/server/main.js:24:3)
at Module._compile (module.js:569:30)
at Object.Module._extensions..js (module.js:580:10)
at Module.load (module.js:503:32)
at tryModuleLoad (module.js:466:12)
at Function.Module._load (module.js:458:3)
at Module.require (module.js:513:17)
at require (internal/module.js:11:18)
at Object. (/home/ubuntu/Krypto-trading-bot/K.js:2:1)