Tfjs: tfjs to use with Node-RED in NVIDIA Jetson Nano

Created on 12 Mar 2020  路  21Comments  路  Source: tensorflow/tfjs

Hi, trying to install tensorflow.js to use with Node-RED in NVIDIA Jetson Nano. Node-RED itself runs perfect in the Nano
Specifically I want to use this Node-RED node: https://flows.nodered.org/node/node-red-contrib-tfjs-coco-ssd))

but it does not work due to required dependencies (@tensorflow) not working. So in attempt to solve this issue, I tried to install @tensorflow manually:

npm install @tensorflow/tfjs-node
and
npm install @tensorflow/tfjs-node-gpu

but it gives error messages

CPU-linux-1.7.0.tar.gz

  • Downloading libtensorflow
    (node:17165) UnhandledPromiseRejectionWarning: Error: Unsupported system: cpu-linux-arm64

and

GPU-linux-1.7.0.tar.gz

  • Downloading libtensorflow
    (node:9011) UnhandledPromiseRejectionWarning: Error: Unsupported system: gpu-linux-arm64

Do we have any solution to make TensorFlow.js via Node.js work on arm64?

node.js builinstall
Source
picture krambriw

Most helpful comment

Understood, but thats a pity since there are many interesting devices/platforms based on arm64 like the NVIDIA Jetson Nano and products from Odroid, they would be a perfect choice for your tjfs but for me and many others, building is very complicated and what you linked to makes room for many user mistakes on the way. If there was a step-by-step instruction how I could build/cross-compile on my mac and then transfer the result to my Jetson Nano, it would maybe be doable and also take much less time

picture krambriw on 19 Mar 2020
👍3

All 21 comments

Hi,

We don't officially support arm64, but here are instructions on building a custom TF library from source, which targets a specific architecture: https://github.com/tensorflow/tfjs/tree/master/tfjs-node#optional-build-optimal-tensorflow-from-source

dsmilkov picture dsmilkov on 12 Mar 2020

Understood, but thats a pity since there are many interesting devices/platforms based on arm64 like the NVIDIA Jetson Nano and products from Odroid, they would be a perfect choice for your tjfs but for me and many others, building is very complicated and what you linked to makes room for many user mistakes on the way. If there was a step-by-step instruction how I could build/cross-compile on my mac and then transfer the result to my Jetson Nano, it would maybe be doable and also take much less time

krambriw picture krambriw on 19 Mar 2020
👍3

can there be a tutorial or something for arm64, if not official support? I'm sure its not uncommon for people to want it on arm64. If things change after that it will be fine because at least we'd have a tutorial we know worked for one of the developers and/or community members

moeiscool picture moeiscool on 10 Apr 2020

@kangyizhang have you looked into arm64 tfjs-node support before?

pyu10055 picture pyu10055 on 10 Apr 2020

here you go: https://github.com/yhwang/node-red-contrib-tf-model#note
there are instructions to install @tensorflow/tfjs-node on Jetson Nano.
Following the instructions there, it will download the pre-built tensorflow shared lib for arm64 and build the node binding locally.

In there, it also provides an example flow for coco-ssd :-)

yhwang picture yhwang on 11 Apr 2020
👍2 👎1

We did support arm64 but it was removed in a later change. I will add it back.

kangyizhang picture kangyizhang on 11 Apr 2020

@kangyizhang just want to double check, I don't see any arm64 support here: https://storage.googleapis.com/tf-builds

yhwang picture yhwang on 11 Apr 2020

@yhwang This is the link for arm binary: https://storage.cloud.google.com/tf-builds/libtensorflow_r1_14_linux_arm.tar.gz. The support was accidentally removed in https://github.com/tensorflow/tfjs/commit/63235661d1c1f26ff5b6853375a59213fe99d9a5#diff-943b21d25f7c485a9de071e9a1d1c18bL80

kangyizhang picture kangyizhang on 11 Apr 2020

@kangyizhang I know, but that's arm32 not arm64 and it's tensorflow 1.14. I think the later tfjs-node depends on tensorflow 1.15

yhwang picture yhwang on 11 Apr 2020

Yes, it is arm32 with tensorflow 1.14. We do not support later version of tensorflow with arm.

kangyizhang picture kangyizhang on 11 Apr 2020

@yhwang
Fantastic! It works now in the NVIDIA Jetson Nano thanks to your great instructions!!!
Do you also know if this in principle would be possible with tfjs-node-gpu? To take advantage of all GPU kernels in the Jetson Nano

krambriw picture krambriw on 12 Apr 2020

@krambriw actually the tensorflow shared lib you installed following the instructions there is using CUDA already! It means you are already using tfjs-node-gpu!

yhwang picture yhwang on 12 Apr 2020

@yhwang
Aha, so I do not need to install tfjs-node-gpu?
Do you know if it is possible for me to verify that GPU really is used and not the CPU?
Is it possible to set some switch to use GPU/CPU to notice the differens?

krambriw picture krambriw on 13 Apr 2020

@krambriw You don't have to install tfjs-node-gpu if you follow the instructions there for Jetson Nano.
The difference between tfjs-node and tfjs-node-gpu is if your tensorflow shared lib is built with GPU configuration or not. For Jetson Nano, I only built tensorflow shard lib with GPU configuration. I guess most of people want to leverage the GPU.

If you run the require() below in interactive node cli

$ node
> const tfjs = require('@tensorflow/tfjs-node');

You may see something like this:

2020-04-13 14:58:13.190037: W tensorflow/core/platform/profile_utils/cpu_utils.cc:98] Failed to find bogomips in /proc/cpuinfo; cannot determine CPU frequency
2020-04-13 14:58:13.190773: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2b48c0b0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-04-13 14:58:13.190830: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-04-13 14:58:13.228780: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-04-13 14:58:13.268523: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-04-13 14:58:13.268682: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
name: NVIDIA Tegra X1 major: 5 minor: 3 memoryClockRate(GHz): 0.9216
pciBusID: 0000:00:00.0
2020-04-13 14:58:13.275176: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2020-04-13 14:58:13.316117: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2020-04-13 14:58:13.346827: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2020-04-13 14:58:13.387466: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2020-04-13 14:58:13.444067: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2020-04-13 14:58:13.465685: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2020-04-13 14:58:13.592877: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7

It means the tensorflow shared lib found CUDA libraries and would use the GPU.

For GPU utilization, you can use tegrastats command. The GR3D_FREQ 0% is the GPU utilization.

For switching different backends, you can use setBackend() API to switch between GPU and CPU. i.e. tf.setBackend('cpu') to CPU backend and tf.setBackend('tensorflow') to tensorflow backend which is using GPU. (assuming you assign the require('@tensorflow/tfjs-node') to tf)

yhwang picture yhwang on 14 Apr 2020

Thank you so very much! Yes, indeed, it uses the GPU
Once again, thank you for providing help to make the Jetson Nano possible to use with tfjs

krambriw picture krambriw on 14 Apr 2020
🎉1

if i install it like this:

ki node # npm install @tensorflow/[email protected]

> @tensorflow/[email protected] install /node/node_modules/@tensorflow/tfjs-node
> node scripts/install.js

CPU-linux-1.5.1.tar.gz
* Downloading libtensorflow
(node:8997) UnhandledPromiseRejectionWarning: Error: Unsupported system: cpu-linux-arm64
    at getPlatformLibtensorflowUri (/node/node_modules/@tensorflow/tfjs-node/scripts/install.js:95:11)
    at downloadLibtensorflow (/node/node_modules/@tensorflow/tfjs-node/scripts/install.js:129:7)
    at async run (/node/node_modules/@tensorflow/tfjs-node/scripts/install.js:190:5)
(node:8997) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:8997) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
npm WARN saveError ENOENT: no such file or directory, open '/node/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/node/package.json'
npm WARN node No description
npm WARN node No repository field.
npm WARN node No README data
npm WARN node No license field.

+ @tensorflow/[email protected]
added 93 packages from 100 contributors and audited 159 packages in 17.512s

1 package is looking for funding
  run `npm fund` for details

found 0 vulnerabilities

ki node # cd node_modules/@tensorflow/tfjs-node
ki tfjs-node # nano scripts/custom-binary.json
ki tfjs-node # cat scripts/custom-binary.json
{
  "tf-lib": "https://s3.us.cloud-object-storage.appdomain.cloud/tfjs-cos/libtensorflow-gpu-linux-arm64-1.15.0.tar.gz"
}
ki tfjs-node # npm install
npm WARN lifecycle @tensorflow/[email protected]~install: cannot run in wd @tensorflow/[email protected] node scripts/install.js (wd=/node/node_modules/@tensorflow/tfjs-node)
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN @tensorflow/[email protected] No license field.

added 364 packages from 244 contributors and audited 784 packages in 25.127s

2 packages are looking for funding
  run `npm fund` for details

found 9 vulnerabilities (2 low, 1 moderate, 5 high, 1 critical)
  run `npm audit fix` to fix them, or `npm audit` for details
ki tfjs-node #

i get this error:

ki node # node
Welcome to Node.js v12.16.2.
Type ".help" for more information.
> const tfjs = require('@tensorflow/tfjs-node');
Uncaught:
Error: Cannot find module '/node/node_modules/@tensorflow/tfjs-node/lib/napi-v5/tfjs_binding.node'
Require stack:
- /node/node_modules/@tensorflow/tfjs-node/dist/index.js
- <repl>
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:980:15)
    at Function.Module._load (internal/modules/cjs/loader.js:862:27)
    at Module.require (internal/modules/cjs/loader.js:1042:19)
    at require (internal/modules/cjs/helpers.js:77:18) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/node/node_modules/@tensorflow/tfjs-node/dist/index.js',
    '<repl>'
  ]
}
>

is there something i missed?

EDIT:

running script/install.js manually helped but then i get this error:

ki node # node
Welcome to Node.js v12.16.2.
Type ".help" for more information.
> const tfjs = require('@tensorflow/tfjs-node');
2020-04-24 14:26:55.743194: W tensorflow/core/platform/profile_utils/cpu_utils.cc:98] Failed to find bogomips in /proc/cpuinfo; cannot determine CPU frequency
2020-04-24 14:26:55.743844: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x17d09690 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-04-24 14:26:55.743898: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-04-24 14:26:55.747573: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-04-24 14:26:55.753334: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-04-24 14:26:55.753473: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: NVIDIA Tegra X1 major: 5 minor: 3 memoryClockRate(GHz): 0.9216
pciBusID: 0000:00:00.0
2020-04-24 14:26:55.753714: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory
2020-04-24 14:26:55.753881: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory
2020-04-24 14:26:55.754046: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory
2020-04-24 14:26:55.754203: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory
2020-04-24 14:26:55.754356: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory
2020-04-24 14:26:55.754511: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory
2020-04-24 14:26:55.754660: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudnn.so.7'; dlerror: libcudnn.so.7: cannot open shared object file: No such file or directory
2020-04-24 14:26:55.754694: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1641] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
2020-04-24 14:26:55.817135: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-04-24 14:26:55.817203: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0
2020-04-24 14:26:55.817235: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N
2020-04-24 14:26:55.821827: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-04-24 14:26:55.822048: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1819f6c0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-04-24 14:26:55.822093: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA Tegra X1, Compute Capability 5.3
undefined
>

seems like there is no support for:

 - Libraries:
   * CUDA: 10.2.89
   * cuDNN: 8.0.0.145
   * TensorRT: 7.1.0.16
celevra picture celevra on 24 Apr 2020
😕1

@krambriw You don't have to install tfjs-node-gpu if you follow the instructions there for Jetson Nano.
The difference between tfjs-node and tfjs-node-gpu is if your tensorflow shared lib is built with GPU configuration or not. For Jetson Nano, I only built tensorflow shard lib with GPU configuration. I guess most of people want to leverage the GPU.

If you run the require() below in interactive node cli

$ node
> const tfjs = require('@tensorflow/tfjs-node');

You may see something like this:

2020-04-13 14:58:13.190037: W tensorflow/core/platform/profile_utils/cpu_utils.cc:98] Failed to find bogomips in /proc/cpuinfo; cannot determine CPU frequency
2020-04-13 14:58:13.190773: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2b48c0b0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-04-13 14:58:13.190830: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-04-13 14:58:13.228780: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-04-13 14:58:13.268523: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-04-13 14:58:13.268682: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
name: NVIDIA Tegra X1 major: 5 minor: 3 memoryClockRate(GHz): 0.9216
pciBusID: 0000:00:00.0
2020-04-13 14:58:13.275176: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2020-04-13 14:58:13.316117: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2020-04-13 14:58:13.346827: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2020-04-13 14:58:13.387466: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2020-04-13 14:58:13.444067: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2020-04-13 14:58:13.465685: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2020-04-13 14:58:13.592877: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7

It means the tensorflow shared lib found CUDA libraries and would use the GPU.

For GPU utilization, you can use tegrastats command. The GR3D_FREQ 0% is the GPU utilization.

For switching different backends, you can use setBackend() API to switch between GPU and CPU. i.e. tf.setBackend('cpu') to CPU backend and tf.setBackend('tensorflow') to tensorflow backend which is using GPU. (assuming you assign the require('@tensorflow/tfjs-node') to tf)

Is there a compilation process, CUDA depends on the wrong version and cannot be used.

land007 picture land007 on 10 May 2020

seems like there is no support for:

 - Libraries:
   * CUDA: 10.2.89
   * cuDNN: 8.0.0.145
   * TensorRT: 7.1.0.16

If someone still ends up here, I managed to get tfjs-node working with these versions by "a few" extra steps:

  1. Create symlinks of the libraries which are printed on startup, either inside /usr/local/cuda/lib64 or /usr/lib/aarch64-linux-gnu depending on where the library exists. E.g.
cd /usr/local/cuda/lib64 && \
sudo ln -s libcudart.so.10.2 libcudart.so.10.0 && \
sudo ln -s libcufft.so.10 libcufft.so.10.0 && \
sudo ln -s libcurand.so.10 libcurand.so.10.0 && \
sudo ln -s libcusolver.so.10 libcusolver.so.10.0 && \
sudo ln -s libcusparse.so.10 libcusparse.so.10.0

cd /usr/lib/aarch64-linux-gnu && \
sudo ln -s libcudnn.so.8 libcudnn.so.7 && \
sudo ln -s libcublas.so.10 libcublas.so.10.0
  1. Make sure you have LD_LIBRARY_PATH set to /usr/local/cuda/lib64:/usr/lib/aarch64-linux-gnu before starting Node-RED. I had to add this line to /etc/systemd/system/multi-user.target.wants/nodered.service as I had NR running via systemd:
Environment="LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/lib/aarch64-linux-gnu"

  1. After this it seemed like tfjs-node springed to life but after a short while I got an error telling me libcudnn does not work, even telling me the major version needs to match the one tfjs-node was built against (v7.6). It took me hours to track but actually a 7.x version of the library built against CUDA 10.2 was readily available at https://developer.nvidia.com/cuda-toolkit/arm even though the latest JetPack comes with CUDA 10.2 and CuDNN 8.0 bundled.

    Installing the deb package libcudnn7_7.6.4.30-1+cuda10.2_arm64.deb did not work as it conflicted with nvidia-cudnn8 already installed. Removing v8 wouldn't have worked either as it would have also removed the CUDA package among bunch of others. I then went for a brute force approach to download the tar package cudnn-10.2-linux-aarch64-v7.6.4.30.tgz.

    From the tar package I first renamed libcudnn_static.a to libcudnn_static_v7.a. I then copied the renamed file withlibcudnn.so.7 and libcudnn.so.7.6.4 to /usr/lib/aarch64-linux-gnu. I had noted /usr/lib/aarch64-linux-gnu/libcudnn_static.a already existed, hence the renaming. The existing one was a symlink to /etc/alternatives/libcudnn_stlib which in turn was a symlink back to /usr/lib/aarch64-linux-gnu/libcudnn_static_v8.a. I made the symlink point to the v7 file:

sudo ln -sf /usr/lib/aarch64-linux-gnu/libcudnn_static_v8.a /etc/alternatives/libcudnn_stlib

Miraculously after this last mod tfjs-node seemed to start succesfully. I've had it now running for 2 days so it's still early to say if this works for the long term.

CAUTION: Making these mods can break something else (or even damage the GPU according to a comment on another issue thread where someone hinted the symlinking approach). Don't do this if you're feeling uncomfortable of doing so!

ristomatti picture ristomatti on 8 Aug 2020
👍1

@ristomatti According to your method, my program can run. But there is a small problem. My program will get stuck in Adding visible gpu devices: 0, wait a long time to continue running. Have you encountered the same problem?

2020-08-19 14:29:51.384747: W tensorflow/core/platform/profile_utils/cpu_utils.cc:98] Failed to find bogomips in /proc/cpuinfo; cannot determine CPU frequency
2020-08-19 14:29:51.387164: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f81db0750 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-08-19 14:29:51.388022: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-08-19 14:29:51.399238: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-08-19 14:29:51.448696: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-19 14:29:51.449048: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
name: Xavier major: 7 minor: 2 memoryClockRate(GHz): 1.377
pciBusID: 0000:00:00.0
2020-08-19 14:29:51.451177: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2020-08-19 14:29:51.456995: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2020-08-19 14:29:51.461691: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2020-08-19 14:29:51.464808: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2020-08-19 14:29:51.474856: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2020-08-19 14:29:51.479987: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2020-08-19 14:29:51.499100: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-08-19 14:29:51.499461: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-19 14:29:51.499954: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-19 14:29:51.500103: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0

Wait very long......(About 10 minutes)

2020-08-19 14:33:07.677999: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-08-19 14:33:07.678254: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0 
2020-08-19 14:33:07.678368: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N 
2020-08-19 14:33:07.679940: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-19 14:33:07.680271: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-19 14:33:07.680519: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-19 14:33:07.680732: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 22155 MB memory) -> physical GPU (device: 0, name: Xavier, pci bus id: 0000:00:00.0, compute capability: 7.2)
2020-08-19 14:33:07.690358: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ecb99a2a0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-08-19 14:33:07.690581: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Xavier, Compute Capability 7.2
Epoch 1 / 1
2020-08-19 14:33:15.513769: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
eta=0.0 ============================================================================> 
8728ms 2181986us/step - loss=2.10

If you have any ideas, please tell me.

land007 picture land007 on 19 Aug 2020

@ristomatti According to your method, my program can run. But there is a small problem. My program will get stuck in Adding visible gpu devices: 0, wait a long time to continue running. Have you encountered the same problem?

2020-08-19 14:29:51.384747: W tensorflow/core/platform/profile_utils/cpu_utils.cc:98] Failed to find bogomips in /proc/cpuinfo; cannot determine CPU frequency
2020-08-19 14:29:51.387164: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f81db0750 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-08-19 14:29:51.388022: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-08-19 14:29:51.399238: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-08-19 14:29:51.448696: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-19 14:29:51.449048: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
name: Xavier major: 7 minor: 2 memoryClockRate(GHz): 1.377
pciBusID: 0000:00:00.0
2020-08-19 14:29:51.451177: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2020-08-19 14:29:51.456995: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2020-08-19 14:29:51.461691: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2020-08-19 14:29:51.464808: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2020-08-19 14:29:51.474856: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2020-08-19 14:29:51.479987: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2020-08-19 14:29:51.499100: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-08-19 14:29:51.499461: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-19 14:29:51.499954: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-19 14:29:51.500103: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0

Wait very long......(About 10 minutes)

2020-08-19 14:33:07.677999: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-08-19 14:33:07.678254: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0 
2020-08-19 14:33:07.678368: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N 
2020-08-19 14:33:07.679940: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-19 14:33:07.680271: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-19 14:33:07.680519: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-19 14:33:07.680732: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 22155 MB memory) -> physical GPU (device: 0, name: Xavier, pci bus id: 0000:00:00.0, compute capability: 7.2)
2020-08-19 14:33:07.690358: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ecb99a2a0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-08-19 14:33:07.690581: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Xavier, Compute Capability 7.2
Epoch 1 / 1
2020-08-19 14:33:15.513769: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
eta=0.0 ============================================================================> 
8728ms 2181986us/step - loss=2.10

If you have any ideas, please tell me.

Sorry, I did not encounter this issue.

ristomatti picture ristomatti on 20 Aug 2020

Through recompilation, the computing power is adjusted to 7.2 and the delay is reduced to 3 seconds. I see that the tensorflow source code can be modified to further reduce the delay, but I have not tried it yet. The compiled tensorflow is made into a docker image.

docker run  -it --runtime nvidia --name l4t-tfjs-node land007/l4t-tfjs-node:latest bash
root@5de9c61dc8a8:/node# node test.js 
2020-08-26 08:06:49.445605: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-08-26 08:06:49.513752: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-26 08:06:49.515138: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
name: Xavier major: 7 minor: 2 memoryClockRate(GHz): 1.377
pciBusID: 0000:00:00.0
2020-08-26 08:06:49.518479: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2020-08-26 08:06:49.525667: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2020-08-26 08:06:49.530436: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2020-08-26 08:06:49.532987: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2020-08-26 08:06:49.540070: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2020-08-26 08:06:49.544743: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2020-08-26 08:06:49.576619: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-08-26 08:06:49.577497: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-26 08:06:49.579292: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-26 08:06:49.580157: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2020-08-26 08:06:52.261992: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-08-26 08:06:52.262174: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0 
2020-08-26 08:06:52.262223: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N 
2020-08-26 08:06:52.262645: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-26 08:06:52.263097: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-26 08:06:52.263330: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
2020-08-26 08:06:52.263548: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 289 MB memory) -> physical GPU (device: 0, name: Xavier, pci bus id: 0000:00:00.0, compute capability: 7.2)
2020-08-26 08:06:52.361631: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x10139450 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-08-26 08:06:52.361836: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Xavier, Compute Capability 7.2
2020-08-26 08:06:52.371013: W tensorflow/core/platform/profile_utils/cpu_utils.cc:98] Failed to find bogomips in /proc/cpuinfo; cannot determine CPU frequency
2020-08-26 08:06:52.371848: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ed965e680 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-08-26 08:06:52.371920: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
Epoch 1 / 1
2020-08-26 08:06:52.735742: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
eta=0.0 =================================================================================================> 
2069ms 517294us/step - loss=73.40 
Tensor

The current image environment is

Ubuntu18.04
  -library:
    * node: 10.20.0
    * @tensorflow/tfjs-node: 1.5.1
    * Tensorflow: 1.15.0
    * CUDA: 10.0.326
    * cuDNN: 7.6.4
    * GLIBC 2.31

If you need C language call, you can use

docker run -it --rm --runtime nvidia land007/l4t-tensorflow-test:latest bash
cd /opt/hello_tf_c_api/build
./batch_interface

If you need to compile it yourself

docker run -it --runtime --name  nvidia l4t-tensorflow-compil land007/l4t-tensorflow-compil:latest bash
cd /opt/tensorflow && ./configuration
bazel build --config=opt --config=monolithic //tensorflow/tools/lib_package:libtensorflow

Thanks to yhwang and dusty-nv for their help

land007 picture land007 on 26 Aug 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

rlexa picture rlexa  路  3Comments
ritikrishu picture ritikrishu  路  4Comments
Arturbarth picture Arturbarth  路  3Comments
chrisdonahue picture chrisdonahue  路  3Comments
beele picture beele  路  3Comments