Deepspeech: could not find a version that satisfies the requirement deepspeech

[mdasari823]

I searched everywhere. I am not sure if this is a generic problem with pip or specific to deepspeech. I am able to install other packages through pip though. Any idea why I am getting this and how to resolve this?

I am trying to install deepspeech using pip3 install deepspeech

[kdavis-mozilla]

Is your platform supported?

[mdasari823]

Aah, I am not sure. I am using Jetson NANO devkit. I don't see any specific supported platforms in the instructions either. So if this is the reason, the only way to do this I assume is that I have to follow this procedure?

[kdavis-mozilla]

The supported platforms are indicated in the 0.5.1 Release Notes. Unfortunately, the Jetson NANO is not on the list.

However, that said Jetson NANO is an interesting target platform, and we'd be interested in any progress you make on a port.

[mdasari823]

Thank you @kdavis-mozilla. I will give it a shot. I will get back here if I can make it. Thanks.

[lissyx]

Given the situation on armv7 / aarch64 for pip wheels, I'm curious to see pip install --verbose deepspeech @mdasari823

[mdasari823]

@lissyx I attached the log.

log.txt

[lissyx]

Sad, no hint. To give context, we cannot upload aarch64 wheels for rpi3 to pypi because of lack of proper ABI definition. So even if we get all the CI in place to build aarch64 cuda, I'm unsure we could host them for pypi.

[mdasari823]

Do you mean you cannot upload for Jetson NANO? You guys already have support for Pi3 right? I am now trying to run deepspeech on RPi3.

[lissyx]

Do you mean you cannot upload for Jetson NANO? You guys already have support for Pi3 right? I am now trying to run deepspeech on RPi3.

Yes, but the upload is not done to the pypi repo, but to one that is enabled by default on Raspbian.

I don't know if a similar setup is done on the system running on the jetson nano.

[mdasari823]

Okay. I will try my luck on this because the deepspeech model on Pi3 is dog slow it seems. It is taking almost a minute for sentence of few words. Will let you know if I succeed. Thanks man for the suggestions.

[mdasari823]

@kdavis-mozilla @lissyx I am consistently getting this error when building deepspeech on Jetson Nano. Any hints on how to resolve this?

I am running

bazel build --workspace_status_command="bash native_client/bazel_workspace_status_cmd.sh" --config=monolithic --config=cuda -c opt --copt=-O3 --copt="-D_GLIBCXX_USE_CXX11_ABI=0" --copt=-fvisibility=hidden //native_client:libdeepspeech.so //native_client:generate_trie

I am getting

INFO: An error occurred during the fetch of repository 'local_config_cuda'
INFO: Call stack for the definition of repository 'local_config_cuda':
 - /home/essence/tensorflow/tensorflow/workspace.bzl:63:5
 - /home/essence/tensorflow/WORKSPACE:94:1
INFO: Repository 'local_config_cuda' used the following cache hits instead of downloading the corresponding file.
 * Hash '85a24f215737af91e0054d3a1cb435bd8ff06178cef14241c029c8a04ff16a79' for https://commondatastorage.googleapis.com/chromium-browser-clang/Linux_x64/clang-348507-1.tgz
If the definition of 'local_config_cuda' was updated, verify that the hashes were also updated.
ERROR: Skipping '//native_client:libdeepspeech.so': error loading package 'native_client': in /root/.cache/bazel/_bazel_root/36a71d6396a743d4c0feae555af85c88/external/org_tensorflow/tensorflow/tensorflow.bzl: Encountered error while reading extension file 'cuda/build_defs.bzl': no such package '@local_config_cuda//cuda': Traceback (most recent call last):

Attached are the tensorflow configure log and complete error log. I am using r1.14 version of yours.

configure-log.txt
complete-error-log.txt

[lissyx]

@mdasari823 Have you ran ./configure with CUDA steps ?

[mdasari823]

I just added --config=cudawhile configuring. Are there any other steps?

[lissyx]

I just added --config=cudawhile configuring. Are there any other steps?

Yes, please refer to upstream for configuration steps.

[mdasari823]

I don't see any other extra configuration specific to CUDA. I following this and I moved a bit further. Now, at least the compilation started and but getting failed in the middle. Here is the error.

WARNING: The following configs were expanded more than once: [cuda]. For repeatable flags, repeats are counted twice and may lead to unexpected behavior. INFO: Analysed 2 targets (0 packages loaded, 0 targets configured). INFO: Found 2 targets... ERROR: /home/essence/tensorflow/native_client/BUILD:150:1: undeclared inclusion(s) in rule '//native_client:generate_trie': this rule is missing dependency declarations for the following files included by 'native_client/generate_trie.cpp': '/usr/lib/gcc/aarch64-linux-gnu/4.8/include/stddef.h' '/usr/lib/gcc/aarch64-linux-gnu/4.8/include/stdarg.h' '/usr/lib/gcc/aarch64-linux-gnu/4.8/include/stdint.h' '/usr/lib/gcc/aarch64-linux-gnu/4.8/include-fixed/limits.h' '/usr/lib/gcc/aarch64-linux-gnu/4.8/include-fixed/syslimits.h' INFO: Elapsed time: 7.372s, Critical Path: 6.32s INFO: 0 processes. FAILED: Build did NOT complete successfully

[lissyx]

I don't see any other extra configuration specific to CUDA

You need to read upstream documentation on building with CUDA. Setup CUDA 10.0, pass TF_* variables at configure time, etc.

[lissyx]

https://www.tensorflow.org/install/gpu

[lissyx]

I don't see any other extra configuration specific to CUDA. I following this and I moved a bit further. Now, at least the compilation started and but getting failed in the middle. Here is the error.

WARNING: The following configs were expanded more than once: [cuda]. For repeatable flags, repeats are counted twice and may lead to unexpected behavior. INFO: Analysed 2 targets (0 packages loaded, 0 targets configured). INFO: Found 2 targets... ERROR: /home/essence/tensorflow/native_client/BUILD:150:1: undeclared inclusion(s) in rule '//native_client:generate_trie': this rule is missing dependency declarations for the following files included by 'native_client/generate_trie.cpp': '/usr/lib/gcc/aarch64-linux-gnu/4.8/include/stddef.h' '/usr/lib/gcc/aarch64-linux-gnu/4.8/include/stdarg.h' '/usr/lib/gcc/aarch64-linux-gnu/4.8/include/stdint.h' '/usr/lib/gcc/aarch64-linux-gnu/4.8/include-fixed/limits.h' '/usr/lib/gcc/aarch64-linux-gnu/4.8/include-fixed/syslimits.h' INFO: Elapsed time: 7.372s, Critical Path: 6.32s INFO: 0 processes. FAILED: Build did NOT complete successfully

That error does not match your link: yours refers to GCC 4.8 while the link documents GCC 5.5.

You might want to have a look at https://github.com/mozilla/tensorflow/blob/r1.14/.bazelrc#L76-L81 and re-use similar flags to perform Aarch64 cross-compilation.

[lissyx]

Ah, I followed those instructions. I am able to compile at least few files so far. The following error is common it seems. Any idea?

Yes, that's inconcistent with your link, again. Bazel will refuse building against headers it does not knows about for reproductibility. So you are likely missing something somewhere. This can be tricky. That's why I suggest relying on our tested aarch64 cross-compilation, I can't debug other's infras.

[mdasari823]

Yes, I am using this procedure and Mozilla's tensorflow. Let me see with the flags that you suggested. Thanks.

[mdasari823]

@lissyx I finally built the deepspeech libraries for Jetson Nano. But, when I run the model I am getting model incompatibility issues. Please see below for the full error. Any idea how I can proceed?

essence@essence-desktop:~$ deepspeech --model models/output_graph.pbmm --alphabet models/alphabet.txt --audio audio/2830-3980-0043.wav Loading model from file models/output_graph.pbmm TensorFlow: v1.14.0-14-g1aad02a78e DeepSpeech: v0.6.0-alpha.4-40-g7f642ed 2019-08-17 19:17:02.068494: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1 2019-08-17 19:17:02.092676: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:972] ARM64 does not support NUMA - returning NUMA node zero 2019-08-17 19:17:02.092822: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: name: NVIDIA Tegra X1 major: 5 minor: 3 memoryClockRate(GHz): 0.9216 pciBusID: 0000:00:00.0 2019-08-17 19:17:02.092863: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, skip dlopen check. 2019-08-17 19:17:02.092970: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:972] ARM64 does not support NUMA - returning NUMA node zero 2019-08-17 19:17:02.093102: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:972] ARM64 does not support NUMA - returning NUMA node zero 2019-08-17 19:17:02.093163: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0 2019-08-17 19:17:02.551550: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix: 2019-08-17 19:17:02.551628: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0 2019-08-17 19:17:02.551658: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N 2019-08-17 19:17:02.551868: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:972] ARM64 does not support NUMA - returning NUMA node zero 2019-08-17 19:17:02.552063: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:972] ARM64 does not support NUMA - returning NUMA node zero 2019-08-17 19:17:02.552222: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:972] ARM64 does not support NUMA - returning NUMA node zero 2019-08-17 19:17:02.552330: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 16 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X1, pci bus id: 0000:00:00.0, compute capability: 5.3) Specified model file version (0) is incompatible with minimum version supported by this client (2). See https://github.com/mozilla/DeepSpeech/#model-compatibility for more information Traceback (most recent call last): File "/usr/local/bin/deepspeech", line 10, in <module> sys.exit(main()) File "/usr/local/lib/python3.6/dist-packages/deepspeech/client.py", line 91, in main ds = Model(args.model, N_FEATURES, N_CONTEXT, args.alphabet, BEAM_WIDTH) File "/usr/local/lib/python3.6/dist-packages/deepspeech/__init__.py", line 23, in __init__ raise RuntimeError("CreateModel failed with error code {}".format(status)) RuntimeError: CreateModel failed with error code 8195

[lissyx]

Are you trying to run 0.5.1 model on master build? You need to have matching versions, we have a 0.6.0a4 model on github, maybe @reuben has the link?

[lissyx]

Try that https://github.com/reuben/DeepSpeech/releases/tag/v0.6.0-alpha.4

Also did you cross compile?

[mdasari823]

@lissyx thank you so much. Yes, I was running 0.5.1 model on master build. Your new model v0.6.0 is running successfully. Thanks.
I built it on Nano itself, did not cross compile. It was really slow to build bazel dependency and had many issues. I will post detailed instructions and all commands that I followed. Thanks.

[lissyx]

I built it on Nano itself, did not cross compile. It was really slow to build bazel dependency and had many issues. I will post detailed instructions and all commands that I followed.

Thanks, I'd really prefer seeing it being cross-compiled: this way it's a first step for us to be able to leverage and provide it.

[mdasari823]

Yes, I realised in the middle that I should have cross-compiled. Would have been faster. I will git it a try some time this week.

[forscience6]

@lissyx thank you so much. Yes, I was running 0.5.1 model on master build. Your new model v0.6.0 is running successfully. Thanks.
I built it on Nano itself, did not cross compile. It was really slow to build bazel dependency and had many issues. I will post detailed instructions and all commands that I followed. Thanks.

@mdasari823 I'm running into similar issues specifically on building TensorFlow with bazel after configuring it. Do you mind letting me know the steps that you took to get it to work on the Jetson Nano?

[mdasari823]

@michaelguo32 I listed the issues that I faced and the resources that I referred here. I had to try out different combinations. Post the issue here if you want me to give you any hint.

[forscience6]

@michaelguo32 I listed the issues that I faced and the resources that I referred here. I had to try out different combinations. Post the issue here if you want me to give you any hint.

When I try to build tensorflow using the following command found at step 7 in this link:
bazel build --config=opt --config=cuda --config=nonccl //tensorflow/tools/pip_package:build_pip_package --verbose_failures --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0"

I get a message saying FAILED: Build did NOT complete successfully

I've successfully built bazel version 0.25.2 following the instructions from the same link above and I'm using the r1.14 branch of the mozilla's tensorflow repository as you suggested as well as gcc-5.

I have attached the full error log and tensorflow configuration below:
error_log2.txt
tf_configuration.txt

[mdasari823]

Looks like compiler problem. Adding --copt=”-std=gnu99” in the bazel command may help but not sure.

[forscience6]

Looks like compiler problem. Adding --copt=”-std=gnu99” in the bazel command may help but not sure.

It's still not working. I didn't notice you building the tensorflow library with the command I posted right above in your instructions. I just see that you built the deepspeech library with the following command right after you ran ./configure for tensorflow:
bazel build --workspace_status_command=”bash native_client/bazel_workspace_status_cmd.sh” --config=monolithic --config=cuda -c opt --copt=-O3 --copt=”-std=gnu99” --copt=”-D_GLIBCXX_USE_CXX11_ABI=0” --copt=-fvisibility=hidden //native_client:libdeepspeech.so //native_client:generate_trie

Are you skipping the tensorflow build step or does this command above cover that?
Lastly, are you using the master branch of the deepspeech repository?

Thanks!

[mdasari823]

Yes, I didn't need tensorflow library. I was compiling for deepspeech library. Did you make changes to following files as described in my document above?

tensorflow/lite/kernels/internal/BUILD,
third_party/aws/BUILD.bazel,
third_party/gpus/crosstool/BUILD.tpl

[forscience6]

Yes, I didn't need tensorflow library. I was compiling for deepspeech library. Did you make changes to following files as described in my document above?

tensorflow/lite/kernels/internal/BUILD,
third_party/aws/BUILD.bazel,
third_party/gpus/crosstool/BUILD.tpl

Yes, I've patched those files. I now tried following your instructions exactly and ran the following command:
bazel build --workspace_status_command=”bash native_client/bazel_workspace_status_cmd.sh” --config=monolithic --config=cuda -c opt --copt=-O3 --copt=”-std=gnu99” --copt=”-D_GLIBCXX_USE_CXX11_ABI=0” --copt=-fvisibility=hidden //native_client:libdeepspeech.so //native_client:generate_trie

I get the following error:
error.txt

[mdasari823]

Wait, are you building deepspeech now?

[forscience6]

Wait, are you building deepspeech now?

Yes, I started the whole process from the beginning. So basically I have installed all the dependencies, built bazel, created a swapfile, linked the natvie_client to the tensorflow directory, setup the configuration, and finally ran the build command above.

[mdasari823]

I am guessing it is not able to locate bazel_workspace_status_cmd.sh or
maybe some of the previous tensorflfow build is conflicting. Not sure man.

On Wed, Sep 4, 2019 at 6:41 PM michaelguo32 [e-mail]
wrote:

Wait, are you building deepspeech now?

Yes, I started the whole process from the beginning. So basically I have
installed all the dependencies, built bazel, created a swapfile, linked the
natvie_client to the tensorflow directory, and finally ran the build
command above.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mozilla/DeepSpeech/issues/2296?email_source=notifications&email_token=AFDCDNF3QKNNOXR45WC522DQH7XMNA5CNFSM4ILCL3Q2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD54MBYI#issuecomment-528007393,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFDCDNCEDGYQPVD3HXF4QTTQH7XMNANCNFSM4ILCL3QQ
.

--
Best Regards,
Mallesh Dasari
WINGS Lab, Computer Science
Stony Brook University
http://www3.cs.stonybrook.edu/~mdasari/
http://www3.cs.stonybrook.edu/~mdasari/

[reuben]

The error message is telling you exactly what's happening:

ERROR: invalid target format: '//native_client:bazel_workspace_status_cmd.sh”'; invalid target name 'bazel_workspace_status_cmd.sh”': target names may not contain non-printable characters: '\x80'

The problem is that weird quote mark there: https://apps.timwhitlock.info/unicode/inspect/hex/201D

\x80 is the second byte in its UTF-8 encoding. bazel is likely decoding that as ASCII. Just clean up your command line to remove any special characters like ” (command line double quotes should be ")

[forscience6]

The error message is telling you exactly what's happening:

ERROR: invalid target format: '//native_client:bazel_workspace_status_cmd.sh”'; invalid target name 'bazel_workspace_status_cmd.sh”': target names may not contain non-printable characters: '\x80'

The problem is that weird quote mark there: https://apps.timwhitlock.info/unicode/inspect/hex/201D

\x80 is the second byte in its UTF-8 encoding. bazel is likely decoding that as ASCII. Just clean up your command line to remove any special characters like ” (command line double quotes should be ")

You are right, it was the double quote that was causing the issue.

So I was able to compile deepspeech, but when I try to compile the language bindings I run into another error:
`

• svs@svs-desktop:~/DeepSpeech/native_client$ make deepspeech
• c++ -std=c++11 -o deepspeech pkg-config --cflags sox client.cc -Wl,--no-as-needed -Wl,-rpath,\$ORIGIN -L/home/svs/tensorflow/bazel-bin/native_client -ldeepspeech pkg-config --libs sox
• /usr/bin/ld: cannot find -ldeepspeech
• collect2: error: ld returned 1 exit status
• Makefile:22: recipe for target 'deepspeech' failed
• make: * [deepspeech] Error 1

`
Am I missing a file or is there a link missing?

UPDATE:
So the symbolic link to libdeepspeech.so is broken. I looked into the parent directory where the actual libdeepspeech.so is located( ~/.cache/bazel directory ), and wasn't able to find anything except for another broken symbolic link to libdeepspeech.so which is the same link so the search took me in a circle. So I'm guessing that the libdeepspeech.so file was never created except for an empty link to it.
Should I compile the whole thing again or is there a easy fix?

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.