spleeter-gpu broken

[swissbeats93]

I followed through with installing spleeter-gpu and when i try to run it this is the ouput I get.

(spleeter-gpu) swissbeats93@ApexHunter:~/Documents/Development$ spleeter separate -i spleeter/audio_example.mp3 -p spleeter:2stems -o output
INFO:spleeter:Loading audio b'spleeter/audio_example.mp3' from 0.0 to 600.0
INFO:spleeter:Audio data loaded successfully
Traceback (most recent call last):
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1356, in _do_call
return fn(*args)
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1341, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1429, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
(0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[{{node conv2d_7/Conv2D}}]]
[[strided_slice_13/_905]]
(1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[{{node conv2d_7/Conv2D}}]]
0 successful operations.
0 derived errors ignored.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/bin/spleeter", line 8, in
sys.exit(entrypoint())
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/spleeter/__main__.py", line 47, in entrypoint
main(sys.argv)
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/spleeter/__main__.py", line 41, in main
entrypoint(arguments, params)
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/spleeter/commands/separate.py", line 178, in entrypoint
output_naming=output_naming)
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/spleeter/commands/separate.py", line 132, in process_audio
for sample in prediction:
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 637, in predict
preds_evaluated = mon_sess.run(predictions)
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/tensorflow/python/training/monitored_session.py", line 754, in run
run_metadata=run_metadata)
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/tensorflow/python/training/monitored_session.py", line 1252, in run
run_metadata=run_metadata)
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/tensorflow/python/training/monitored_session.py", line 1353, in run
raise six.reraise(original_exc_info)
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/six.py", line 696, in reraise
raise value
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/tensorflow/python/training/monitored_session.py", line 1338, in run
return self._sess.run(args, *kwargs)
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/tensorflow/python/training/monitored_session.py", line 1411, in run
run_metadata=run_metadata)
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/tensorflow/python/training/monitored_session.py", line 1169, in run
return self._sess.run(args, **kwargs)
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 950, in run
run_metadata_ptr)
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1173, in _run
feed_dict_tensor, options, run_metadata)
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1350, in _do_run
run_metadata)
File "/home/swissbeats93/anaconda3/envs/spleeter-gpu/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1370, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
(0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[node conv2d_7/Conv2D (defined at /lib/python3.7/site-packages/spleeter/model/functions/unet.py:90) ]]
[[strided_slice_13/_905]]
(1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[node conv2d_7/Conv2D (defined at /lib/python3.7/site-packages/spleeter/model/functions/unet.py:90) ]]
0 successful operations.
0 derived errors ignored.

Errors may have originated from an input operation.
Input Source operations connected to node conv2d_7/Conv2D:
strided_slice_3 (defined at /lib/python3.7/site-packages/spleeter/model/__init__.py:195)

Input Source operations connected to node conv2d_7/Conv2D:
strided_slice_3 (defined at /lib/python3.7/site-packages/spleeter/model/__init__.py:195)

Original stack trace for 'conv2d_7/Conv2D':
File "/bin/spleeter", line 8, in
sys.exit(entrypoint())
File "/lib/python3.7/site-packages/spleeter/__main__.py", line 47, in entrypoint
main(sys.argv)
File "/lib/python3.7/site-packages/spleeter/__main__.py", line 41, in main
entrypoint(arguments, params)
File "/lib/python3.7/site-packages/spleeter/commands/separate.py", line 178, in entrypoint
output_naming=output_naming)
File "/lib/python3.7/site-packages/spleeter/commands/separate.py", line 132, in process_audio
for sample in prediction:
File "/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 619, in predict
features, None, ModeKeys.PREDICT, self.config)
File "/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1146, in _call_model_fn
model_fn_results = self._model_fn(features=features, *kwargs)
File "/lib/python3.7/site-packages/spleeter/model/__init__.py", line 392, in model_fn
return builder.build_predict_model()
File "/lib/python3.7/site-packages/spleeter/model/__init__.py", line 334, in build_predict_model
output_dict = self._build_output_dict()
File "/lib/python3.7/site-packages/spleeter/model/__init__.py", line 130, in _build_output_dict
self._params['model']['params'])
File "/lib/python3.7/site-packages/spleeter/model/functions/unet.py", line 173, in unet
return apply(apply_unet, input_tensor, instruments, params)
File "/lib/python3.7/site-packages/spleeter/model/functions/__init__.py", line 26, in apply
params=params)
File "/lib/python3.7/site-packages/spleeter/model/functions/unet.py", line 90, in apply_unet
conv1 = conv2d_factory(conv_n_filters[0], (5, 5))(input_tensor)
File "/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 634, in __call__
outputs = call_fn(inputs, *args, *kwargs)
File "/lib/python3.7/site-packages/tensorflow/python/keras/layers/convolutional.py", line 196, in call
outputs = self._convolution_op(inputs, self.kernel)
File "/lib/python3.7/site-packages/tensorflow/python/ops/nn_ops.py", line 1079, in __call__
return self.conv_op(inp, filter)
File "/lib/python3.7/site-packages/tensorflow/python/ops/nn_ops.py", line 635, in __call__
return self.call(inp, filter)
File "/lib/python3.7/site-packages/tensorflow/python/ops/nn_ops.py", line 234, in __call__
name=self.name)
File "/lib/python3.7/site-packages/tensorflow/python/ops/nn_ops.py", line 1953, in conv2d
name=name)
File "/lib/python3.7/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 1071, in conv2d
data_format=data_format, dilations=dilations, name=name)
File "/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(args, *kwargs)
File "/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3616, in create_op
op_def=op_def)
File "/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 2005, in __init__
self._traceback = tf_stack.extract_stack()

[aolney]

I'm not a dev here, but this looks like the kind of error you might get if you didn't have the right CUDA/cuDNN installed for TF 1.14.

[swissbeats93]

Interesting, I used their yml file to install the environment but I'll look into that now, I'll try running spleeter on my dev environment and post what I get

[Faylixe]

Sound like a Cuda installation issues. Closing.

We will soon provide a ready to use GPU version from conda forge.

[aidv]

@Faylixe I've looked around, and I might have missed it, but is the GPU version of spleeter ready from conda forge yet?

[aidv]

Never mind. I still haven't figured out why I can't GPU split.

[aolney]

I used pip. Something like this:

tensorflow-gpu==1.14.0
pandas==0.25.1
requests
pip
museval==0.3.0
musdb==0.3.1
norbert==0.2.1
spleeter
ffmpeg-python

Make sure you already have python 3.7 and ffmpeg installed

[aidv]

@aolney are those pip commands? How do I apply those things to my system?

[swissbeats93]

Write them to a text file (requirements.txt) and then do a pip install - r
requirements.txt

I'll try creating a conda yaml and see how it goes

On Wed, 04 Dec 2019, 16:55 aidv, [e-mail] wrote:

@aolney https://github.com/aolney are those pip commands? How do I
apply those things to my system?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/deezer/spleeter/issues/60?email_source=notifications&email_token=AHLHSDP7EKPWMW2QMJIUIEDQW7HGLA5CNFSM4JLAW7IKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF5PUJQ#issuecomment-561707558,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHLHSDKC5X5RFHRU53L4ZJDQW7HGLANCNFSM4JLAW7IA
.

[aidv]

@swissbeats93 @aolney it seems to work Nice!!!!!!!!!!!

[swissbeats93]

Not for me yet, I have Cuda 10.0 and Cuda 10.1 so there's some conflict there I guess

[aolney]

TF 1.14 aligns with CUDA 10.0, see this page.

Here's more information about installing TF.

You can build TF from source if you want to (first link) and use alternative CUDA versions.

However, If I were you, I'd purge CUDA and install version 10.0, which is appropriate for TF (first link) as well as Pytorch.

[swissbeats93]

I'll look into that when I have some time, it's quite busy for me at the moment or I might just wait for 20.04 LTS and fix everything then

[swissbeats93]

I did the Cuda 10.0 installation for those reasons but it was after I had already set up 10.1, everything else works on my system except for when I test spleeter-gpu. I'll try testing spleeter on my development environment and see if it works

[aidv]

I don't remember exactly what I tried prior to your suggestions, but I do remember that I

1. Uninstalled Cuda 10.1

2. Installed TF 1.14 from Conda shell using the following instrunction: https://www.tensorflow.org/install/gpu

[swissbeats93]

Update: I just tested it now after installing spleeter-gpu from conda forge and it's working perfectly

[xielangtc]

Update: I just tested it now after installing spleeter-gpu from conda forge and it's working perfectly

how to install and use spleeter-gpu ? some commads like:"conda install -c conda-forge spleeter-gpu"?