Wav2letter: Requirement of building wav2letter to install wav2letter python bindings

[tarang-jain]

Question

I am trying to run wav2vec 2.0 fine-tuning on a custom dataset. However, it requires wav2letter python bindings to be installed. So in order to install wav2letter python bindings, do I need to build wav2letter with all its dependencies?

Additional Context

I looked at this for building python bindings but there is no 'bindings' folder present beforehand in the wav2letter clone. I am new to wav2letter. Apologies if this question comes across as trivial.

[iwhitey]

Hello @tarang-jain, I also started with this two days ago, but you can git checkout e7c4d174ab581ce28df7cd3518ad936eaa752cea commit. As far as I see, this is the last commit where the structure of the project was compatible with the https://github.com/facebookresearch/wav2letter/wiki/Building-Python-bindings. After that they did ef022a9edc04c19300534d0f7d77b83aeb5f1adf which changed the project structure.

I was trying to do the same thing. and I am not quite sure what's the expected behavior of this building story. Building wav2letter (on latest commit on master) will not produce python-bindings related content - as a matter of fact, I built wav2letter "successfully" but make -j8 hasn't produced any of the Train, Test, and Decode binaries (the only target built was SerializeConvLM) and cmake was apparently happy with everything (but also hasn't reported anything except ArrayFire, flashlight and OpenMP dependencies, which is weird). I'm not familiar with cmake syntax nor cmake/make internals at all, and I didn't dive in to see which commits could be responsible for this behavior (or maybe it's my system's fault, not sure). There also aren't any errors or warnings which would imply why the build finished successfully without producing binaries. Maybe because 1 or 2 tests in make test fail for ArrayFire and flashlight, but it could also be that some things simply aren't supported - they haven't documented that all ArrayFire and flashlight tests must pass in order to successfully build wav2letter, at least I haven't seen this note :).

[tarang-jain]

Hello @iwhitey , thanks for the help. I did git checkout e7c4d17 and found the structure that was compatible with https://github.com/facebookresearch/wav2letter/wiki/Building-Python-bindings.

I did not build wav2letter completely (because in my current working environment, it is really difficult to build all it's dependencies eg. flashlight). I followed the steps in https://github.com/facebookresearch/wav2letter/wiki/Building-Python-bindings and got the following error.

Obtaining file:///home/tarang/wav2letter/bindings/python
Installing collected packages: wav2letter
  Running setup.py develop for wav2letter
    Complete output from command /usr/bin/python3 -c "import setuptools, tokenize;__file__='/home/tarang/wav2letter/bindings/python/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps --user --prefix=:
    running develop
    running egg_info
    writing wav2letter.egg-info/PKG-INFO
    writing dependency_links to wav2letter.egg-info/dependency_links.txt
    writing top-level names to wav2letter.egg-info/top_level.txt
    reading manifest file 'wav2letter.egg-info/SOURCES.txt'
    writing manifest file 'wav2letter.egg-info/SOURCES.txt'
    running build_ext
    make: *** No targets specified and no makefile found.  Stop.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/home/tarang/wav2letter/bindings/python/setup.py", line 109, in <module>
        zip_safe=False,
      File "/home/tarang/.local/lib/python3.6/site-packages/setuptools/__init__.py", line 163, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/home/tarang/.local/lib/python3.6/site-packages/setuptools/command/develop.py", line 38, in run
        self.install_for_development()
      File "/home/tarang/.local/lib/python3.6/site-packages/setuptools/command/develop.py", line 140, in install_for_development
        self.run_command('build_ext')
      File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/home/tarang/wav2letter/bindings/python/setup.py", line 48, in run
        self.build_extensions()
      File "/home/tarang/wav2letter/bindings/python/setup.py", line 91, in build_extensions
        ["cmake", "--build", "."] + build_args, cwd=self.build_temp
      File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j4']' returned non-zero exit status 2.

    ----------------------------------------
Command "/usr/bin/python3 -c "import setuptools, tokenize;__file__='/home/tarang/wav2letter/bindings/python/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps --user --prefix=" failed with error code 1 in /home/tarang/wav2letter/bindings/python/

Do you know what this indicates? Does it mean that I need to build wav2letter completely before trying to build python bindings?

[tarang-jain]

@iwhitey Ok I was able to install python bindings by removing the build directory before executing thepip install -e . command. Thanks for the help.