Spacy: Packaging spacy with pyinstaller: ImportError: cannot import name _custom_kernels

Created on 19 Feb 2020 · 4Comments · Source: explosion/spaCy

Overview

I'm packaging a python script which imports spacy with pyinstaller https://github.com/pyinstaller/pyinstaller into an executable. I get the error in the title when running the executable.

Environment

Operating System: Linux-4.15.0-88-generic-x86_64-with-Ubuntu-18.04-bionic
Python Version Used: 3.6.9
spaCy Version Used: 2.2.3
Spacy location: /home/user/spacy-pyinstaller/venv/lib/python3.6/site-packages/spacy
pyinstaller Version Used: 4.0.dev0+a1f92c6a08

Reproduction steps

All necessary files for reproduction are available at https://github.com/michaelhochleitner/spacy-pyinstaller .

I'm packaging the following python script.

import spacy

I use the following command to package the script.

pyinstaller nlp.spec

Here is my .spec file:

# -*- mode: python ; coding: utf-8 -*-

block_cipher = None


a = Analysis(['nlp.py'],
             pathex=['/home/user/spacy-pyinstaller'],
             binaries=[],
             datas=[],
             hiddenimports=['srsly.msgpack.util','cymem','cymem.cymem','preshed.maps','thinc.linalg','murmurhash'],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher,
             noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          [],
          exclude_binaries=True,
          name='nlp',
          debug=False,
          bootloader_ignore_signals=False,
          strip=False,
          upx=True,
          console=True )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=False,
               upx=True,
               upx_exclude=[],
               name='nlp')

When running

./dist/nlp/nlp

I get the following output

Traceback (most recent call last):
  File "nlp.py", line 1, in <module>
    import spacy
  File "/home/mh/spacy-pyinstaller/venv/lib/python3.6/site-packages/PyInstaller/loader/pyimod03_importers.py", line 489, in exec_module
    exec(bytecode, module.__dict__)
  File "spacy/__init__.py", line 10, in <module>
  File "/home/mh/spacy-pyinstaller/venv/lib/python3.6/site-packages/PyInstaller/loader/pyimod03_importers.py", line 489, in exec_module
    exec(bytecode, module.__dict__)
  File "thinc/neural/__init__.py", line 4, in <module>
  File "/home/mh/spacy-pyinstaller/venv/lib/python3.6/site-packages/PyInstaller/loader/pyimod03_importers.py", line 489, in exec_module
    exec(bytecode, module.__dict__)
  File "thinc/neural/_classes/model.py", line 11, in <module>
  File "/home/mh/spacy-pyinstaller/venv/lib/python3.6/site-packages/PyInstaller/loader/pyimod03_importers.py", line 489, in exec_module
    exec(bytecode, module.__dict__)
  File "thinc/neural/train.py", line 7, in <module>
  File "optimizers.pyx", line 14, in init thinc.neural.optimizers
  File "ops.pyx", line 24, in init thinc.neural.ops
ImportError: cannot import name _custom_kernels
[16858] Failed to execute script nlp

I don't know if this issue should be in the pyinstaller project or the spacy project. So I'm opening issues in both.

install

Source

michaelhochleitner

Most helpful comment

@michaelhochleitner this can be closed - it's an issue with PyInstaller, not spacy. I've left an answer on the SO question.

Legorooj on 23 Feb 2020

👍2

All 4 comments

@michaelhochleitner this can be closed - it's an issue with PyInstaller, not spacy. I've left an answer on the SO question.

Legorooj on 23 Feb 2020

👍2

Thanks for the note, @Legorooj !

svlandeg on 23 Feb 2020

@svlandeg you're welcome. All spacy needed is what is called a PyInstaller "hook", which tells PyInstaller how to package the library. I've written up one that's a bit of a hack, and posted that on SO. Hopefully it'll work without one soon; I've got a PR on the way that will massively decrease the need for hooks.

Legorooj on 23 Feb 2020

👍1

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.