Cellprofiler: Still can't manage to conda-install the cellprofiler package

[SandorAlbert]

So it's been two entire work days and I'm still very frustrated that it won't install properly. Maybe I'm doing something wrong, maybe there isn't enough documentation about how to properly install.

It seems like https://github.com/CellProfiler/CellProfiler/wiki/Conda-Installation is outdated. Following the recommended steps won't work and results in a mess. I had to manually install javabridge, mysql_python, wxpython, mysqlclient (the whls from https://www.lfd.uci.edu/~gohlke/pythonlibs/), it tells me for jupyter to work I have to upgrade 'pyzmq' which has to stay downgraded in order to work for cellprofiler. It tells me to install Oracle Java Runtime Environment (JRE) 1.6 and when I finally manage to install it somehow (for Python 2.7), it tells me that cellprofiler.pipeline could not be found, among other errors. I didn't manage to install it for Python 3.7 neither.

I read you are currently working on a Python 3 version and would therefore highly appreciate if you could work on the documentation a little bit.
If there's anything I overlooked, my apologies.

Thank you and kind regards

[SandorAlbert]

Another try: this is what I did step by step:
1) Install Miniconda, Java JDK, Microsoft Visual C++ Compiler for Python 2.7
2) Create an environment.yml file on my Desktop with following content:

# run: conda env create -f environment.yml
# run: conda env update -f environment.yml
# run: conda env remove -n cellprofiler
name: cellprofiler
# in order of priority: highest (top) to lowest (bottom)
channels:
  - anaconda
  - bioconda
  - cyclus # java-jdk for windows
  - conda-forge # libxml2 for windows
dependencies:
  - appdirs
  - boto3
  - cython
  - h5py
  - ipywidgets
  - java-jdk
  - joblib
  - jupyter
  - libtiff
  - libxml2
  - libxslt
  - lxml
  - packaging
  - pillow
  - pip
  - python=2
  - pyzmq=15.3.0
  - mahotas
  - matplotlib!=2.1.0,>2.0.0
  - mysqlclient
  - numpy
  - raven
  - requests
  - scikit-image>=0.13
  - scikit-learn
  - scipy
  - sphinx
  - tifffile
  - wxpython
  - pip:
    - cellh5
    - centrosome
    - inflect
    - prokaryote==2.4.0
    - javabridge==1.0.15
    - python-bioformats==1.4.0
    - git+https://github.com/CellProfiler/[email protected]

3) CMD:

• cd Desktop
• conda env create -f environment.yml

Result: environment is created, but everything that came after "- pip:" in the environment.yml was not installed:

    - pip:
    - cellh5
    - centrosome
    - inflect
    - prokaryote==2.4.0
    - javabridge==1.0.15
    - python-bioformats==1.4.0
    - git+https://github.com/CellProfiler/[email protected]

So:
4) I tried to manually install "git+https://github.com/CellProfiler/[email protected]" by using
CMD: pip install git+https://github.com/CellProfiler/[email protected]

Result:

• Failed building wheel for javabridge
• Failed building wheel for mysqlclient

So:
5) Install "javabridge‑1.0.18‑cp27‑cp27m‑win_amd64.whl" via https://www.lfd.uci.edu/~gohlke/pythonlibs/
-> Success
6) Install "mysqlclient-1.4.4-cp27-cp27m-win_amd64" via https://www.lfd.uci.edu/~gohlke/pythonlibs/
-> Success
7) pip install git+https://github.com/CellProfiler/[email protected]

Result:

• Failed building wheel for mysqlclient (__mysql.c(29) : fatal error C1083: Cannot open include file: 'mysql.h': No such file or directory_)

So:
8) pip install mysqlclient
->"_Requirement already satisfied: mysqlclient in c:\users\albert\appdata\local\continuum\miniconda3\envs\cellprofiler\lib\site-packages (1.4.4)_"
9) pip install MySQL_python-1.2.5-cp27-none-win_amd64.whl via https://www.lfd.uci.edu/~gohlke/pythonlibs/
->Success
10) pip install git+https://github.com/CellProfiler/[email protected]

Result:

• Failed building wheel for mysqlclient (__mysql.c(29) : fatal error C1083: Cannot open include file: 'mysql.h': No such file or directory_)

11) conda install -c anaconda mysql-connector-python
-> Success
12) pip install git+https://github.com/CellProfiler/[email protected]

Result:

• Failed building wheel for mysqlclient (__mysql.c(29) : fatal error C1083: Cannot open include file: 'mysql.h': No such file or directory_)

Even though I install the mysqlclient wheel it keeps downloading it when pip installing git+https://github.com/CellProfiler/[email protected]. At this point I don't know what else to do.

Thanks!

[bethac07]

The version of mysqlclient that's available on the Gohlke wheel site is not the right one for 3.1.8- try 3.1.9, I believe that allows mysqlclient 1.4+ (and is otherwise nearly entirely the same as 3.1.8)

[SandorAlbert]

The version of mysqlclient that's available on the Gohlke wheel site is not the right one for 3.1.8- try 3.1.9, I believe that allows mysqlclient 1.4+ (and is otherwise nearly entirely the same as 3.1.8)

If finally works now:

• conda env create -f environment.yml
• pip install mysqlclient-1.4.4-cp27-cp27m-win_amd64.whl
• pip install javabridge-1.0.18-cp27-cp27m-win_amd64
• pip install git+https://github.com/CellProfiler/[email protected]

Thank you very much!

*Just a side note: when trying to open Jupyter Notebook I get "AttributeError: type object 'IOLoop' has no attribute 'initialized'". It's always been this way after trying to install CellProfiler.

[SandorAlbert]

I did come this far before and it gave me the exact same error (among others). It just can't find certain attributes and package contents.

import cellprofiler
pipeline = cellprofiler.pipeline.Pipeline()
pipeline.load(r"pipelines\Website\ExampleYeastColonies.cppipe")
measurements = pipeline.run()

AttributeError: 'module' object has no attribute 'pipeline'

Also, when calling "cellprofiler" in the command prompt:

AttributeError: 'module' object has no attribute 'AboutDialogInfo'

Installing wxPython_common-3.0.2.0-py2-none-any.whl (from https://www.lfd.uci.edu/~gohlke/pythonlibs/) returns:

ImportError: No module named wx

[bethac07]

Can I ask why you're installing from source rather than running built?

These sorts of errors are hard to troubleshoot remotely; it works in our group to have CP3.1.9 installed from source on both mac and windows (though I can say I don't think we currently in the group have anyone running it from Conda).

At a minimum, can you post your pip freeze inside your CellProfiler environment?

[emmenlau]

I'm slightly frustrated with how hard it has become to build CellProfiler from source. I've just tried to build the latest stable release on Linux based on https://github.com/CellProfiler/CellProfiler/wiki/Source-Installation-(Ubuntu-18.04-LTS) and https://github.com/CellProfiler/CellProfiler/wiki/Conda-Installation and neither of them works. Furthermore the different instructions seem to contradict themselves. Package versions listed in the wiki do not match versions given in https://github.com/CellProfiler/CellProfiler/blob/master/setup.py which makes me wonder why they co-exist in the first place.

Would it be possible to make a single, complete instruction set for installing CellProfiler from source? Ideally include instructions for Linux, but if that is not possible I can abstract them from other platforms.

[bethac07]

I apologize; stuff has been changing a lot, especially as we are halfway through the transition between Python2 and Python3, and we haven't been as diligent as we should be about keeping the wiki up to date. I will try to get to that in the next week or so; what tag specifically are you trying to build? I recommend 3.1.9.

[emmenlau]

Hi Beth! Please forgive my slightly frustrated rant from two days ago! In the same night I got better mileage with the conda install on Linux, and I'll try to update the build instructions myself soon. Not too many packages are out of sync between setup.py and the conda wiki page, so I could finally get something working with moderate effort. Another point of confusion was exactly the mention of Python 2 in the docs compared to Python 3 in setup.py. Its nice to learn that you're aiming for Python 3! Would you recommend developers already try CellProfiler with Python 3 when using master HEAD? My incentive is that I'd rather live on the edge for a few months than re-do the build instructions in a few months again. But if current HEAD / Python 3 is known to have blocker issues then that would change my mind :-)

[bethac07]

Yeah, master (which is Python3) is currently pretty unstable. For dev
work, if you don't mind being a super awesome beta tester, please feel free
to use it, it's close and we'd love your feedback! For production, it's
probably safer right now (for at least the next couple weeks) to pin to
3.1.9, which is Python 2.

On Fri, Oct 4, 2019 at 3:48 PM Mario Emmenlauer [e-mail]
wrote:

Hi Beth! Please forgive my slightly frustrated rant from two days ago! In
the same night I got better mileage with the conda install on Linux, and
I'll try to update the build instructions myself soon. Not too many
packages are out of sync between setup.py and the conda wiki page, so I
could finally get something working with moderate effort. Another point of
confusion was exactly the mention of Python 2 in the docs compared to
Python 3 in setup.py. Its nice to learn that you're aiming for Python 3!
Would you recommend developers already try CellProfiler with Python 3 when
using master HEAD? My incentive is that I'd rather live on the edge for a
few months than re-do the build instructions in a few months again. But if
current HEAD / Python 3 is known to have blocker issues then that would
change my mind :-)

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/CellProfiler/CellProfiler/issues/3838?email_source=notifications&email_token=ABTI724G5S54ZCG6PQIFP6DQM6MY3A5CNFSM4IVXJWBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAMWXDI#issuecomment-538536845,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABTI723BEAVLOB2IZAROA33QM6MY3ANCNFSM4IVXJWBA
.

--
Beth Cimini, PhD
Computational Biologist, Imaging Platform
Broad Institute
415 Main St Room 5011
Cambridge, MA 02142
Current office number- [phone]
Alternate office number- [phone]

[nichollsfj]

Hi Beth,
Is this still your advice, to pin to 3.1.9 for production? I have ~3 months worth of image analysis that I need to run on our Linux cluster but I'm worried about starting it in 3.1.9/python2 if I might have to switch part way through and start again.
I ran the last lot on cellprofiler 2.2 but started having issues with that so thought I'd try something more recent...

[bethac07]

3.1.9 will be stable for the forseeable (though keep in mind Python EOL); 4.0.0rc1 is NOT production ready yet.

[nichollsfj]

I'm trying to switch branches using "git checkout v3.1.9" but getting the error that it's not a git repository... Do I have the name wrong? I tried with and without "v"

[bethac07]

Have you fetched recently? Or are you maybe in the wrong directory (I do
that maybe twice a week!)? The below works fine in my hands.

git checkout v3.1.9

On Wed, Nov 6, 2019 at 12:00 PM nichollsfj [e-mail] wrote:

I'm trying to switch branches using "git checkout v3.1.9" but getting the
error that it's not a git repository... Do I have the name wrong? I tried
with and without "v"

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/CellProfiler/CellProfiler/issues/3838?email_source=notifications&email_token=ABTI727OFCK7CYPDG2VJK6DQSLZ3XA5CNFSM4IVXJWBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDHH53A#issuecomment-550403820,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABTI72ZWGWUJYUSD5YQUOCLQSLZ3XANCNFSM4IVXJWBA
.

--
Beth Cimini, PhD
Computational Biologist, Imaging Platform
Broad Institute
415 Main St Room 5011
Cambridge, MA 02142
Current office number- [phone]
Pronouns - She/her/hers
I will sometimes send or respond to emails outside of my local office
hours, but I never expect responses outside of your local office hours.

[nichollsfj]

Thanks Beth. Not sure what was going on but it seems fine now I've sorted out my environment file

[SSICreative83]

Hi,bethac07,

when I run cellprofiler, I can login GUI, but Identifyprimaryobject and Identifysecondaryobject modules cannot be imported with the following issue

cjj@wrs-super01:~$ cellprofiler

stauffer@wrs-super01:~$ cellprofiler
Could not load cellprofiler.modules.identifyprimaryobjects
Traceback (most recent call last):
File "/home/stauffer/software/CellProfiler-3.1.9/CellProfiler/cellprofiler/modules/__init__.py", line 325, in add_module
m = __import__(mod, globals(), locals(), ['__all__'], 0)
File "/home/stauffer/software/CellProfiler-3.1.9/CellProfiler/cellprofiler/modules/identifyprimaryobjects.py", line 9, in
import centrosome.threshold
File "/home/stauffer/.local/lib/python2.7/site-packages/centrosome/threshold.py", line 82
*kwargs,
^
SyntaxError: invalid syntax
Could not load cellprofiler.modules.identifysecondaryobjects
Traceback (most recent call last):
File "/home/stauffer/software/CellProfiler-3.1.9/CellProfiler/cellprofiler/modules/__init__.py", line 325, in add_module
m = __import__(mod, globals(), locals(), ['__all__'], 0)
File "/home/stauffer/software/CellProfiler-3.1.9/CellProfiler/cellprofiler/modules/identifysecondaryobjects.py", line 9, in
from cellprofiler.modules import _help, threshold
File "/home/stauffer/software/CellProfiler-3.1.9/CellProfiler/cellprofiler/modules/threshold.py", line 21, in
import centrosome.threshold
File "/home/stauffer/.local/lib/python2.7/site-packages/centrosome/threshold.py", line 82
*kwargs,
^
SyntaxError: invalid syntax
Could not load cellprofiler.modules.measureimagequality
Traceback (most recent call last):
File "/home/stauffer/software/CellProfiler-3.1.9/CellProfiler/cellprofiler/modules/__init__.py", line 325, in add_module
m = __import__(mod, globals(), locals(), ['__all__'], 0)
File "/home/stauffer/software/CellProfiler-3.1.9/CellProfiler/cellprofiler/modules/measureimagequality.py", line 9, in
import centrosome.threshold
File "/home/stauffer/.local/lib/python2.7/site-packages/centrosome/threshold.py", line 82
*kwargs,
^
SyntaxError: invalid syntax
Could not load cellprofiler.modules.threshold
Traceback (most recent call last):
File "/home/stauffer/software/CellProfiler-3.1.9/CellProfiler/cellprofiler/modules/__init__.py", line 325, in add_module
m = __import__(mod, globals(), locals(), ['__all__'], 0)
File "/home/stauffer/software/CellProfiler-3.1.9/CellProfiler/cellprofiler/modules/threshold.py", line 21, in
import centrosome.threshold
File "/home/stauffer/.local/lib/python2.7/site-packages/centrosome/threshold.py", line 82
*kwargs,
^
SyntaxError: invalid syntax
could not load these modules: cellprofiler.modules.identifyprimaryobjects,cellprofiler.modules.identifysecondaryobjects,cellprofiler.modules.measureimagequality,cellprofiler.modules.threshold
10:10:48 PM: Debug: ClientToScreen cannot work when toplevel window is not shown

could you advise me about how to fix it?

Thanks,

[SSICreative83]

python2.7 -m pip freeze
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
altgraph==0.17
asn1crypto==0.24.0
backports-abc==0.5
backports.functools-lru-cache==1.6.1
backports.shutil-get-terminal-size==1.0.0
bioformats==0.1.15
boto3==1.13.19
botocore==1.16.19
-e git+https://github.com/CellProfiler/CellProfiler.git@082930ea95add7b72243a4fa3d39ae5145995e9c#egg=CellProfiler
CellProfiler-Analyst @ file:///home/stauffer/software/CellProfiler-Analyst
centrosome @ file:///home/stauffer/software/CellProfiler/centrosome
certifi==2020.4.5.1
cftime==1.1.3
chardet==3.0.4
cloudpickle==1.3.0
configparser==4.0.2
contextlib2==0.6.0.post1
cryptography==2.1.4
cycler==0.10.0
dask==1.2.2
decorator==4.4.2
deprecation==2.1.0
dis3==0.1.3
Django==1.11.29
docutils==0.15.2
enum34==1.1.10
Fabric==1.14.0
funcsigs==1.0.2
future==0.18.2
futures==3.3.0
h5py==2.10.0
idna==2.9
image==1.5.32
importlib-metadata==1.6.0
inflect==3.0.2
ipaddress==1.0.17
ipython==5.10.0
ipython-genutils==0.2.0
javabridge==1.0.19
jmespath==0.10.0
joblib==0.14.1
kiwisolver==1.1.0
kwargs-only==1.1.1
mahotas==1.4.9
matplotlib==2.2.5
mock==3.0.5
mysqlclient==1.4.6
networkx==2.2
neuralpy==1.3.0
nose==1.3.7
numpy==1.15.4
olefile==0.45.1
packaging==20.4
pandas==0.24.2
paramiko==2.0.0
pathlib2==2.3.5
pexpect==4.8.0
pickleshare==0.7.5
Pillow==6.2.2
prokaryote==2.4.1
prompt-toolkit==1.0.18
ptyprocess==0.6.0
pyasn1==0.4.2
pycrypto==2.6.1
pyfaidx==0.5.8
Pygments==2.5.2
PyInstaller==3.6
pyparsing==2.4.7
python-bioformats==1.5.2
python-dateutil==2.8.1
pytz==2020.1
PyVCF==0.6.8
PyWavelets==1.0.3
pyzmq==15.3.0
raven==6.10.0
requests==2.23.0
s3transfer==0.3.3
scandir==1.10.0
scikit-image==0.14.0
scikit-learn==0.20.4
scipy==1.1.0
seaborn==0.9.1
simplegeneric==0.8.1
singledispatch==3.4.0.3
six==1.15.0
subprocess32==3.2.7
toolz==0.10.0
tornado==5.1.1
traitlets==4.3.3
urllib3==1.25.9
verlib==0.1
wcwidth==0.1.9
wxPython==3.0.2.0
wxPython-common==3.0.2.0
zipp==1.2.0