Poetry: can't install tensorflow==2.0.0rc0 with poetry (python3.6.5)

[zachmoshe]

• [x] I am on the latest Poetry version.
  
  
  
  • (I'm actually on 1.0.0b1)
  
  
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: MacOS 10.14.5
• Poetry version: 1.0.0b1
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/zachmoshe/36ec3cc128545aa8ab49a2d4dcd9cd52

Issue

1. Open a new project (poetry new .)
2. run poetry add tensorflow==2.0.0rc0

Fails because it can't install functools32 (for python2 only).
debug logs - https://gist.github.com/zachmoshe/8bd72026dd727380f4b88dc4ef55ff4f

Installing previous versions works and installing rc0 with pip also works.
I understand functools32 is a new dependency and that somehow we try to install a version which only work with python2. I just can't understand why do we try this specific version.

Any solution or a workaround?

Thanks!

[Caligatio]

It appears that tensorflow 2.0.0-rc0's setup.py dynamically changes its requirements depending on the environment by executing code. My guess is that Poetry does not handle this case well for some reason as it appears ALL of the Python2-linked dependencies are being grabbed.

I've never really played with the ast module and I'm only taking guesses here...

[zachmoshe]

Makes sense, but I'm not familiar with the internals as well.. definitely not a standard setup.py.
Can anyone suggest at least a workaround until a real solution is deployed?

[mike239x]

Yeap, I am yet another one with the same issue.
@zachmoshe just use pip for now?
I also was thinking - one could get the exact versions one needs from the pip-install (or from $ poetry debug:resolve tensorflow==2.0.0rc0), and then hard code them into poetry.lock file and run $ poetry install. I sadly do not know how exactly to operate directly on the lock-file though...

[aflc]

I found some reasons that functools32 dependency found even if Python3 projects you have:

1. poetry find a dependencies from PyPI's "requires_dist", and tensorflow 2.0.0rc0 or rc1 has functools32 in it:

https://pypi.org/ pypi/tensorflow/2.0.0rc0/json:

"requires_dist": [
      "absl-py (>=0.7.0)",
      "astor (>=0.6.0)",
      "gast (>=0.2.0)",
      "google-pasta (>=0.1.6)",
      "keras-applications (>=1.0.8)",
      "keras-preprocessing (>=1.0.5)",
      "numpy (<2.0,>=1.16.0)",
      "opt-einsum (>=2.3.2)",
      "six (>=1.10.0)",
      "protobuf (>=3.6.1)",
      "tb-nightly (<1.15.0a20190807,>=1.15.0a20190806)",
      "tf-estimator-nightly (<1.14.0.dev2019080602,>=1.14.0.dev2019080601)",
      "termcolor (>=1.1.0)",
      "wrapt (>=1.11.1)",
      "grpcio (>=1.8.6)",
      "wheel",
      "mock (>=2.0.0)",
      "functools32 (>=3.2.3)",
      "backports.weakref (>=1.0rc1); python_version < \"3.4\"",
      "enum34 (>=1.1.6); python_version < \"3.4\""
    ],

ref: https://github.com/sdispater/poetry/blob/cb8dc0d94e68042f120441b4e30d89c92598b40e/poetry/repositories/pypi_repository.py#L286

So at first, we need to modify tensorflow package metadata. I think it's a tensorflow team matter.

1. Even if requires_dist is null, poetry tries to get dependencies from universal bdist_wheel. and tensorflow doesn't have a universal wheel, it chooses a first wheel to inspect (it's a cp27 wheel).

ref: https://github.com/sdispater/poetry/blob/cb8dc0d94e68042f120441b4e30d89c92598b40e/poetry/repositories/pypi_repository.py#L350

So poetry need to be fixed these logics, IMO, it should respect Python version of venv.

[gorjanradevski]

Is there a particular workaround at the moment so that TF 2.0 can be installed?

[sdispater]

Unfortunately, this is not on Poetry's end. If you take a look at https://pypi.org/pypi/tensorflow/json in the requires_dist section of the info property, functools32 is specified as a dependency without any environment marker so Poetry rightly assumes that it's an unconditional dependency.

So, this issue must be fixed upstream by the tensorflow maintainers

[aflc]

So, this issue must be fixed upstream by the tensorflow maintainers

https://github.com/tensorflow/tensorflow/pull/32758
It may fixes the issue, but need to wait until 2.1 release.

[ColonelThirtyTwo]

so Poetry rightly assumes

Uh, if pip installs it fine, but poetry screws up, is it really "rightly"?

[grisaitis]

poetry add tensorflow@^2.1.0rc0

works for me

tensorflow 2.1.0 should be released soon.

[Magnati]

In addition to @grisaitis answer
I had to fulfill some other prerequisites:

• poetry > 0.12 (used 1.0.0)
• python < 3.8 (tensorflow does support only up to 3.7, yeah a no brainer :P)

then his command poetry add tensorflow@^2.1.0rc0 worked for me too.

[mjlbach]

I'm also running into problems due to this this: It seems to be failing to pull in dependencies matching *

❯ poetry add [email protected]

Updating dependencies
Resolving dependencies... (0.1s)

[SolverProblemError]
Because tf-nightly (2.2.0.dev20200202) depends on tf-estimator-nightly (*) which doesn't match any versions, tf-nightly is forbidden.
So, because model_repo depends on tf-nightly (2.2.0-alpha.20200202), version solving failed.

If you look in:

at https://pypi.org/pypi/tf-nightly/json in the requires_dist section of the info property you can see that tf-estimator-nightly has no specified version.

One workaround is to explicitly install tf-estimator-nightly, but it would be nice if poetry could patch around this for now...

[mohelm]

@mjlbach : i could not get this work. is the succession of commands you are proposing

poetry add tf-estimator-nightly
poetry add tf-nightly

The first one works, the second one does not. I get this error:

[EnvCommandError]
Command ['/Users/moritz.helm/Projects/recsys/oyster/.venv/bin/pip', 'install', '--no-deps', 'tf-nightly==2.2.0.dev20200202'] errored with the following return code 1, and output:
ERROR: Could not find a version that satisfies the requirement tf-nightly==2.2.0.dev20200202 (from versions: none)
ERROR: No matching distribution found for tf-nightly==2.2.0.dev20200202

[mjlbach]

@mohelm I don't think the release I used in my specific command is available anymore. Using the commands

poetry add tf-estimator-nightly
poetry add tf-nightly

I was able to install tf-nightly on a fresh vm.

[mohelm]

@mjlbach : Many thanks! Do you remember the Python version? I am actually trying this with Python 3.8.

[mjlbach]

@mohelm Ah! That's your problem, tensorflow is only packaged for 3.6/3.7 currently (I believe)

[mohelm]

Hm, these nightlies should work with 3.8 I believe.

Michael Lingelbach [e-mail] schrieb am Do., 26. März
2020, 21:21:

@mohelm https://github.com/mohelm Ah! That's your problem, tensorflow
is only packaged for 3.6/3.7 currently (I believe)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/python-poetry/poetry/issues/1330#issuecomment-604664646,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEJC4NK7BXIEI35QTZDTIV3RJO2NRANCNFSM4IPQRYUA
.

[mjlbach]

Hm, these nightlies should work with 3.8 I believe. Michael Lingelbach [e-mail] schrieb am Do., 26. März 2020, 21:21:
…
@mohelm https://github.com/mohelm Ah! That's your problem, tensorflow is only packaged for 3.6/3.7 currently (I believe) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1330 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEJC4NK7BXIEI35QTZDTIV3RJO2NRANCNFSM4IPQRYUA .

You're correct, I just managed to get tf-nightly installed on 3.8... Not sure why you are having the issue.

[mohelm]

Interesting. I tried on Mac, but I also saw just now the caveats (go to
caveats section) here

https://github.com/tensorflow/tensorflow/releases

Which I guess do apply??

May I ask you on which operating system you succeeded?

Michael Lingelbach [e-mail] schrieb am Do., 26. März
2020, 21:46:

Hm, these nightlies should work with 3.8 I believe. Michael Lingelbach
[email protected] schrieb am Do., 26. März 2020, 21:21:
… <#m_-403593830552407337_m_-732227986706241720_>
@mohelm https://github.com/mohelm https://github.com/mohelm Ah! That's
your problem, tensorflow is only packaged for 3.6/3.7 currently (I believe)
— You are receiving this because you were mentioned. Reply to this email
directly, view it on GitHub <#1330 (comment)
https://github.com/python-poetry/poetry/issues/1330#issuecomment-604664646>,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEJC4NK7BXIEI35QTZDTIV3RJO2NRANCNFSM4IPQRYUA
.

You're correct, I just managed to get tf-nightly installed on 3.8... Not
sure why you are having the issue.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/python-poetry/poetry/issues/1330#issuecomment-604676625,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEJC4NOBQNBYRXXLXIC7QMTRJO5MBANCNFSM4IPQRYUA
.

[mjlbach]

@mohelm nixOS, works on ubuntu as well

[mohelm]

Awesome will try tomorrow thanks.

Michael Lingelbach [e-mail] schrieb am Do., 26. März
2020, 21:51:

@mohelm https://github.com/mohelm nixOS, works on ubuntu as well

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/python-poetry/poetry/issues/1330#issuecomment-604679016,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEJC4NLQ5YC7D4FRHLIDOJDRJO56PANCNFSM4IPQRYUA
.

[logicpeters]

This is also a problem with Tensorflow 1.15, and some python libraries require the 1.x version of Tensorflow.

[ortsaCniveK]

For anyone having similar issues, I'd recommend trying poetry shell then pip install -U pip to upgrade pip. I was having issues with poetry finding [email protected], and found that the poetry virtualenv pip was on version 17, even though mine was on 20.

[lmarsden]

This is also a problem with Tensorflow 1.15, and some python libraries require the 1.x version of Tensorflow.

I also ran into this issue with Tensorflow 1.15.0, but Tensorflow 1.14.0 installs OK.

[tsabsch]

Installing Tensorflow 1.15.2 or 1.15.3 works fine for me.