Pyyaml: Cannot install/build on Python 3.7-dev

[hugovk]

aws-cli depends on PyYAML>=3.10,<=3.12.

However, when running on Python 3.7-dev on Travis CI, PyYAML cannot be built:

    ext/_yaml.c:24215:11: error: ‘PyThreadState’ has no member named ‘exc_type’
         tstate->exc_type = local_type;
               ^
    ext/_yaml.c:24216:11: error: ‘PyThreadState’ has no member named ‘exc_value’
         tstate->exc_value = local_value;
               ^
    ext/_yaml.c:24217:11: error: ‘PyThreadState’ has no member named ‘exc_traceback’
         tstate->exc_traceback = local_tb;
               ^
    error: command 'gcc' failed with exit status 1

https://travis-ci.org/hugovk/aws-cli/jobs/335698944#L1466

This also affects other projects which depend on aws-cli, like pika:

  ext/_yaml.c: In function ‘__Pyx__ExceptionSave’:
  ext/_yaml.c:24143:19: error: ‘PyThreadState’ has no member named ‘exc_type’
       *type = tstate->exc_type;
                     ^
  ext/_yaml.c:24144:20: error: ‘PyThreadState’ has no member named ‘exc_value’
       *value = tstate->exc_value;
                      ^
  ext/_yaml.c:24145:17: error: ‘PyThreadState’ has no member named ‘exc_traceback’
       *tb = tstate->exc_traceback;
                   ^
  ext/_yaml.c: In function ‘__Pyx__ExceptionReset’:
  ext/_yaml.c:24152:22: error: ‘PyThreadState’ has no member named ‘exc_type’
     member named ‘exc_traceback’
       tstate->exc_traceback = local_tb;
             ^
  error: command 'gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for PyYAML
  Running setup.py clean for PyYAML
Failed to build PyYAML

https://travis-ci.org/pika/pika/jobs/335516418

Any idea what's up?

[lukebakken]

There may be a workaround in aws/aws-cli#2290

Pinning awscli is a valid workaround, at least for Pika.

[The-Compiler]

I think the Cython code just needs to be regenerated to be compatible with Python 3.7? Installing from git+https://github.com/yaml/pyyaml.git instead seems to work for me.

See https://github.com/cython/cython/issues/1955

[lmazuel]

I also have the same problem with 3.7-dev, but the error is not exactly the same (but considering the generic title of this issue, I think it's the right place :))
https://travis-ci.org/Azure/azure-sdk-for-python/jobs/341574626

../../../virtualenv/python3.7-dev/lib/python3.7/site-packages/yaml/__init__.py:70: in load
    loader = Loader(stream)
../../../virtualenv/python3.7-dev/lib/python3.7/site-packages/yaml/cyaml.py:31: in __init__
    CParser.__init__(self, stream)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
>   ???
E   AttributeError: 'str' object has no attribute 'read'
ext/_yaml.pyx:269: AttributeError

All other Python jobs are working as expected.

[jaraco]

Installing from [github] seems to work for me.

Same for me. pip install git+https://github.com/yaml/pyyaml.git. Of course, this workaround is a clumsy one, requiring manual intervention on any project adopting Python 3.7. It would be nice to see a updated release with wheels on PyPI to help ease the transition for other projects as they seek to support Python 3.7.

[wumpus]

Somewhat ironic, but the latest checkin to git https://github.com/yaml/pyyaml/commit/298e07907ae526594069f6fdf31f2f1278cc1ae3 adds a fallback to the install process to use pure python if the C compile fails. So installing from git can have 2 pretty different outcomes, with different bugs.

[jakirkham]

Honestly just a fresh tag and new sdist would be great. The last tag is from 2016.

[alexprengere]

I am repeating what has been said in this thread, but since Python 3.7 release is approaching (b4 is out now), it would be truly awesome if a new package could be uploaded to PyPI (problem is fixed when installing from github). This would make the experience of testing the upcoming release much smoother.

[tankoyarik]

When I've tried to run tests in Travis with Python 3.7-dev the following issue occurs:
Failed building wheel for pyyaml (Failed to build pyyaml
). Tried to add dependency_links=['https://github.com/yaml/pyyaml#egg=pyyaml'] (dependency_links=['https://github.com/yaml/pyyaml/archive/master.tar.gz#egg=pyyaml']) into setup.py and 'pyyaml' into install_requires. Still did not fix the issue. Waiting for stable build of the Pyyaml compatible with Python 3.7

[diefans]

I use https://bitbucket.org/ruamel/yaml since a while...

[askoretskiy]

Any plans on PyYAML 3.13 release? That would resolve these issues with soon-coming Python 3.7.

[ingydotnet]

Yes, plans are in progress. Working with @alex to get this done. Will file an umbrella issue for the release process later today.

[peterkmurphy]

Ingy and Artem,

While this is not quite relevant, is there going to be a libyaml release
soon as well? I actually have my own proposed bug-fix for issue #94
"Unicode anchor and alias names are not supported."

https://github.com/yaml/pyyaml/issues/94

When testing this in Travis, it appears that a resolution of this issue
requires a resolution of the existing libyaml parser, if one uses libyaml.
So I think I need to put my fix for libyaml in as well.

https://travis-ci.org/peterkmurphy/pyyaml/jobs/301371997 (lines 1167-1185)

Best regards,
Peter

On Tue, May 29, 2018 at 6:27 AM, Ingy döt Net [e-mail]
wrote:

Yes, plans are in progress. Working with @alex https://github.com/alex
to get this done. Will file an umbrella issue for the release process later
today.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/yaml/pyyaml/issues/126#issuecomment-392598584, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA-AtUHXfAQzHUwsftTJYeQOmeYWk6VCks5t3F3KgaJpZM4R0QyS
.

--
Email: [email protected]
WWW: http://www.pkmurphy.com.au/

[askoretskiy]

Fine, @ingydotnet . Hoping for a quick release with @alex . Python 3.7 is to be released in the end of June.

[jaraco]

I encounter this issue several times a week and continue to add workarounds in dozens of projects - workarounds that I'll have to remove later. Is there anything that can be done to cut a new release or even just publish wheels for the last release?

[hugovk]

PyYAML 4.1 has been released, and I can confirm it can be installed on Python 3.7.

https://travis-ci.org/pylast/pylast/jobs/397353343#L479

And the workaround of installing from GitHub can be removed: https://github.com/pylast/pylast/pull/275.

Thanks everyone for the new release!

[ingydotnet]

The 4.1 release has been removed from PyPI due to several issues. 4.2 is being worked on now. In the meantime, reopening this issue.

Sorry for the hiccups and thanks for your patience. I hope to have things resolved ASAP.

[The-Compiler]

@ingydotnet No worries, some hiccups are bound to happen with the first release after almost two years. Thanks for all your work, glad to see PyYAML is finally gaining some traction again! :sparkling_heart:

[jakirkham]

No worries. Also thanks for letting us know. Happy to wait until things are ready to try again. :)

[lyricnz]

homebrew just updated python3 to 3.7, which caused this to become a Big Problem. Expect a bunch of people arriving here with the same issue.

[hugovk]

The PyYAML team is working hard to get a release out for Python 3.7 in the next few days.

See https://github.com/yaml/pyyaml/issues/193 for the PyYAML 4.2 release plan.

[ingydotnet]

@lyricnz Try this on homebrew 3.7:

pip3 install pyyaml==4.2b4

It worked for me. Hopefully that will be good enough until we can get 4.2 out.

[webknjaz]

@ingydotnet I'd recommend .dev releases for user-testing rather than betas, since they'll not be automatically picked up by pip unless user explicitly uses --pre

[dvorapa]

@webknjaz Pip understands betas as well and correctly doesn't propagate them as new releases until chosen explicitly

[webknjaz]

oh, cool! i actually never tried betas, thought dev is safer :)

[ViktorHaag]

Since I believe that awscli pins to an older version window for pyyaml, am I right in thinking that "simply" releasing a new version is not necessarily going to ease the pain of people depending on packages like awscli that continue to pin to older versions?

[hugovk]

Correct. Once the release is made, other packages will need to update (or remove) their pins, like this https://github.com/aws/aws-cli/pull/3414.

[ViktorHaag]

Hmmm, @hugovk -- since the PR against awscli effectively pins to 4.1, will a new PR against awscli get approved/merged that re-pins to 4.2?

[hugovk]

@ViktorHaag Yes, something like that. That PR was closed because the last PyYAML release was pulled. Let's wait for the next release then make a fresh PR.

[ingydotnet]

FYI, we had a slight change in plans yesterday https://github.com/yaml/pyyaml/issues/193#issuecomment-401990322

PyYAML 3.13 should go out today or tomorrow. That will have PyYAML exact same as 3.12 but works with Python 3.7.

PyYAML 4.2 will be out in another week or more after 3.13.

[BrettFagerlund]

python:latest on dockerhub has pushed up to 3.7 release today which will break things for a few more people too

[her]

I'm not sure if anyone has a better work around, but my solution was just to grab a 3.6 version of Python for awscli. You can downgrade brew python if you'd previously had 3.6 installed, however I didn't have that option since I was setting up a new machine.

@ingydotnet I attempted your solution first, but it didn't work for me. FYI.

[drnic]

Some instructions for installing old python 3.6.5 via brew at https://stackoverflow.com/a/51125014/36170

[zeyuye-airwallex]

python 3.7 was released on 27 Jun 2018, i think this issue still exist

[prokher]

@zeyuye-airwallex I can confirm, it is still here.

[ViktorHaag]

@zeyuye-airwallex and @prokher -- please see issue #193 for the release plan to cope with this; it's being actively worked on.

[NaamaGertel]

What are the commands that I need to use in a script besides pip3 install pyyaml==4.2b4 in order to install awscli and to update python and pyyaml versions to a working versions? I'm running docker with python image and get errors while running command pip install awscli

[BrettFagerlund]

@NaamaGertel with the python image, the easiest way would be revert to a 3.6 tag if you can. awscli will still install fine on that

[Tensho]

Pinned CIrcleCI primary image to circleci/python:3.6.6-stretch and successfully installed awscli with one command: pip install awscli --upgrade --user.

[ingydotnet]

PyYAML-3.13 has been released and it works with Python 3.7.

[hugovk]

@ingydotnet Thank you very much to you, the whole PyYAML team and contributors for your hard work over the last week getting all this stuff sorted!

[antonioned]

@ingydotnet I am not sure if this is still for you guys but I still have problems on CircleCI step pip install awsebcli that keeps failing when I use the python:3 docker image (python 3.7.0). I fixed this issue the same way @Tensho did, using the python:3.6.6 docker image.
I have already contacted CircleCI, pending their response as well.

[alysivji]

Got this same error inside of a container. I installed libyaml via apt-get

RUN apt-get update -y \
    && apt-get install -y libyaml-dev \
    && apt-get clean

[MatthewRalston]

I know this thread is kind of dead, but Python 3.7 has been out for a while and is used in production often. I'm still having gcc errors with pyyaml, which again is a dependency of many packages.

gcc 8.3.0
Python3.7.3

[23doors]

This thread is kind of dead because pyyaml==3.13 fixed it. Simply use 3.13 with python 3.7.