Twine: Twine not properly uploading long_description

[mdagost]

I've been trying to upload a package with twine, and no matter what I do, I can't get the long_description to render. I'm left with the generic The author of this package has not provided a project description. . There is no error message during the twine upload process.

However, if I upload the package via python setup.py sdist upload, the long_description renders just fine, as you can see here.

Does anyone have any idea what's going on?

Code is here.

[sigmavirus24]

The issue is a common one: You are specifying the license text here which breaks rendering. That is now how you're supposed to specify the license. It should be a single line, e.g., Apache-2.0.

[mdagost]

Huh. That's odd for a few reasons:

• There's no warning printed.
• Why does the license field effect the long_description field?
• Why does it work with setup.py but not twine?
• This seems to break any kind of custom licensing.
• This is also out of step with how licenses are defined in most template projects. For example, https://github.com/kennethreitz/samplemod

[sigmavirus24]

There's no warning printed.

Twine uploads what the (sdist/wheel)s given. The warning would be generated if you used python setup.py check -r -s provided by the readme package. It doesn't build it. If we built it, we'd give that warning.

Why does the license field effect the long_description field?

This is PyPI behaviour. I don't fully understand it.

Why does it work with setup.py but not twine?

Again, unclear as the behaviour is primarily PyPI's. It could be the way the tools order the fields. I really haven't bothered to look into it since it's not the correct usage of the field.

This seems to break any kind of custom licensing.

That custom licensing should be included in the long description.

This is also out of step with how licenses are defined in most template projects.

A thing worked once upon a time but is no longer the correct way of doing things. The official documentation and sample project demonstrate the appropriate usage of the field.

[mdagost]

Okay. For the record, there aren't any warnings generated:

(py3) [ mdagostino: ~/jupyter-notify-mdagost ]$ python setup.py check -r -s
running check
(py3) [ mdagostino: ~/jupyter-notify-mdagost ]$ 

[sigmavirus24]

My mistake then. I thought that caught this.

[sixpearls]

Why is this closed when the problem hasn't been addressed? Twine should not be the recommended tool if it cannot upload long_descriptions. After several hours of debugging, I also had to resort to using python setup.py sdist upload.

[sixpearls]

I also don't have a custom license.

[sixpearls]

In my case, only the first line of my long_description was uploaded. If the long_description started with a blank first line, I would get the error The author of this package has not provided a project description.. I checked the PKG_INFO from sdist, and the field is rendered. I'm guessing that twine is not properly parsing the PKG_INFO file.

This happened with test.pypi.org, pypi.org, and pypi.python.org.

[sigmavirus24]

Why is this closed when the problem hasn't been addressed? Twine should not be the recommended tool if it cannot upload long_descriptions. After several hours of debugging, I also had to resort to using python setup.py sdist upload.

Twine can and does upload long_descriptions. You seem to have a separate bug which deserves its own issue.

[sixpearls]

I'm having trouble isolating the bug to create a minimum example, but the version of my library on test.pypi.org is the same as the one on pypi.org with the only difference being how it was uploaded (twine upload vs python setup.py upload). Using twine, the long_description didn't get rendered correctly.

[brainwane]

@sixpearls Thanks for reporting the issue. Could you try using Twine 1.9.1 to twine upload a newer version of simupy on test.pypi.org and -- if there is still a problem -- open a new issue (and link to this one)? That'll be easier for us to track, in my opinion. Thanks!

(I also want to check the slim possibility that Warehouse was messing things up but has fixed the bug somewhere in the past ~6 months.)

[tmirzoev]

@brainwane I've encountered the same problem as @sixpearls and stumbled upon this article in search for solution.
Here's the lib - https://test.pypi.org/project/robotframework-requestslogger/

• 7 efforts to upload via twine, every one with different editing of readme.rst, all in vain
• 1 successful with setup.py

[di]

@tmirzoev Can you give us the exact output of twine --version, as well as the command you ran which failed?

[sixpearls]

Sorry, @brainwane / others: I've been swamped at school. Next time I release a version (of any project) I'll circle back to help debug if it hasn't been addressed yet. At any rate, I think this issue (or one attempting to address this bug) should remain open since this is clearly a problem.

[sigmavirus24]

At any rate, I think this issue (or one attempting to address this bug) should remain open since this is clearly a problem.

It is not clearly an issue @sixpearls. No one has given us the information to determine clearly whether it's a real issue or one of an outdated version.

[sixpearls]

Thanks, I will stick with using setup.py upload since it works reliably.

[brainwane]

@sixpearls, hey, I hope you try Twine again (the current release is 1.11.0) and let us know if you're running into the same problem so we can fix it. @sigmavirus24 is right that we don't yet know enough about the bug to know where it's coming from, but clearly you are experiencing a problem, and the more we know, the more reliably we can reproduce and track it down.

[tmirzoev]

@di

@tmirzoev Can you give us the exact output of twine --version, as well as the command you ran which failed?

twine version 1.11.0 (pkginfo: 1.4.2, requests: 2.12.4, setuptools: 39.0.1, requests-toolbelt: 0.8.0, tqdm: 4.20.0)

I've simply checked the readme.rst for errors

python setup.py check -r -s

built dist with

python setup.py sdist bdist_wheel

and then uploaded it with

twine upload --repository-url https://test.pypi.org/legacy/ dist/*

There was no errors output on any of the steps, but long_description didn't make its way to the package page.

[brainwane]

What version of 'wheel' is installed?

[tmirzoev]

wheel 0.31.0

[brainwane]

There's a new version of readme_renderer in case you'd like to try with that.

[sigmavirus24]

That's enough info to re-open this in my opinion (it's reported against reasonably recently released versions since we thought we'd fixed this).

[AmitAronovitch]

Confirm similar problem happens for my project as well (uploads fine with setup.py upload, but bad with twine):
Working upload: https://test.pypi.org/project/radcalnet/0.1.0.dev3/ (uploaded using setup.py upload).
Previous versions were using twine and failed (Identical upload using twine: https://test.pypi.org/project/radcalnet/0.1.0.dev2/ ).

Previous attempts:

$ python setup.py check -r -s
running check
The project's long description is valid RST.

Previous attempts of tweaking the license tag:

• No license at all
• License using the license parameter in the setup() call
• License specified using both the license parameter AND the classifier string

All ended the same way (uploaded successfully, without any warning, but showing raw text and tags instead of proper html)

Please let me know what if any more info is required to debug this.

[AmitAronovitch]

readme==0.7.1
readme-renderer==21.0
twine==1.11.0

[brainwane]

@di could you take a look at this?

[di]

The problem can be seen here in the METADATA file for the wheel distribution for radcalnet-0.1.0.dev2

...
Summary: Python package for easy access to the measurements published by RadCalNet.org
Requires-Dist: pandas

Home-page: https://github.com/satellogic/radcalnet

Specifically the extra newline is causing Twine to (correctly) interpret everything after this as the message body, which becomes the long description.

Distutils doesn't support the message-body-as-long-description feature yet (I believe) so it's probably just ignoring this extra empty line.

It's not clear to me where in the setup.py for radcalnet this newline is coming from, but IMO this is probably a bug in wheel, it should be stripping newlines from the Requires-Dist values, and it shouldn't have been possible to generate this METADATA file.

It's _also_ possible this bug has already been fixed, the wheel in question was built with wheel=0.30.0 which is not the latest release, so upgrading to the latest might fix it.

[AmitAronovitch]

wheel 0.31.1 still generates that empty line (did not try uploading yet - just rebuilt the wheel and checked)...

re: tracking the issue into wheel package:
As far as I can see, the METADATA is built based on the PKG-INFO file from the egg-info. That file already contains a blank line before the Home-Page: header.
Still not sure what generates the pkg-info in turn (I am guessing it's in setuptools).

How should we proceed with this (fwd the issue to wheel/setuptools? further debugging? ideas on how to make twine/other tool/ detect and warn about such issues?)

[AmitAronovitch]

hmm I think I got it: there's a newline at the end of the short description (description= parameter of setup()). If I remove that the blank line disappears.

Is newline allowed within the description field? If yes, something (wheel? setuptools? twine?) should take measures to avoid this issue. If not, we can check for it and fail/warn.

[sigmavirus24]

I don't think it should have newlines in the description field. That's not something twine can warn about until we start building things for you. We might be able to warn after the fact but this is probably also something @jaraco might consider for setuptools

[jaraco]

Agreed - any build tool, including Setuptools, should validate all metadata fields. See pypa/setuptools#1390 where that work is happening.

[bittner]

I'm not totally sure, but I think we could close this issue in favor of https://github.com/pypa/setuptools/issues/1390 (as a duplicate).

If setuptools generates the related metadata that's where the validation should go. There's no need to duplicate the efforts, is it?

[di]

I agree, this can be closed as a duplicate of pypa/setuptools#1390

[di]

Closing as a duplicate of pypa/setuptools#1390.