Conan: Upload error while building multiple configurations

[jcaden]

Operating system: Linux
Python version: 3.6.2
Conan version: 1.8.2
Conan-package-tools: 0.19.4

When uploading a package built with multiple configurations using conan package tools I get the following error while trying to upload second build.
The user has write permissions but not delete permissions.

Uploading conanmanifest.txt                                           
ERROR: 
Error uploading file: conanmanifest.txt, '{
  "errors" : [ {
    "status" : 500,
    "message" : "Not enough permissions to overwrite artifact 'local-conan:user/zlib/1.2.11-14/master/export/conanmanifest.txt' (user 'user' needs DELETE permission)."
  } ]
}'

Same operation works without problems when using version 1.7.4 and same version of conan package tools.

To help us debug your issue please explain:

• [x] I've read the CONTRIBUTING guide.
• [X] I've specified the Conan version, operating system version and any tool that can be relevant.
• [X] I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.

[lasote]

Hi, are you using Artifactory, right?
Are you also uploading packages from different operating systems (including windows), right?
I think what it could be happening is when you are doing the git clone of the source repository (or calling the source() method) git is messing with the endlines. so:

• One job uploads the recipe (let's say in windows with \r\n) and the packages.
• A different job uploads the recipe and the packages (the recipe is always uploaded) but with \n, so the manifest changes and it fails to override it because of the permissions configured (the permissions configured like that are right). When the recipe and the manifest are equal it doesn't fail because it really doesn't upload anything.

You can find information about how to adjust it here:
https://help.github.com/articles/dealing-with-line-endings/#platform-all

We have a related FAQ here:
https://docs.conan.io/en/latest/faq/using.html#packages-got-outdated-when-uploading-an-unchanged-recipe-from-a-different-machine

Related with #2997.

[jcaden]

Yes, I'm using artifactory, but I'm not using Windows at all. Actually all the publication is done by just one command that builds Release and Debug versions of the same package.

Could the issue be related with the fact that the package version depends on a environment variable for adding a suffix with ci build version:

def get_version(version):
    buildId = os.getenv("CONAN_BUILD_ID", "")
    if buildId != "":
        return "%s-%s" % (version,buildId)
    else:
        return version

class MyLibrary(ConanFile):
    name = "MyLibrary"
    libversion = "1.0.0"
    version = get_version(version = libversion)

[jcaden]

Let me remark that running the same script with the same conanfile.py with a previous version of conan 1.7.4 works perfectly so I don't think this is related with the issue you linked but with the latest conan release 1.8.2.

[jcaden]

Could it be related to this change? https://github.com/conan-io/conan/pull/3480

[lasote]

Could the issue be related with the fact that the package version depends on a environment variable for adding a suffix with ci build version:

I don't think so.

Let me remark that running the same script with the same conanfile.py with a previous version of conan 1.7.4 works perfectly so I don't think this is related with the issue you linked but with the latest conan release 1.8.2.

Thanks for the info. It could be a bug related with #3480 yes.
Could you send us both the manifest it is trying to upload and the remote one?

[lasote]

I cannot reproduce the issue. Something is changing in your exported recipes from the first upload to the second one. If you could compare also the contents of the "export" folder would be nice.
Are the two configurations built in different machines or in the same?

[jcaden]

While I gather more information for you. Do you have DELETE permissions in the repository you are trying to upload to? This issue is only present if the user does not have them.

[lasote]

I don't have DELETE permissions. I tried to change the recipe and upload again and I receive the same error. But generating multiple confs works ok (no changes in the exported files).

[jcaden]

Repository is a private installation of Artifactory Version 6.3.3

I'm trying to build a fork of https://github.com/conan-community/conan-zlib where the conanfile is changed with the attached one.

conanfile.zip

Command used to build is this one.

CONAN_UPLOAD=<my repo url> CONAN_LOGIN_USERNAME=publisher CONAN_PASSWORD=<repo password> CONAN_USERNAME=publisher CONAN_CHANNEL=debug CONAN_ARCHS=x86_64 CONAN_BUILD_POLICY=outdated CONAN_GCC_VERSIONS=5 CONAN_DOCKER_IMAGE=lasote/conangcc5 CONAN_BUILD_TYPES=Release,Debug python3 build.py

After building the Release build the package is uploaded correctly, but during the build of the Debug version, upload fails with the following message.

Uploading conanmanifest.txt                                           
ERROR: 
Error uploading file: conanmanifest.txt, '{
  "errors" : [ {
    "status" : 500,
    "message" : "Not enough permissions to overwrite artifact 'conan:publisher/zlib/1.2.11-1/debug/export/conanmanifest.txt' (user 'publisher' needs DELETE permission)."
  } ]
}'

[lasote]

Thanks for the info! I'll try to reproduce it right now and will report the results.

[lasote]

Reproduced. On my way to investigate more.

[lasote]

it is not a bug in Conan but in Conan package tools using the Conan 1.8.
It is adjusting by default an upload policy that forces always to upload the files, causing the failure.
I will release cpt ASAP.

[lasote]

CPT 0.19.7 released, please report if you still have any problem.

[jcaden]

New release works perfectly. Thank you!

[sunxfancy]

Hi, @lasote
I am facing the same problem but the version is a little different:
OS: Linux
Python: 3.5.2
Conan version: 1.10.0
Conan-package-tools: 0.21.1
I am trying to use docker image to build the LLVM library, there are two kinds of build, shared library or not. All the shared library can upload to bintray.com, however, all the static library got an error message:

Uploading llvm/7.0.0@sunxfancy/stable to remote 'upload_repo'
Compressing recipe... [==================================================]
Recipe is up to date, upload skipped
Uploading package 1/1: a8d4081268103040bec81e29f44b87b9a8d689bb to 'upload_repo'
Compressing package...
[==================================================]
Requesting upload urls...Done!
Uploading conanmanifest.txt
Uploading conaninfo.txt
Uploading conan_package.tgz
ERROR:
Error uploading file: conan_package.tgz,
<html> <head><title>405 Not Allowed</title></head> <body bgcolor="white"> <center><h1>405 Not Allowed</h1></center> <hr><center>nginx</center> </body> </html>
You can have a look about the build log in this link: https://travis-ci.org/sunxfancy/conan-llvm/jobs/465274620#L6255
Sorry, I am not sure is this the problem in conan system or a bug in the server part or some mistake made by me. Please give me a hand, thank you.

[lasote]

That is probably a bintray quota limitation. How big are your packages?

[sunxfancy]

Maybe that's possible, the package is almost 300MB. What's the limitation for that?