Conan: `conan alias` creates empty packages leading to FileNotFoundError

[monsdar]

When calling conan alias it generates an empty packages under certain circumstances which will lead to further issues. E.g. conan copy --all will not work anymore and crash with a FileNotFoundError.

Repro with the following commands:

• conan new alias_copy_repro/1.0.0@user/testing
• Add short_paths = True to the recipe
• conan profile update settings.build_type=Release default
• conan create . user/testing -s build_type=Debug
• conan alias alias_copy_repro/ALIAS@user/testing alias_copy_repro/1.0.0@user/testing
• conan copy --all alias_copy_repro/1.0.0@user/testing test/testing

Output:

#conan copy --all alias_copy_repro/1.0.0@user/testing test/testing

Copied alias_copy_repro/1.0.0@user/testing to alias_copy_repro/1.0.0@test/testing
Copied sources alias_copy_repro/1.0.0@user/testing to alias_copy_repro/1.0.0@test/testing
Traceback (most recent call last):
  File "c:\python36-32\lib\site-packages\conans\client\command.py", line 1570, in run
    method(args[0][1:])
  File "c:\python36-32\lib\site-packages\conans\client\command.py", line 922, in copy
    force=args.force, packages=args.package or args.all)
  File "c:\python36-32\lib\site-packages\conans\client\conan_api.py", line 93, in wrapper
    return f(*args, **kwargs)
  File "c:\python36-32\lib\site-packages\conans\client\conan_api.py", line 772, in copy
    self._user_io, self._remote_manager, self._loader, force=force)
  File "c:\python36-32\lib\site-packages\conans\client\cmd\copy.py", line 39, in cmd_copy
    package_copy(ref, user_channel, package_ids, cache, user_io, short_paths, force)
  File "c:\python36-32\lib\site-packages\conans\client\cmd\copy.py", line 83, in package_copy
    shutil.copytree(package_path_origin, package_path_dest, symlinks=True)
  File "c:\python36-32\lib\shutil.py", line 309, in copytree
    names = os.listdir(src)
FileNotFoundError: [WinError 3] Das System kann den angegebenen Pfad nicht finden: 'C:\\.conan\\1789f2\\1'

This is what I've found out so far:

• This seems to happen when the package contains a different type of package as the default-profile which is used on the system.
• Alias creates an empty package for the default-profile
• This only happens when using short_paths as far as I can tell
• This at least crashes conan copy --all, but I'm not sure what else could be affected

Happens to me with the following environments:

• Conan 1.13.0, 1.13.1 and 1.13.2
• Windows 7 and 10
• Python 2.7 and Python 3.6.1

This did not happen with Conan 1.3.3.

[monsdar]

I know this issue sounds oddly specific... But this is actually what our CI does and which led to our builds crashing all of a sudden after we updated to Conan 1.13.0 :/

[lasote]

Thanks, Nils, yes, it looks like a bug with the alias. The 1.14 is hopefully be released next week so we will include this fix. Have a great weekend!

[danimtb]

I have tried to reproduce this issue in the current develop branch and Conan .1.13.0 with this test:

    def alias_short_paths_test(self):
        client = TestClient()

        client.run("new alias_copy_repro/1.0.0@user/testing")
        conanfile_path = os.path.join(client.current_folder, "conanfile.py")
        new_content = load(conanfile_path).replace("version = \"1.0.0\"",
                                                   "version = \"1.0.0\"\n    short_paths = True")
        client.save({"conanfile.py": new_content})
        client.run("profile new default --detect")
        client.run("profile update settings.build_type=Release default")
        client.run("create . user/testing -s build_type=Debug")
        client.run("alias alias_copy_repro/ALIAS@user/testing alias_copy_repro/1.0.0@user/testing")
        client.run("copy --all alias_copy_repro/1.0.0@user/testing test/testing")
        print(client.all_output)

The only variable out of scope is the profile you might be using (here I am using the default on Windows), but I don't that is the cause.

Any clue @monsdar ?

[danimtb]

Did also a check changing the last command just in case and worked fine too:

client.run("copy --all alias_copy_repro/ALIAS@user/testing test/testing")

[monsdar]

Hmm... I'm using the default profile as well, should be no surprise:

[settings]
os=Windows
os_build=Windows
arch=x86_64
arch_build=x86_64
compiler=Visual Studio
compiler.version=15
build_type=Release
[options]
[build_requires]
[env]

Have you tried the above commands directly on Commandline? For me the empty package gets generated everytime I run the alias command. Then copy fails because it tries to copy a package that doesn't exist.

I was able to replicate this on the two computers where I checked this on. Also our CI builds were failing because of that (at least 3 Windows 7 runners where that happened).

[memsharded]

I have tried too, couldn't make it fail either.
I am just copying your commands above into my terminal.

I don't see how the alias is used here, because the copy is being done to the other package. The alias is like it doesn't exist in the copy command.

[memsharded]

I don't know, maybe putting the above commands in a bat or python script, just to make sure we are not missing something?

[monsdar]

I tried with a modified version of the script from @danimtb. With that I couldn't repro the issue as well. I will check again on my desktop PC at home. There I definitely can repro the issue...

from conans.test.utils.tools import TestClient
import os

client = TestClient()
client.run("new alias_copy_repro/1.0.0@user/testing")
conanfile_path = os.path.join(client.current_folder, "conanfile.py")
new_content = open(conanfile_path).read().replace("version = \"1.0.0\"",
                                           "version = \"1.0.0\"\n    short_paths = True")
client.save({"conanfile.py": new_content})
client.run("profile new default --detect")
client.run("profile update settings.build_type=Release default")
client.run("create . user/testing -s build_type=Debug")
client.run("alias alias_copy_repro/ALIAS@user/testing alias_copy_repro/1.0.0@user/testing")
client.run("copy --all alias_copy_repro/1.0.0@user/testing test/testing")
print(client.all_output)

[danimtb]

Moving this out of the release as we were not able to reproduce. Please @monsdar share your conclusions when you give it a try. Thanks!

[monsdar]

Sorry for taking so long to answer. I haven't changed my environment since reporting last time and checked on my home desktop now.

With the repro python script I posted a few days ago this issue is not reproducable even on that machine.

However when running the above commands directly in cmd or using a bash script it shows the issue once more. I'm not sure what the Python-based version does different - but there must be something.

Here's my bash script:

@echo off
conan new alias_copy_repro/1.0.0@user/testing
echo #### Please add "short_paths = True" to the conanfile.py now... ####
pause
conan profile update settings.build_type=Release default
conan create . user/testing -s build_type=Debug
conan alias alias_copy_repro/ALIAS@user/testing alias_copy_repro/1.0.0@user/testing
conan copy --all alias_copy_repro/1.0.0@user/testing test/testing

Please check if this is helping to repro on your side as well...

[monsdar]

The above was reproduced with Conan 1.13.2

I just checked with 1.14.1 and I can confirm that this seems not to be happening anymore. So close as solved? 🤷‍♂️

[danimtb]

Hi @monsdar,

I have given it another try. Tested under Windows 10, with both python 2.7.15 and 3.7.0 and I failed to reproduce the issue in Conan 1.13.2 and 1.14.1

I have tested with your script above and the python test I posted some messages ago.

As you reported you are not able to reproduce it in 1.14.1, I think we could consider it as fixed or not reproducible at least...

Please, report back if you continue to have any trouble related to this. Thanks a lot for the help 😄