Conan: exports_sources fails to properly handle symlinks

[Paultergeist]

To help us debug your issue please explain:

• [x] I've read the CONTRIBUTING guide.
• [x] I've specified the Conan version, operating system version and any tool that can be relevant.
• [x] I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.

OS: macOS 10.14.5
Conan: 1.16.0 (installed with pip into virtualenv, but also fails with 1.16.0 from Brew). Works with 1.15.1.
Python: 3.7.3 (installed from MacPorts)

conanfile.py has exports_sources=src/*, there is src/foo.framework with symlinks:

.
├── Headers -> Versions/Current/Headers
├── Resources -> Versions/Current/Resources
├── Versions
│   ├── A
│   │   ├── Headers
│   │   │   └── foo.h
│   │   ├── Resources
│   │   │   ├── Info.plist
│   │   │   └── en.lproj
│   │   │       └── InfoPlist.strings
│   │   └── foo
│   └── Current -> A
└── foo -> Versions/Current/foo

conan create . test/test fails with the following error:

% conan create . test/test                                                                                                                                                                                                                                          master
Exporting package recipe
symlinks/1.0.0@test/test exports_sources: Copied 1 '.txt' file: CMakeLists.txt
symlinks/1.0.0@test/test exports_sources: Copied 1 '.cpp' file: hello.cpp
symlinks/1.0.0@test/test exports_sources: Copied 2 '.h' files: hello.h, foo.h
symlinks/1.0.0@test/test exports_sources: Copied 2 files: foo, foo
symlinks/1.0.0@test/test exports_sources: Copied 1 '.plist' file: Info.plist
symlinks/1.0.0@test/test exports_sources: Copied 1 '.strings' file: InfoPlist.strings
symlinks/1.0.0@test/test: The stored package has not changed
symlinks/1.0.0@test/test: Source folder is corrupted, forcing removal
Configuration:
[settings]
[options]
[build_requires]
[env]

symlinks/1.0.0@test/test: Forced build from source
Installing package: symlinks/1.0.0@test/test
Requirements
    symlinks/1.0.0@test/test from local cache - Cache
Packages
    symlinks/1.0.0@test/test:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Build

symlinks/1.0.0@test/test: WARN: Build folder is dirty, removing it: /Users/paul/.conan/data/symlinks/1.0.0/test/test/build/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9
symlinks/1.0.0@test/test: WARN: Trying to remove corrupted source folder
symlinks/1.0.0@test/test: WARN: This can take a while for big packages
symlinks/1.0.0@test/test: Configuring sources in /Users/paul/.conan/data/symlinks/1.0.0/test/test/source
ERROR: [Errno 17] File exists: 'Versions/Current/Resources' -> '/Users/paul/.conan/data/symlinks/1.0.0/test/test/source/src/foo.framework/Resources'

Sample project: conan_symlinks

[memsharded]

Please @jgsogo have a look at this, it could be a regression from latest changes in 1.16 related to the SCM and symlinks

[jgsogo]

Hi @Paultergeist! Thanks for reporting this issue. I can confirm that we introduced the broken behavior in #5237 trying to fix an issue related to symlinked folders.

Nevertheless, it is not exactly a regression as it was in fact broken in 1.15 but we didn't have any test with the symlink structure you are using. It was working, but we were duplicating some symlinked folders inside the source directory.

I expect #5342 to solve this issue and to improve how we are handling symlinked folders. It should be released soon in a 1.16.1 version.

[memsharded]

Merged #5432, will release 1.16.1 asap

[drozel]

I still have this problem in 1.16.1:
my structure in repo, i use this repo for a lib with SCM feature:
.

├── examples
│   ├── webchat
            ├── htdocs
            ├── bin
                  ├── htdocs -> ../htdocs

if i try conan create for this lib, conan can't checkout my repo:

lib/2.9.0@me/develop: Configuring sources in /home/me/.conan/data/lib/2.9.0/me/develop/source
lib/2.9.0@me/develop: Getting sources from url: '[email protected]:app/lib.git'
ERROR: [Errno 17] File exists: '../htdocs' -> '/home/me/.conan/data/lib/2.9.0/me/develop/source/examples/webchat/bin/htdocs'

if i try to ls these folders, they are created:

tree /home/me/.conan/data/lib/2.9.0/me/develop/source/examples/webchat 
├── bin
│   └── htdocs -> ../htdocs
├── htdocs
│   ├── index.html
├── main.cpp