Conan: [question] Where is default root which Conan pulls files from in package() method?

[michaelmaguire]

I'm working with a recipe I was passed which calls CMake.install() in its build() method:

    def build(self):
        cmake = CMake(self)

        # do some setup here
       [...]

        cmake.configure()
        cmake.build()
        cmake.install()

I see from the conan build output that the build is performed in:

bde/3.11.0.4@PORT/stable: Building your package in C:\Users\mmaguir1\.conan\data\bde\3.11.0.4\PORT\stable\build\28796df6684cd3c3490c089a9f9a2a6a3993b4fc

However, later on in the build output, I see that cmake.install() is moving created products into the package/hashXXX dir:

     [...]
     3>Done Building Project "C:\Users\mmaguir1\.conan\data\bde\3.11.0.4\PORT\stable\build\28796df6684cd3c3490c089a9f9a2a6a3993b4fc\ALL_BUILD.vcxproj" (default targets).
     1>PrepareForBuild:
         Creating directory "x64\Release\INSTALL\".
         Creating directory "x64\Release\INSTALL\INSTALL.tlog\".
       InitializeBuildStatus:
         Creating "x64\Release\INSTALL\INSTALL.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
       PostBuildEvent:
         setlocal
         "C:\Program Files\CMake\bin\cmake.exe" -DBUILD_TYPE=Release -P cmake_install.cmake
         if %errorlevel% neq 0 goto :cmEnd
         :cmEnd
         endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
         :cmErrorLevel
         exit /b %1
         :cmDone
         if %errorlevel% neq 0 goto :VCEnd
         :VCEnd
         -- Install configuration: "Release"
         -- Installing: C:/Users/mmaguir1/.conan/data/bde/3.11.0.4/PORT/stable/package/28796df6684cd3c3490c089a9f9a2a6a3993b4fc/include/balb_assertiontracker.h

I'm a little confused. The Conan docs suggest that the build() method only fiddles in the build/hashXXX folder:

Then, for each different configuration of settings and options, a package ID will be computed in the form of a SHA-1 hash of such configuration. Sources will be copied to the build/hashXXX folder, and the build() method will be triggered.

After that, the package() method will be called to copy artifacts from the build/hashXXX folder to the package/hashXXX folder.

https://docs.conan.io/en/1.3/creating_packages/understand_packaging.html#the-package-creation-process

But from my build logs, it seems clear that invoking CMake.install() has now fiddled in the package/hashXXX folder, without the package() method doing anything.

Questions:

1) Who told CMake.install() in my build() method to use package/hashXXX? Was it the conan_basic_setup() we added in our CMakeLists.txt?

set(CONAN_BLD_INFO ${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
if (WIN32 AND EXISTS $CONAN_BLD_INFO)
    include($CONAN_BLD_INFO)
    conan_basic_setup()
endif()

2) Does my own conanfile.py package() do anything anymore?

Will the package .tgz be generated from whatever was INSTALLED in package/hashXXX?

If I need to re-arrange or filter out files, can I still do something in package()? If so, is the default root directory in which package() operations still my build/hashXXX directory?

To help us debug your issue please explain:

• [ x] I've read the CONTRIBUTING guide.
• [ Windows Server 2012 R2 64, Conan version 1.5.2] I've specified the Conan version, operating system version and any tool that can be relevant.
• [x] I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.

[memsharded]

Hi, @michaelmaguire

Yes, you are right, the cmake.install() method call is installing to the "package folder". It is very possible that the docs are a bit outdated or do not contemplate this case, because cmake.install() was developed after that section of the docs.

It is not something in the conanbuildinfo.cmake file, it is defined by the CMake helper, which would be injecting CMAKE_INSTALL_PREFIX, the standard cmake way to do it, in the command line invocation of cmake that is done actually the first time cmake is called, that is, cmake.configure()

Yes, the conan_package.tgz will be generated from the files in the package/hashxxxx folder. As you figured out, the package() method is no longer strictly necessary, though if you still need to do some rearrangements or any other copy of files that was not implemented in the cmake.install(), then sure, you can use the package() method too.

Please, let me know if this helps, or have any further question. Thanks!

[michaelmaguire]

Thanks.

Can you give some info on what operations I can do inside the package() method? Is the default src folder within the package/hashXXXX or the build/hashXXXX dir?

[danimtb]

Hi @michaelmaguire,

The default src folder is usually the source folder in the build directory, however this depends on some situations. There is a good explanation in the docs: https://docs.conan.io/en/latest/reference/conanfile/methods.html#package

Hope you find it useful! Thanks 😄

[michaelmaguire]

Thanks @danimtb !

I've read the link you provided, I just need a more clarification.

The docs on package link you mentioned above said:

src (Optional, Defaulted to ""): The folder where you want to search the files in the build folder.

and above you said:

The default src folder is usually the source folder in the build directory

When I look at my .conan/data cache for my package, it appears to me that my 'source' folder and my 'build/hashXXXX' folders are different and not one within the other.

e.g. root of my 'PORT/stable' stream's data area:

C:\Users\mmaguir1\.conan\data\bde-classic\3.11.0.4\PORT\stable>dir
 Volume in drive C is MAGUIRE-C
 Volume Serial Number is 82B2-3C24

 Directory of C:\Users\mmaguir1\.conan\data\bde-classic\3.11.0.4\PORT\stable

09/20/2018  06:03 AM    <DIR>          .
09/20/2018  06:03 AM    <DIR>          ..
09/20/2018  06:03 AM    <DIR>          build
09/20/2018  06:03 AM    <DIR>          export
09/20/2018  06:03 AM    <DIR>          export_source
09/20/2018  05:54 AM    <DIR>          locks
09/20/2018  06:07 AM    <DIR>          package
09/20/2018  06:03 AM                46 scm_folder.txt
09/20/2018  06:03 AM    <DIR>          source
               1 File(s)             46 bytes
               8 Dir(s)  11,830,685,696 bytes free

Contents of my 'source' directory are as I expect:

C:\Users\mmaguir1\.conan\data\bde-classic\3.11.0.4\PORT\stable\source>dir
 Volume in drive C is MAGUIRE-C
 Volume Serial Number is 82B2-3C24

 Directory of C:\Users\mmaguir1\.conan\data\bde-classic\3.11.0.4\PORT\stable\source

09/20/2018  06:03 AM    <DIR>          .
09/20/2018  06:03 AM    <DIR>          ..
09/13/2018  10:46 AM               245 .arcconfig
09/13/2018  10:46 AM               618 .arclint
09/13/2018  10:46 AM               775 .gitignore
09/20/2018  06:03 AM    <DIR>          build
09/13/2018  11:01 AM               463 CMakeLists.txt
09/20/2018  06:03 AM    <DIR>          debian
09/20/2018  06:03 AM    <DIR>          groups
09/20/2018  06:03 AM             1,951 output.log
09/13/2018  10:46 AM               149 project.cmake
09/13/2018  10:46 AM             1,230 README.md
09/13/2018  10:46 AM               890 wscript
               8 File(s)          6,321 bytes
               5 Dir(s)  11,830,677,504 bytes free

Contents of my 'build' directory also make sense to me (two hashXXXX flavours, Release and Debug):

C:\Users\mmaguir1\.conan\data\bde-classic\3.11.0.4\PORT\stable\build>dir
 Volume in drive C is MAGUIRE-C
 Volume Serial Number is 82B2-3C24

 Directory of C:\Users\mmaguir1\.conan\data\bde-classic\3.11.0.4\PORT\stable\build

09/20/2018  06:03 AM    <DIR>          .
09/20/2018  06:03 AM    <DIR>          ..
09/20/2018  05:57 AM    <DIR>          65ee3edfa966a0905abd673449edbce6fad84ddb
09/20/2018  06:07 AM    <DIR>          7cb96cc632de263f1ef208e4050917b21e8325d8
               0 File(s)              0 bytes
               4 Dir(s)  11,830,677,504 bytes free

Contents of my 'package' directory also make sense to me (two hashXXXX flavours, Release and Debug):

C:\Users\mmaguir1\.conan\data\bde-classic\3.11.0.4\PORT\stable\package>dir
 Volume in drive C is MAGUIRE-C
 Volume Serial Number is 82B2-3C24

 Directory of C:\Users\mmaguir1\.conan\data\bde-classic\3.11.0.4\PORT\stable\package

09/20/2018  06:07 AM    <DIR>          .
09/20/2018  06:07 AM    <DIR>          ..
09/20/2018  05:58 AM    <DIR>          65ee3edfa966a0905abd673449edbce6fad84ddb
09/20/2018  06:07 AM    <DIR>          7cb96cc632de263f1ef208e4050917b21e8325d8
               0 File(s)              0 bytes
               4 Dir(s)  11,830,566,912 bytes free

No 'source' directory in my 'build' directory:

C:\Users\mmaguir1\.conan\data\bde-classic\3.11.0.4\PORT\stable\build\65ee3edfa966a0905abd673449edbce6fad84ddb>cd source
The system cannot find the path specified.

Can you clear up for me of the paths above, which one is the intended root when I make relative path references in the src parameter of the copy() command?

[danimtb]

Actually build directories in your case are:

• C:\Users\mmaguir1.conan\databde-classic\3.11.0.4\PORT\stablebuild\65ee3edfa966a0905abd673449edbce6fad84ddb
• C:\Users\mmaguir1.conan\databde-classic\3.11.0.4\PORT\stablebuild\7cb96cc632de263f1ef208e4050917b21e8325d8

One for each configuration you run conan create with.

Inside those build directories you should see a copy of the sources folder (C:\Users\mmaguir1.conan\databde-classic\3.11.0.4\PORT\stable\source). This is the default behavior in Conan to isolate the sources from the build phase, as some build systems build inside the source folder (and we don't want to mix binaries inside the sources folder, that's why Conan makes the copy).

This is also explained at the bottom of the docs section I linked:

• Build folder and source folder are the same: Normally during conan create source folder content is copied to the build folder. In this situation src parameter of self.copy() will point to the build folder in the local cache.

[michaelmaguire]

Inside those build directories you should see a copy of the sources folder (C:\Users\mmaguir1.conan\databde-classic\3.11.0.4\PORT\stable\source). This is the default behavior in Conan to isolate the sources from the build phase

Got it. I see the whole contents of C:\Users\mmaguir1.conan\data\bde-classic\3.11.0.4\PORT\stable\source have been dumped into C:\Users\mmaguir1\.conan\data\bde-classic\3.11.0.4\PORT\stable\build\65ee3edfa966a0905abd673449edbce6fad84ddb

I guess I was confused when you said:

The default src folder is usually the source folder in the build directory

because:

1) There's no source folder in the 'build' directory, instead the default src folder is usually the build/hashXXXX directory, which contains a copy of all files from the 'source' folder

2) there is a 'source' folder in the C:\Users\mmaguir1.conan\data\bde-classic\3.11.0.4\PORT\stable directory (which a noob like myself might consider a 'build' directory), but that's not the one you meant

Sorry -- I'm new to Conan. Thanks for the clarification.

[danimtb]

Glad you found the information useful @michaelmaguire. As a recommendation: you might be interested in the CppLang #conan slack channel. There is a growing community that can help you with this kind of questions and it is really helpful for people that are starting to look into Conan 😃

If your doubts were solved please close this issue. Thanks a lot