Conan: [SCM] capture does not clone recursively

[mmha]

I'm currently trying to package the Vulkan SDK. Some of the source code is added though a git submodule. Unfortunately the new scm attribute does not clone the submodules of a repository:

    scm = {
        "type": "git",
        "url": "https://github.com/LunarG/VulkanTools",
        "revision": "master"
    }
# ...
    def build(self):
        tools.Git(self.build_folder).checkout("sdk-%s" % self.version)
        cmake = CMake(self)
        cmake.definitions["GLSLANG_INSTALL_DIR"] = self.deps_cpp_info["glslang"].rootpath
        cmake.configure()
        cmake.build()
        cmake.install()

produces this error message:

CMake Error at CMakeLists.txt:83 (add_subdirectory):
  The source directory

    /.../vulkan_sdk/1.1.73.0/mmha/testing/build/c7ea395c54c08ccce35984132fd791dc738046dc/submodules/Vulkan-LoaderAndValidationLayers

  does not contain a CMakeLists.txt file.

A possible workaround is to init all the submodules manually during the build (git.run("submodule update --init --recursive")), but this does not work well with the --keep-source flag as it causes a reinitialization every time the build is invoked. I think It would be a nice improvement if scm gains a recursive flag:

    scm = {
        "type": "git",
        "url": "https://github.com/LunarG/VulkanTools",
        "revision": "master",
        "recursive": True
    }

[memsharded]

Hi @mmha

Yes, I think the recursive field in SCM could make sense (in theory, git submodules are a kind of dependency management that could be managed by the package manager too, but I see no reason to disallow their usage), and it should be easy to implement. I am labelling this as easy, to encourage contributions.

Thanks for the feedback!

[Minimonium]

@memsharded @mmha SCM currently does not use the update at all, so would it make more sense to make it call update by default and provide an option to pass update parameters?

[danimtb]

@Minimonium wouldn't it be necessary to call git submodule init before git submodule update? If that is the case maybe a git clone --recrusive should be easier

[Minimonium]

@danimtb maybe something like this? https://docs.gitlab.com/ee/ci/yaml/README.html#git-submodule-strategy

[danimtb]

@Minimonium yeah, I like the strategies followed there but I was pointing togit clone --recursive vs. git submodule commands as the latest tend to be more error prone and at the end of the day we are not doing anything fancy with git repos in the scm feature but a clone / checkout

[Minimonium]

@danimtb I start to think it'd be better to leave an option for users to not clone submodules, since they can be quite big. So a simple submodule: [none, normal, recursive] would suffice? That's why it'd be better to work with git submodule than git clone --recursive, allows for a more flexible experience.

[memsharded]

Implemented in https://github.com/conan-io/conan/pull/3114