Conan: Conan install cmd Errors on using options (-o).

[Twoflower2]

Ref: https://github.com/conan-io/conan/issues/1556 my last entry on this ERROR issuing conan info

As per Ref above, Lib1 was packaged with conan package_files and using options with config @memsharded suggested in Ref above.

As next step I'm trying to consume Lib1 by issuing a simple conan install but this also erroring. Below is my config for the consumer, the Conan cmd and the Error output.

conanfile.py

from conans import ConanFile
import os

class ConsumerConan(ConanFile):
    name = "Consumer"
    channel = "testing" 
    version = "1.0.0"   
    description = "Consumer"
    author = "user"
    url = "http://"
    license = "BSD"
    options = { "variations": ['d1', 'd2', 'd3'] }

    def requirements(self):
        self.requires("Lib1/1.2.4@user/testing" )

    def imports(self):
      self.copy("*", dst="inc", src="inc", root_package="Lib1") 
      self.copy("*", dst="lib", src="bin", root_package="Lib1")

install.py for the Consumer's conanfile

import os, shutil, sys

os.system("conan install . -o Lib1:variations=d1 -o variations=d1") 

Error

> python install.py

Requirements
    Lib1/1.2.4@user/testing from conan-local
Packages
    Lib1/1.2.4@user/testing:28bff110e580b5ac3ab51e5d05ee1e666409a05c

Lib1/1.2.4@user/testing: Already installed!
PROJECT: Generator txt created conanbuildinfo.txt
PROJECT: Generated conaninfo.txt
ERROR: Conanfile: 'variations' value not defined

[lasote]

Do you have both recipes Lib1 and consumer in a repository or zip? I don't understand where the error comes from, I can debug it. Also, I have some questions, in the consumer conanfile you are not using the options at all, have you tried to remove it?

[lasote]

Try to execute:

os.system("conan install . -o Lib1:variations=d1 -o Consumer:variations=d1")

[Twoflower2]

Hi @lasote

Do you have both recipes Lib1 and consumer in a repository or zip?

Lib1 resides in Artifactory (also in Conan local cache) which are placed into Artifactory as per #1556. The consumer artefacts will eventually also be placed into Artifactory using the same method.

I don't understand where the error comes from, I can debug it.

That will be good. When I add option values set in conanfile.py configure() method:

self.options.variations = ['d1', 'd2', 'd3']

these errors do not show up, but @memsharded have explained why this is not needed as per #1556

Also, I have some questions, in the consumer conanfile you are not using the options at all, have you tried to remove it?

Yes I tried removing it in the install like so:
os.system("conan install . -o Lib1:variations=d1
but get the same error.

Please let me what else you need.

[lasote]

I can't reproduce the error. Did you tried to specify both library names in the -o?

os.system("conan install . -o Lib1:variations=d1 -o Consumer:variations=d1")

[Twoflower2]

@lasote

Do you have both recipes Lib1 and consumer in a repository or zip?

'recipes'? The conanfile.py?

I don't understand where the error comes from, I can debug it. Also, I have some questions, in the consumer conanfile you are not using the options at all, have you tried to remove it?

Yes then it complains: ERROR: Conanfile: 'variations' value not defined

[Twoflower2]

@lasote

I can't reproduce the error. Did you tried to specify both library names in the -o?
os.system("conan install . -o Lib1:variations=d1 -o Consumer:variations=d1")

Yes and still an error.

Try to reproduce with two libs say Lib1 and Lib2, put into Artifactory with conan package_files with options ['d1', 'd2', 'd3']

ex:
os.system("conan package_files Lib1/1.2.4@user/testing --package_folder=123 -f -o Lib1:variations=d1" )

So then in consumer conanfile.py:

 def requirements(self):
        self.requires("Lib1/1.2.4@user/testing" )
        self.requires("Lib2/2.2.4@user/testing" )

and run this script:

os.system("conan install . -o Lib1:variations=d1 -o Consumer:variations=d1")
os.system("conan install . -o Lib2:variations=d1 -o Consumer:variations=d1")

[lasote]

It fails in the first line because you haven't specified a default value for Lib2:variations.

conan install . -o Lib1:variations=d1 -o Lib2:variations=d1 -o Consumer:variations=d1

[Twoflower2]

@lasote

It fails in the first line because you haven't specified a default value for Lib2:variations.

I do not understand what you mean. What default? Why so for Lib2? What about Lib1?

[lasote]

All the options need to have a value. you have several choices, if you specify the attribute default_options = "variations=d1" in the recipes you won't need to pass a -o if you don't want to override the default specified value.

So, you are not specifying a value for Lib2:variations=d1 in the first conan install execution, that's why I said that you need a value for Lib2:variations.

[Twoflower2]

So, you are not specifying a value for Lib2:variations=d1 in the first conan install execution, that's why I said that you need a value for Lib2:variations.

True, but the first install only is applicable on Lib1 (os.system("conan install . -o Lib1:variations=d1 -o Consumer:variations=d1")) I thought. Why is Lib2 also referenced? Is it because of the requirements() method?

Could you explain a bit what is happing behind the scenes (code logic) for a use case like this with two or more requirements the consumer might have?

[lasote]

If the consumer is requiring lib1 + lib2, if you install the consumer, the lib1 + lib2 will be retrieved, first the recipe (conanfile.py + exported files), it will compute the SHA for all the required binary packages and finally, the needed binary packages will be retrieved.

[Twoflower2]

Strange now: Using one install or all requirements in one cmd: conan install . -o Lib1:variations=d1 -o Lib2:variations=d1 -o Consumer:variations=d1

with:

def requirements(self):
        self.requires("Lib1/1.2.4@user/testing" )
        self.requires("Lib2/2.2.4@user/testing" )

    def imports(self):
        #Copy from Lib1
        self.copy("*", dst="../../../lib/Lib1/inc", src="inc", root_package="Lib1") 
        self.copy("*", dst="../../../lib/Lib1/lib", src="lib", root_package="Lib1") 

        #Copy from Lib2
        self.copy("*", dst="../../../lib/Lib2/inc", src="inc", root_package="Lib2") 
        self.copy("*", dst="../../../lib/Lib2/lib", src="lib", root_package="Lib2") 

seem to ignore the root_package="Lib2" and uses root_package="Lib1" for the Lib2 copy cmds, so that the Lib1 content also ends up in Lib2 location.

How can this be? (I'm on Conan 0.26)

[lasote]

Try to comment Lib1 copy's and make sure Lib2 artifacts are copied ok, then the opposite, comment Lib2 copy's. to make sure that the copy commands work as expected.

[Twoflower2]

My mistake, works now. Many thanks.

[Twoflower2]

Now, install two of the options variations=d1 and variations=d2 I should use two cmds.
conan install . -o Lib1:variations=d1 -o Lib2:variations=d1 -o Consumer:variations=d1
conan install . -o Lib1:variations=d2 -o Lib2:variations=d2 -o Consumer:variations=d2

Conanfile.py

     def requirements(self):
        self.requires("Lib1/1.2.4@user/testing" )
        self.requires("Lib2/2.2.4@user/testing" )

    def imports(self):
        #Copy from Lib1
        self.copy("*", dst="../../../lib/Lib1/d1/inc", src="d1/inc", root_package="Lib1") 
        self.copy("*", dst="../../../lib/Lib1/d1/lib", src="d1/lib", root_package="Lib1") 
        self.copy("*", dst="../../../lib/Lib1/d2/inc", src="d2/inc", root_package="Lib1") 
        self.copy("*", dst="../../../lib/Lib1/d2/lib", src="d2/lib", root_package="Lib1") 

        #Copy from Lib2
        self.copy("*", dst="../../../lib/Lib2/d1/inc", src="d1/inc", root_package="Lib2") 
        self.copy("*", dst="../../../lib/Lib2/d1/lib", src="d1/ib", root_package="Lib2")
        self.copy("*", dst="../../../lib/Lib2/d2/inc", src="d2/inc", root_package="Lib2") 
        self.copy("*", dst="../../../lib/Lib2/d2/lib", src="d2/lib", root_package="Lib2") 
 ```

It seems that Conan will never know that `src` is the `d1` or `d2` folder as it was packaged as. Conan will just do a copy when it matches like for 

`conan install . -o Lib1:variations=d1 -o Lib2:variations=d1 -o Consumer:variations=d1`

it will only do the copy for 

    self.copy("*", dst="../../../lib/Lib1/d1/inc", src="d1/inc", root_package="Lib1") 
    self.copy("*", dst="../../../lib/Lib1/d1/lib", src="d1/lib", root_package="Lib1")

````
the other copies will just be ignored/fail, right?

Any better way to do this?

[lasote]

You can read the option value to copy only the things you want to copy:

if self.options["Lib1"].variations ==  "inc":
    do some thing

[Twoflower2]

@lasote

To further and finally spice things up more; I added a requirement to the consumer for a 3rd lib, ConsLib which in itself consumes Lib1 too. The ConsLib artefacts are placed in Artifactory fine, so within ConsLib config/recipe the requirement Lib1 works just fine.

Now the working 2x Lib consumer as configured as just above, fails when adding ConsLib with:

ERROR: Lib1/1.2.4@user/testing: 'variations' value not defined

with the cmd:

conan install . -o Lib1:variations=d1 -o Lib2:variations=d1 -o ConsLib:variations=d1 -o Consumer:variations=d1

Consumer has changed adding ConsLib:

    def requirements(self):
        self.requires("Lib1/1.2.4@user/testing" )
        self.requires("Lib2/2.2.4@user/testing" )
        self.requires("ConsLib/3.2.4@user/testing" )


    def imports(self):
        #Copy from Lib1
        self.copy("*", dst="../../../lib/Lib1/d1/inc", src="d1/inc", root_package="Lib1") 
        self.copy("*", dst="../../../lib/Lib1/d1/lib", src="d1/lib", root_package="Lib1") 
        self.copy("*", dst="../../../lib/Lib1/d2/inc", src="d2/inc", root_package="Lib1") 
        self.copy("*", dst="../../../lib/Lib1/d2/lib", src="d2/lib", root_package="Lib1") 

        #Copy from Lib2
        self.copy("*", dst="../../../lib/Lib2/d1/inc", src="d1/inc", root_package="Lib2") 
        self.copy("*", dst="../../../lib/Lib2/d1/lib", src="d1/ib", root_package="Lib2")
        self.copy("*", dst="../../../lib/Lib2/d2/inc", src="d2/inc", root_package="Lib2") 
        self.copy("*", dst="../../../lib/Lib2/d2/lib", src="d2/lib", root_package="Lib2") 

        #Copy from ConsLib
        self.copy("*", dst="../../../lib/ConsLib/d1/inc", src="d1/inc", root_package="ConsLib") 
        self.copy("*", dst="../../../lib/ConsLib/d1/lib", src="d1/ib", root_package="ConsLib")
        self.copy("*", dst="../../../lib/ConsLib/d2/inc", src="d2/inc", root_package="ConsLib") 
        self.copy("*", dst="../../../lib/ConsLib/d2/lib", src="d2/lib", root_package="ConsLib") 

What can be the problem or better how can I remedy this kind of problem?

[lasote]

I've no idea, it should work the same for N requirements. I tried adding a new dependency to my previous test and it works fine. Check the spelling maybe?

[Twoflower2]

I tried adding a new dependency to my previous test and it works fine.

Does this new dependency depend on one of the already dependencies?

So you have no problems like in this example where a library Conslib depends on a library Lib1 and the consumer depends on library ConsLib and library Lib1?

[lasote]

Confirmed. No problem at all.

[Twoflower2]

@lasote Found my problem building the options string I pass to Conan install cmd, thanks.

Hopefully one last question:

The library ConsLib actually depends(requires) on libraries Lib1, Lib5 and Lib6, but the consumer only depends on library ConsLib and library Lib1 like said.

Now the consumer complains:
ERROR: Lib5/5.2.4@user/testing: 'variations' value not defined

I guess the same msg will be for Lib6, but Conan just complains on the first error found.

When I add the Lib5 and Lib6 options to the conan install cmd then these error/s go away.

Why should I do this because Lib5 and Lib6 are not as requirements in the consumer's conanfile.py and used only as requirements for the ConsLib's conanfile.py?

[Twoflower2]

@lasote or @memsharded Any reply on https://github.com/conan-io/conan/issues/1568#issuecomment-327435182 please?

[memsharded]

If I understood correctly, those libraries are also transitive dependencies of your consumer. As the consumer is requiring ConsLib, and ConsLib is requiring Lib5 and Lib6, they are all transitive dependencies of your consumer, and their state/configuration has to be fully defined, in this case, the `variations option.

You might be interested in a new feature that will be released in 0.28 (hopefully this week), is the patterns for options, so you can specify the same variation to all dependency graph like:

$ conan install .. -o *:variation=d1
# would be equivalent to -o Pkg1:variation=d1 -o Pkg2:variation=d2...

[Twoflower2]

@memsharded PERFECT!