conan create -> conan upload

[lasote]

It will create a file with the reference created (or the install reference if specified). It will be a pref (without revision).
Together with #5196 would alleviate nicely the conan create -> conan upload typical CI flow that now requires painful parsings.

@solvingj

[solvingj]

Thanks for doing this. I guess it’s trivial to implement and it uses an existing pattern, and is indeed better than the current solution.

[memsharded]

This is not possible right now, because conan create doesn't allow generators as an option (it doesn't have a --install-folder either). So we would need some other way or flow to implement this.

I think this is a good idea and necessary, just we need to think a bit more how to implement this, not possible for next 1.16, which is too close, lets move it to 1.17

[solvingj]

We could easily do conan create, conan install -g ref, conan upload.

[lasote]

Mhh, but the install will install the deps so won't tell you the ref of the package being created.

[solvingj]

Bringing forward relevant comments from https://github.com/conan-io/conan/pull/5224

Additionally, following the syntax used by other programs like gcc, conan upload @filepath could read the reference from a file.

Followed by

Probably not a good idea, we plan to be able to specify references like @user/channel to disambiguate.

I see the logic of @filepath pointing to a text file which contains nothing but a reference, and I also see the issue that this conflicts with another plan to accept @user/channel in some context, though it's not clear in which context that is being planned for. Can you clarify the current thinking about this issue?

I'm about to refactor our "upload" code again, want to know if I can wait for 1.17 for something first-class.

[lasote]

So, @memsharded (correct me if I'm wrong) was thinking about generating, as part of the install process, a file with the reference that might help with the local commands also. The lockfile feature will allow the conan create to specify an install_folder so that file might be generated also.

About taking it automatically with a first class mechanism, we just need to find a good syntax for it, as you read, the @file syntax can conflict with the reference, because we are thinking about disambiguate the paths vs references by always forcing the @ (Conan 2.0).

So, let's suggest some!

The round brackets could help disambiguate? but probably forces us to quote the string to avoid shell interpretations

"@(path_to_file)" 
"f(path_to_file)"
"file(path_to_file)"
f#path_to_file
...

[lasote]

It has to wait until we decide how and when a file with the reference is generated, still not clarified. Sorry.

[danimtb]

Hi @solvingj,

I was wondering if you have tried the jq utility (https://stedolan.github.io/jq/) together with the JSON output. An expression like this should get the reference for the next command:

$ conan create hello/1.0@user/channel --json create.json
...
$ jq ."installed"[0]."recipe"."id" create.json
"hello/1.0@user/channel"

[solvingj]

Yes, i'm familiar with that, I actually wanted to write such a tool myself two years ago, but found it already existed and was really amazing. In the context of this discussion, sadly it represents third-party executable dependency just to take text output from one conan command, and use it in a following conan command. It just doesn't seem reasonable as a strategy for one of the most common "command pairs" in all of the conan workflows.

[memsharded]

My current view after implementing the lockfiles, and what is done with the graph_info.json, etc:

• Conan commands like export, install or create should save one or more files with the reference of the created or the installed package.
• There could be more than one file, like "conan.ref" to contain the reference, "conan.pref", to contain the package reference, or if it becomes necessary, "conan.rref" to include the revision too. Things like that.
• I like the possibility to reference them directly in the command line with some syntax. "@" might be challenging/confusing for its current usage, so maybe think a different pattern: $ conan upload &conan.ref .... or something similar. Yes, brackets or other of the above also looks good.
• It is not a generator, it is built-in in conan. For commands like conan create which doesn't generate output in the current folder, it might be done only if --lockfile is specified, or maybe some --ref or other argument.
• The conan.ref file will also be automatically used by other commands. If it is present (maybe typically besides the "conan.lock" file), it will be used (as it is more or less done now with the info from graph_info.json
• graph_info.json should be deprecated (probably Conan 2.0)

[solvingj]

seems like you have a clear perspective and strong vision for how to make this work. everything you said makes sense, looking forward to it.

[lasote]

@memsharded please open an issue with these ideas to plan accordingly.

[lasote]

Latest news. I'm closing this issue and we will open 2 new ones:

1. For generating a "conan.ref" and "conan_package.ref" or whatever file names as an output of the conan create.

2. Allow an special pattern like f(path/to/file) (format to be defined) to be used in the "conan upload" and others probably. Conan will automatically read the contents of the file an will apply it as an argument.

[lasote]

https://github.com/conan-io/conan/issues/5744
https://github.com/conan-io/conan/issues/5743