Vcpkg: Package versioning suggestions

[cmpute]

This is a discussed topic, but I think there is no satisfying solution by now for version specifiying. Related discussions: #25, #1458, #1481, #1681.
Multiple versions installation is not the topic of this issue

I hope the ability to install certain version of a package can be added. For example, I want to add a port of V-REP, which depends on Lua5.1, but the current version of lua is 5.3 and Lua has many breaking changes since 5.1. It's not able to use Lua5.1 now because the initial version of Lua in vcpkg is 5.3

Current way to do version roll-back

git checkout <refspec> -- ports\<library>

Pros:

• > Minimizes the burden on library authors and maintainers (https://github.com/Microsoft/vcpkg/issues/1681#issuecomment-324160402)
• Minimizes the possibility to break the dependency tree.

Cons:

• Unable to roll back to versions before vcpkg was developed.
• Hard to find the git checksum for certain version of an package.
• Unable to use a new version of port B if roll back port A to when port B is not updated in time.

The proposed ways that could help

vcpkg pin <library> <version> or vcpkg install <library>=<version>

The details in my opinion:

• The port folder structure changes from
  ports |--- <library> |--- CONTROL |--- portfile.cmake |--- ...
  to
  ports |--- <library> |--- <version> |--- CONTROL |--- portfile.cmake |--- ... |--- <version> |--- CONTROL |--- portfile.cmake |--- ... |--- latest |--- CONTROL |--- portfile.cmake |--- ...

The `<version>` may be a pattern that can match a series of versions like `1.2.x` (or `1.2._`) matches `1.2.3` and `1.2.4`. When the maintainer wants to update the version of a library, you just have to edit the files in the `latest` folder, and move the original files into corresponding verison folder **if necessary** (if the upgrade doesn't break something and a existing version pattern covers the old version, you can just edit the file in the `latest` folder).

• Dependency specifying in the CONTROL file changes from
  Build-Depends: dp1, dp2, ...
  to
  Build-Depends: dp1=<ver1>, dp2=<ver2>, ...

Here versions are match patterns as well.

• If a certain version of library is installed, vcpkg install <library>=<another version> will remove the previous version. If existing version of package A doesn't meet the demand of package B, then vcpkg install B will remove the current version of A.

[cmpute]

In addition, if this feature is going to be added, it should be implemented in time to reduce future maintaining burden.

[sldr]

Yes please add support for supporting multipole versions of a library.

Thanks,
SLDR
(Stephen L. De Rudder)

[s-martin]

It would be really great, if multi version support could be implemented.

Until then vcpkg is no real solution for our projects, because we use many different versions of libraries depending on external requirements.

[ras0219-msft]

if multi version support could be implemented.

@s-martin Could you go into detail about what your scenario is? Also, have you tried using multiple vcpkg instances to manage each set of library versions? (Over email at [email protected] would be great too!)

[s-martin]

Currently I would need to checkout two different vcpkg instances at different revisions to accomplish that, if I understand vcpkg correctly. From my point of view this approch wouldn't work, if I have multiple dependencies with different versions, e.g.

• Project A uses Lib X with version 1.2, Lib Y with version 2.5 and Lib Z with version 1.0
• Project B uses Lib X with version 1.1 and Lib Z with version 1.1.2

To really solve my problems with dependency libs I would like to define:

• Project A uses Lib X:1.2, Lib Y:2.5, Lib Z:1.0
• Project B uses Lib X:1.1, Lib Z:1.1.2
• Project C uses Lib U:1.0

To be honest, I'm don't know, if any other package manager does this at the moment, but this would be a unique selling point to me.

[ras0219-msft]

We envision this as being handled by three different vcpkg instances. For example, Project A has a local vcpkg\ subdirectory with the commit for [email protected], [email protected], [email protected]. Since the entire system (including all supporting machinery such as the tool and helper scripts) is versioned together, you get guaranteed reproducibility across machines as long as you check out that same commit for vcpkg.

B and C each get their own separate copies as well.

Of course, this only makes sense when A and B aren't going to be loaded into the same process -- otherwise the diamond dependency on X at different versions will cause undefined behavior for at least one of them! Preventing this diamond clash is why a single vcpkg instance carries a single version for a particular dependency; as long as everyone who wants to share a process shares the same vcpkg instance, you're guaranteed to get consistent versions.

[s-martin]

Thanks for the info. That’s more or less what I concluded from the docs.

[s-martin]

But how can I ensure that all required libs have the version I need?

E.g. if X:1.2 is only available in commit abcd, but Y has already version 3.1 in that commit?

[cmpute]

if X:1.2 is only available in commit abcd, but Y has already version 3.1 in that commit?

@s-martin That is exactly what I posted this issue for!

[MattRyder]

I hope there's some movement towards an implementation of this, as I'm currently migrating to vcpkg and one of the libraries being moved over is now unworkable within the system.

I'm trying to use spdlog, but the vcpkg port (quite logically) installs the vcpkg version of a dependency fmt v5.0. The spdlog library is still yet to implement the changes required to support the API for fmt v5.0, so I'll have to fudge vcpkg to get v4.0 installed, then install spdlog, I presume.

It's possibly the only problem I've encountered using the tool, I hope there's a decent resolution to versioning soon. I'd also like to see support for pessimistic version constraints down the road.

EDIT: The portfile for spdlog just got updated the other night. Terrific...! But my spdlog client in GibEngine hasn't been updated, ergo, CI builds fail while pulling an unconstrained version.

An incredible, long-needed, non-sucky package manager for C++ and it's heel is donated from Achilles himself. I see vcpkg builds vcpkglib, maybe I can modify it to include internal git checkout <ref> ports\<package> checks. It appears it's the dependency graph which is the killer here.

[justinmeiners]

@ras0219-msft

Can more documentation be provided about the rationale behind having a single version, and how multiple instances can resolve this? The current docs have only a sentence or two. It sounds like you and your team have thought this through, but the many comments relating to this indicate it is not being communicated.

[paulsapps]

"An incredible, long-needed, non-sucky package manager for C++ and it's heel is donated from Achilles himself."

I agree, not being able to pin to certain versions actually makes this some what useless. If I do a build machine build its impossible to reproduce the build if its always using the latest of everything :(

[aaronang]

Hope my comment isn't just noise but I got very excited finding out that vcpkg supports macOS and Linux. However, I agree with the views expressed in this issue that vcpkg is not as user-friendly when there is no version pinning available.

Are there any plans of making this a reality? I would love to help out if needed. Please let me know how I can help out making this a reality 😁

[ras0219-msft]

...machine build its impossible to reproduce the build if its always using the latest of everything

@paulsapps You're absolutely right -- this is why we do support pinning the entire system very easily! Simply check out a specific commit and the entire system (tool and library versions) will be pinned.

You can get your current commit by running:

$ git rev-parse HEAD
a3785d8eeaf380aec41490f7f92a06cf2193be8a

Then, you can add this to your build script to always get exactly the versions you want by checking out the specific commit id:

$ git checkout a3785d8eeaf380aec41490f7f92a06cf2193be8a

Then do the normal steps:

$ ./bootstrap-vcpkg.sh
$ ./vcpkg remove --outdated --recurse
$ ./vcpkg install <libraries>

[MattRyder]

I'd like to leave my travis-install.sh file, with a working example of how I'm doing version constraints within Travis-CI builds where you want to run the latest vcpkg repo but leave a few packages behind.

It's probably not as tight as it could be syntax-wise, but I can guarantee it'll work.

Hopefully it'll help someone get up and running.

[MichaelMitchellM]

@MattRyder How do you find the commit hashes? Do you search for "fmt" and look through all the commits to see which one has the version you want?

[MattRyder]

@MattRyder How do you find the commit hashes? Do you search for "fmt" and look through all the commits to see which one has the version you want?

@MichaelMitchellM Usually by checking the git history of the portfile.cmake file, for example, fmt/portfile.cmake.

Then figure out where I'd like to pin the version at, and grab the SHA hash from there. In my case with this issue, the latest I could apply before v5.0 came about, so I settled on v4.1.

[gocarlos]

why not to adapt to something like npm does? each package shows its dependencies with the versions...

I think that checking out different version of this repo and tracking commits its a no go...
ideally vcpkg should be run together with cmake during the configure step and get the version number from the find_package command...

the thing with the buildtool also does not work well everywhere, during e.g. cross compilation with yocto/openembedded we generate already custom toolchain files

[justinmeiners]

@gocarlos

The npm model works because of JavaScripts runtime system. Its totally acceptable to have multiple versions of a file in the same process. In C and C++ you have symbol collision and this does not work. You can only have one version of each dependency in a single process.

[gocarlos]

@justinmeiners true, but I thing that in 95% of the projects this problem does not happen as we habe only one version at the end...

if you have version ranges it will likelly not be a problem, e.g. if lib a suports version of 1 to 3 of lib b and lib c only supports lib version 2 of b there is no problem, we take version 2 of lib b... otherwise its not prossible and there would be an error/warning...

[justinmeiners]

@gocarlos Definitely. But, then it starts to look like cabal, and every other package manager.

It just complicates things quite a bit to resolve all those constraints for each dependency and then it introduces the possibility of two dependencies not being compatible. I think the vcpkg idea is to favor simplicity and avoid that issue all together. (That's what I gathered anyway, but I I can't speak for the dev team.)

[tstavrianos]

Although this would be a great feature, how would it work with the current structure of the vcpkg->installed folder? Wouldn't it require a separate folder for each package+triplet+version instead of one per triplet as it is now?

[MichaelMitchellM]

@tstavrianos Maybe in each projects' folder in buildtrees there would be a version folder for each built version. Then, the main installed directory would be for the version that is being used. To support this, there would need to be a vcpkg command that does a "checkout" on a version of a package. So you could have multiple built versions and then set which one you want in installed by telling vcpkg that's the one you want to use right now.

[soroshsabz]

vcpkg must be support versioning for production grade projects, because we cannot upgrade to newer dependency in enterprise project ( specific version of dependency maybe stable and newer version maybe not stable ), so vcpkg must be provide mechanism to install specific version of library, like Debian ( I can install specific version of boost with apt command ), but I think vcpkg can very better support to versioning than Debian.

I think this reddit discussion about comparison between vcpkg and conan and build2 is interesting.

[david-antiteum]

I have coded a small prototype to deal with versions. The basic ideas are there but is far from perfect (and probably I´m not considering scenarios that makes this solution nonviable). I wrote a blog post with the basics ideas: http://www.antiteum.com/2019/05/vcpkg-versioning.html

The repository is here: https://github.com/david-antiteum/vcpck-versions

[vpaladino778]

Check out my comment here. It's similar to your @cmpute 's idea, except that instead of a latest folder, the latest would just be on the top level. I think this would be better because packages can stay exactly the same and they'd still work as intended. No reason to place an additional 'latest' folder inside each and every single portfile.

[maksqwe]

Check out already merged PR [latest] feature for the Qt5 libraries.
https://github.com/microsoft/vcpkg/pull/8212

[cbezault]

And in general ports can support the latest version of a library by using the HEAD_REF variable when getting source from github.

[Bennett-Yang]

Considering vcpkg has more packages, I will give up conan if vcpkg solves this problem.

[niitsuma]

Aim of vcpkg is same to gentoo-prefix.
Just import gentoo-prefix way will resolve this.