Conda-forge.github.io: Please hotfix bob packages for their boost dependency

[183amir]

A lot of bob packages link to boost but they either did not pin boost or did not list boost as a dependency.
I have compiled a list of these packages. Please fix them (don't delete them) using the script I have provided here: https://github.com/conda-forge/conda-forge.github.io/pull/170

$ conda execute conda-forge.github.io/scripts/hotfix.py conda-forge /tmp/infile.txt -r 'boost 1.60.*'
$ cat /tmp/infile.txt
bob.blitz 2.0.8 np110py27_0 linux-64: boost
bob.blitz 2.0.8 np110py27_1 linux-64: boost
bob.blitz 2.0.8 np110py27_2 linux-64: boost
bob.blitz 2.0.8 np110py27_3 linux-64: boost
bob.blitz 2.0.8 np110py27_4 linux-64: boost
bob.blitz 2.0.8 np110py27_5 linux-64: boost
bob.blitz 2.0.8 np110py27_6 linux-64: boost
bob.blitz 2.0.8 np110py34_0 linux-64: boost
bob.blitz 2.0.8 np110py34_1 linux-64: boost
bob.blitz 2.0.8 np110py34_2 linux-64: boost
bob.blitz 2.0.8 np110py34_3 linux-64: boost
bob.blitz 2.0.8 np110py34_4 linux-64: boost
bob.blitz 2.0.8 np110py34_5 linux-64: boost
bob.blitz 2.0.8 np110py34_6 linux-64: boost
bob.blitz 2.0.8 np110py35_0 linux-64: boost
bob.blitz 2.0.8 np110py35_1 linux-64: boost
bob.blitz 2.0.8 np110py35_2 linux-64: boost
bob.blitz 2.0.8 np110py35_3 linux-64: boost
bob.blitz 2.0.8 np110py35_4 linux-64: boost
bob.blitz 2.0.8 np110py35_5 linux-64: boost
bob.blitz 2.0.8 np110py35_6 linux-64: boost
bob.blitz 2.0.8 np111py27_0 linux-64: boost
bob.blitz 2.0.8 np111py27_1 linux-64: boost
bob.blitz 2.0.8 np111py27_2 linux-64: boost
bob.blitz 2.0.8 np111py27_3 linux-64: boost
bob.blitz 2.0.8 np111py27_4 linux-64: boost
bob.blitz 2.0.8 np111py27_5 linux-64: boost
bob.blitz 2.0.8 np111py27_6 linux-64: boost
bob.blitz 2.0.8 np111py34_0 linux-64: boost
bob.blitz 2.0.8 np111py34_1 linux-64: boost
bob.blitz 2.0.8 np111py34_2 linux-64: boost
bob.blitz 2.0.8 np111py34_3 linux-64: boost
bob.blitz 2.0.8 np111py34_4 linux-64: boost
bob.blitz 2.0.8 np111py34_5 linux-64: boost
bob.blitz 2.0.8 np111py34_6 linux-64: boost
bob.blitz 2.0.8 np111py35_0 linux-64: boost
bob.blitz 2.0.8 np111py35_1 linux-64: boost
bob.blitz 2.0.8 np111py35_2 linux-64: boost
bob.blitz 2.0.8 np111py35_3 linux-64: boost
bob.blitz 2.0.8 np111py35_4 linux-64: boost
bob.blitz 2.0.8 np111py35_5 linux-64: boost
bob.blitz 2.0.8 np111py35_6 linux-64: boost
bob.blitz 2.0.8 np110py27_0 osx-64 : boost
bob.blitz 2.0.8 np110py27_1 osx-64 : boost
bob.blitz 2.0.8 np110py27_2 osx-64 : boost
bob.blitz 2.0.8 np110py27_3 osx-64 : boost
bob.blitz 2.0.8 np110py27_4 osx-64 : boost
bob.blitz 2.0.8 np110py27_5 osx-64 : boost
bob.blitz 2.0.8 np110py34_0 osx-64 : boost
bob.blitz 2.0.8 np110py34_1 osx-64 : boost
bob.blitz 2.0.8 np110py34_2 osx-64 : boost
bob.blitz 2.0.8 np110py34_3 osx-64 : boost
bob.blitz 2.0.8 np110py34_4 osx-64 : boost
bob.blitz 2.0.8 np110py34_5 osx-64 : boost
bob.blitz 2.0.8 np110py35_0 osx-64 : boost
bob.blitz 2.0.8 np110py35_1 osx-64 : boost
bob.blitz 2.0.8 np110py35_2 osx-64 : boost
bob.blitz 2.0.8 np110py35_3 osx-64 : boost
bob.blitz 2.0.8 np110py35_4 osx-64 : boost
bob.blitz 2.0.8 np110py35_5 osx-64 : boost
bob.blitz 2.0.8 np110py35_6 osx-64 : boost
bob.blitz 2.0.8 np111py27_0 osx-64 : boost
bob.blitz 2.0.8 np111py27_1 osx-64 : boost
bob.blitz 2.0.8 np111py27_2 osx-64 : boost
bob.blitz 2.0.8 np111py27_3 osx-64 : boost
bob.blitz 2.0.8 np111py27_4 osx-64 : boost
bob.blitz 2.0.8 np111py27_5 osx-64 : boost
bob.blitz 2.0.8 np111py27_6 osx-64 : boost
bob.blitz 2.0.8 np111py34_0 osx-64 : boost
bob.blitz 2.0.8 np111py34_1 osx-64 : boost
bob.blitz 2.0.8 np111py34_2 osx-64 : boost
bob.blitz 2.0.8 np111py34_3 osx-64 : boost
bob.blitz 2.0.8 np111py34_4 osx-64 : boost
bob.blitz 2.0.8 np111py34_5 osx-64 : boost
bob.blitz 2.0.8 np111py34_6 osx-64 : boost
bob.blitz 2.0.8 np111py35_0 osx-64 : boost
bob.blitz 2.0.8 np111py35_1 osx-64 : boost
bob.blitz 2.0.8 np111py35_2 osx-64 : boost
bob.blitz 2.0.8 np111py35_3 osx-64 : boost
bob.blitz 2.0.8 np111py35_4 osx-64 : boost
bob.blitz 2.0.8 np111py35_5 osx-64 : boost
bob.blitz 2.0.8 np111py35_6 osx-64 : boost
bob.core 2.1.2 py27_0 linux-64     :
bob.core 2.1.2 py27_1 linux-64     :
bob.core 2.1.2 py27_2 linux-64     :
bob.core 2.1.2 py27_3 linux-64     :
bob.core 2.1.2 py27_4 linux-64     :
bob.core 2.1.2 py34_0 linux-64     :
bob.core 2.1.2 py34_1 linux-64     :
bob.core 2.1.2 py34_2 linux-64     :
bob.core 2.1.2 py34_3 linux-64     :
bob.core 2.1.2 py34_4 linux-64     :
bob.core 2.1.2 py35_0 linux-64     :
bob.core 2.1.2 py35_1 linux-64     :
bob.core 2.1.2 py35_2 linux-64     :
bob.core 2.1.2 py35_3 linux-64     :
bob.core 2.1.2 py35_4 linux-64     :
bob.math 2.0.3 np110py27_nomkl_1 linux-64: boost
bob.math 2.0.3 np110py27_nomkl_2 linux-64: boost
bob.math 2.0.3 np110py27_nomkl_3 linux-64: boost
bob.math 2.0.3 np110py34_nomkl_1 linux-64: boost
bob.math 2.0.3 np110py34_nomkl_2 linux-64: boost
bob.math 2.0.3 np110py34_nomkl_3 linux-64: boost
bob.math 2.0.3 np110py35_nomkl_1 linux-64: boost
bob.math 2.0.3 np110py35_nomkl_2 linux-64: boost
bob.math 2.0.3 np110py35_nomkl_3 linux-64: boost
bob.math 2.0.3 np111py27_nomkl_1 linux-64: boost
bob.math 2.0.3 np111py27_nomkl_2 linux-64: boost
bob.math 2.0.3 np111py27_nomkl_3 linux-64: boost
bob.math 2.0.3 np111py34_nomkl_1 linux-64: boost
bob.math 2.0.3 np111py34_nomkl_2 linux-64: boost
bob.math 2.0.3 np111py34_nomkl_3 linux-64: boost
bob.math 2.0.3 np111py35_nomkl_1 linux-64: boost
bob.math 2.0.3 np111py35_nomkl_2 linux-64: boost
bob.math 2.0.3 np111py35_nomkl_3 linux-64: boost
bob.math 2.0.3 py27_nomkl_0 linux-64: boost
bob.math 2.0.3 py34_nomkl_0 linux-64: boost
bob.math 2.0.3 py35_nomkl_0 linux-64: boost
bob.math 2.0.3 np110py27_nomkl_1 osx-64: boost
bob.math 2.0.3 np110py27_nomkl_2 osx-64: boost
bob.math 2.0.3 np110py27_nomkl_3 osx-64: boost
bob.math 2.0.3 np110py34_nomkl_1 osx-64: boost
bob.math 2.0.3 np110py34_nomkl_2 osx-64: boost
bob.math 2.0.3 np110py34_nomkl_3 osx-64: boost
bob.math 2.0.3 np110py35_nomkl_1 osx-64: boost
bob.math 2.0.3 np110py35_nomkl_2 osx-64: boost
bob.math 2.0.3 np110py35_nomkl_3 osx-64: boost
bob.math 2.0.3 np111py27_nomkl_1 osx-64: boost
bob.math 2.0.3 np111py27_nomkl_2 osx-64: boost
bob.math 2.0.3 np111py27_nomkl_3 osx-64: boost
bob.math 2.0.3 np111py34_nomkl_1 osx-64: boost
bob.math 2.0.3 np111py34_nomkl_2 osx-64: boost
bob.math 2.0.3 np111py34_nomkl_3 osx-64: boost
bob.math 2.0.3 np111py35_nomkl_1 osx-64: boost
bob.math 2.0.3 np111py35_nomkl_2 osx-64: boost
bob.math 2.0.3 np111py35_nomkl_3 osx-64: boost
bob.math 2.0.3 py27_nomkl_0 osx-64 : boost
bob.math 2.0.3 py34_nomkl_0 osx-64 : boost
bob.math 2.0.3 py35_nomkl_0 osx-64 : boost
bob.io.matlab 2.0.4 py27_0 linux-64: 
bob.io.matlab 2.0.4 py27_3 linux-64: 
bob.io.matlab 2.0.4 py34_0 linux-64: 
bob.io.matlab 2.0.4 py34_3 linux-64: 
bob.io.matlab 2.0.4 py35_0 linux-64: 
bob.io.matlab 2.0.4 py35_3 linux-64: 
bob.io.matlab 2.0.4 py27_0 osx-64  : 
bob.io.matlab 2.0.4 py27_3 osx-64  : 
bob.io.matlab 2.0.4 py34_0 osx-64  : 
bob.io.matlab 2.0.4 py34_3 osx-64  : 
bob.io.matlab 2.0.4 py35_0 osx-64  : 
bob.io.matlab 2.0.4 py35_3 osx-64  : 
bob.io.audio 2.0.0 py27_0 linux-64 : 
bob.io.audio 2.0.0 py27_1 linux-64 : 
bob.io.audio 2.0.0 py34_0 linux-64 : 
bob.io.audio 2.0.0 py34_1 linux-64 : 
bob.io.audio 2.0.0 py35_0 linux-64 : 
bob.io.audio 2.0.0 py35_1 linux-64 : 
bob.io.audio 2.0.0 py27_1 osx-64   : 
bob.io.audio 2.0.0 py34_1 osx-64   : 
bob.io.audio 2.0.0 py35_1 osx-64   : 
bob.measure 2.1.1 py27_0 linux-64  : 
bob.measure 2.1.1 py27_1 linux-64  : 
bob.measure 2.1.1 py34_0 linux-64  : 
bob.measure 2.1.1 py34_1 linux-64  : 
bob.measure 2.1.1 py35_0 linux-64  : 
bob.measure 2.1.1 py35_1 linux-64  : 
bob.measure 2.1.1 py27_1 osx-64    : 
bob.measure 2.1.1 py34_1 osx-64    : 
bob.measure 2.1.1 py35_1 osx-64    : 
bob.io.video 2.0.6 py27_0 linux-64 : 
bob.io.video 2.0.6 py27_1 linux-64 : 
bob.io.video 2.0.6 py34_0 linux-64 : 
bob.io.video 2.0.6 py34_1 linux-64 : 
bob.io.video 2.0.6 py35_0 linux-64 : 
bob.io.video 2.0.6 py35_1 linux-64 : 
bob.io.video 2.0.6 py27_1 osx-64   : 
bob.io.video 2.0.6 py34_1 osx-64   : 
bob.io.video 2.0.6 py35_1 osx-64   : 
bob.learn.activation 2.0.4 py27_0 linux-64: 
bob.learn.activation 2.0.4 py27_1 linux-64: 
bob.learn.activation 2.0.4 py34_0 linux-64: 
bob.learn.activation 2.0.4 py34_1 linux-64: 
bob.learn.activation 2.0.4 py35_0 linux-64: 
bob.learn.activation 2.0.4 py35_1 linux-64: 
bob.learn.activation 2.0.4 py27_1 osx-64: 
bob.learn.activation 2.0.4 py34_1 osx-64: 
bob.learn.activation 2.0.4 py35_1 osx-64: 
bob.learn.boosting 2.0.7 py27_0 linux-64: 
bob.learn.boosting 2.0.7 py27_1 linux-64: 
bob.learn.boosting 2.0.7 py34_0 linux-64: 
bob.learn.boosting 2.0.7 py34_1 linux-64: 
bob.learn.boosting 2.0.7 py35_0 linux-64: 
bob.learn.boosting 2.0.7 py35_1 linux-64: 
bob.learn.boosting 2.0.7 py27_1 osx-64: 
bob.learn.boosting 2.0.7 py34_1 osx-64: 
bob.learn.boosting 2.0.7 py35_1 osx-64: 
bob.learn.libsvm 2.0.3 py27_0 linux-64: 
bob.learn.libsvm 2.0.3 py27_1 linux-64: 
bob.learn.libsvm 2.0.3 py34_0 linux-64: 
bob.learn.libsvm 2.0.3 py34_1 linux-64: 
bob.learn.libsvm 2.0.3 py35_0 linux-64: 
bob.learn.libsvm 2.0.3 py35_1 linux-64: 
bob.ip.color 2.0.4 py27_0 linux-64 : 
bob.ip.color 2.0.4 py27_1 linux-64 : 
bob.ip.color 2.0.4 py34_0 linux-64 : 
bob.ip.color 2.0.4 py34_1 linux-64 : 
bob.ip.color 2.0.4 py35_0 linux-64 : 
bob.ip.color 2.0.4 py35_1 linux-64 : 
bob.ip.color 2.0.4 py27_1 osx-64   : 
bob.ip.color 2.0.4 py34_1 osx-64   : 
bob.ip.color 2.0.4 py35_1 osx-64   : 
bob.ip.gabor 2.0.5 py27_0 linux-64 : 
bob.ip.gabor 2.0.5 py27_1 linux-64 : 
bob.ip.gabor 2.0.5 py34_0 linux-64 : 
bob.ip.gabor 2.0.5 py34_1 linux-64 : 
bob.ip.gabor 2.0.5 py35_0 linux-64 : 
bob.ip.gabor 2.0.5 py35_1 linux-64 : 
bob.ip.gabor 2.0.5 py27_1 osx-64   : 
bob.ip.gabor 2.0.5 py34_1 osx-64   : 
bob.ip.gabor 2.0.5 py35_1 osx-64   : 
bob.learn.linear 2.0.7 py27_0 linux-64: 
bob.learn.linear 2.0.7 py27_1 linux-64: 
bob.learn.linear 2.0.7 py34_0 linux-64: 
bob.learn.linear 2.0.7 py34_1 linux-64: 
bob.learn.linear 2.0.7 py35_0 linux-64: 
bob.learn.linear 2.0.7 py35_1 linux-64: 
bob.learn.linear 2.0.7 py27_1 osx-64: 
bob.learn.linear 2.0.7 py34_1 osx-64: 
bob.learn.linear 2.0.7 py35_1 osx-64: 
bob.learn.mlp 2.0.11 py27_0 linux-64: 
bob.learn.mlp 2.0.11 py27_1 linux-64: 
bob.learn.mlp 2.0.11 py34_0 linux-64: 
bob.learn.mlp 2.0.11 py34_1 linux-64: 
bob.learn.mlp 2.0.11 py35_0 linux-64: 
bob.learn.mlp 2.0.11 py35_1 linux-64: 
bob.learn.mlp 2.0.11 py27_1 osx-64 : 
bob.learn.mlp 2.0.11 py34_1 osx-64 : 
bob.learn.mlp 2.0.11 py35_1 osx-64 : 
bob.learn.em 2.0.8 py27_0 linux-64 : 
bob.learn.em 2.0.8 py27_1 linux-64 : 
bob.learn.em 2.0.8 py34_0 linux-64 : 
bob.learn.em 2.0.8 py34_1 linux-64 : 
bob.learn.em 2.0.8 py35_0 linux-64 : 
bob.learn.em 2.0.8 py35_1 linux-64 : 
bob.learn.em 2.0.8 py27_1 osx-64   : 
bob.learn.em 2.0.8 py34_1 osx-64   : 
bob.learn.em 2.0.8 py35_1 osx-64   : 
bob.ip.facedetect 2.0.7 py27_0 linux-64: 
bob.ip.facedetect 2.0.7 py34_0 linux-64: 
bob.ip.facedetect 2.0.7 py35_0 linux-64: 

There are 250 packages in the list.
The script may need further modifications in the uploading part but the person who will do this will know how to edit it. (probably you need to add -t token.binstar and -c main)
Thank you.

[183amir]

@ocefpaf would you be willing to look into this please?

[ocefpaf]

Not sure what you need me here for?

I am against "hotfix" as IMO any fix like that deserves a build number bump for proper reproducibility and provenance of the change. Although, you have 250 packages there!

Maybe the best approach is to use MNT PRs to update the pinnings. I will check with @pelson an strategy for this.

[183amir]

deserves a build number bump

They all have a gotten a build number bump. Maybe you are not a good guy for this :) I thought you agree to hotfixes. The reasons are very well explained here: https://github.com/conda-forge/conda-forge.github.io/pull/170 and here: https://github.com/conda-forge/conda-forge.github.io/issues/126 I am not going to explain it here.
@jakirkham could you look into this please? Can we at least move them from conda-forge/main to conda-forge/archive or something? I don't want to delete them.

[ocefpaf]

Can we at least move them from conda-forge/main to conda-forge/archive or something? I don't want to delete them.

That is definitely the right way to go.

Maybe you are not a good guy for this :)

I followed that thread for a while and I don't know if a decision was made, but if you bumped the build number there is nothing to do here besides archiving the old builds.

We don't have an automated way for that yet, but writing a script that moves them should not be too difficult.

[183amir]

besides archiving the old builds.

Yes but either archiving or hotfixing them should be done before I can continue to work and iterate in conda-forge. This is blocking my work, I need it to be resolved as soon as possible.

[pelson]

I am against "hotfix" as IMO any fix like that deserves a build number bump for proper reproducibility and provenance of the change.

I agree. Because conda has a local package cache hotfixing neither supports reproducibility _nor_ fixing the stack for everybody, and in general I'm against it.

@183amir - definitely happy to help here, but please don't get a negative response from one person and then immediately request the same thing of somebody else. I understand that this is a blocker for you, but that kind of approach will ultimately lead to conflict that is entirely avoidable.

So, in summary, would removing those packages listed above be helpful @183amir?

[183amir]

I only asked someone else because I thought we had an understanding why I need these packages hotfixed. Sorry about that.

So, in summary, would removing those packages listed above be helpful @183amir?

No, removing them would take away reproducibility. Please archive them in anyway you see fit.

[ocefpaf]

but please don't get a negative response from one person

I did not mean to be negative :smile: I am just trying to understand the issue and figure out a proper strategy.

and then immediately request the same thing of somebody else

True. We are a community and someone else cannot do/undo what is still under discussion.

So, in summary, would removing those packages listed above be helpful @183amir?

I guess that @183amir does not want them deleted, right?

Are you willing to download them locally and archive them locally until we figure out a script to move them to another channel? (Although I believe writing that script would be easier. I am at SciPy now and my time is limited this week, but I am interested in giving that a try.)

This is blocking my work, I need it to be resolved as soon as possible.

I understand that. My work has been blocked by conda-forge's changes/policies for months (the numpy+openblas issue).

However, the right way forward is to do what we can and keep discussing it until we reach an agreement and a solution that serves the whole community. Remember that people are not paid to solve any specific issues here, and particular/specific solutions affects everybody.

(If it was up to my I would have removed the slow single threaded openblas months ago and got my stack back on track, but I am waiting on the community as everybody else :smile:)

[183amir]

I understand that. My work has been blocked by conda-forge's changes/policies for months (the numpy+openblas issue).

I have been waiting for months too. I am pushing this because not only that discussing it is getting boring: https://github.com/conda-forge/conda-forge.github.io/pull/170#issuecomment-229377350 https://github.com/conda-forge/conda-forge.github.io/pull/170#issuecomment-229653905 but also we need to come up with a policy around this soon.

This is a problem that's gonna to happen to all packages and not just bob packages. An example would be updating the ffmpeg package to version 3; every package that depended on it is gonna break. Did you guys know that you should also pin ffmpeg too? probably yes if you thought about it. So the situation is when we want to update ffmpeg to version 3 we wait until all packages that build against ffmpeg get a pinned version with ffmpeg 2 and delete/archive/hotfix ALL old packages which did not pin ffmpeg.

[ocefpaf]

TL;DR

• hotfix is still under discussion and we can add it to our next meeting agenda. I recommend you to do so and attend the meeting. (I am happy to trade places with you in case the meeting exceeds the max of 10 attendees BTW.)
• deleting is the current solution
• arching, IMO is a compromise to a missing feature in conda itself, but needs tooling
• fixing conda would be ideal

I have been waiting for months too.

We "have" a solution, it is just not good one :wink:
Our solution so far is to delete the previous binaries.

https://github.com/conda-forge/conda-forge.github.io/pull/170#issuecomment-229377350 https://github.com/conda-forge/conda-forge.github.io/pull/170#issuecomment-229653905 but also we need to come up with a policy around this soon.

Assuming a risk to make this even more boring let me say this. Ideally, a lower build number should never get selected unless explicitly specified. Meaning we should not need to delete/move them at all. Unfortunately, conda does not honor that, and we are forced to use ugly workarounds like "hotfix," deleting, or archiving.

[jakirkham]

hotfix is still under discussion and we can add it to our next meeting agenda.

I added this earlier.

[jakirkham]

At this stage, we have ways both to mark packages as broken and hotfix the metadata in the repo. So this kind of problem could be addressed if needed.

That said, it doesn't seem this is needed for these packages any more. So closing.