I'm already feeling overwhelmed with keeping track of recipes that either need Windows modification or need attention at some later date (e.g. to include some Continuum changes that I know are available)
Is there any way we can consolidate these recipes/repos into a list that is easier to approach and to track? Perhaps a wiki page? Some Google spreadsheet? Create issues on those repos and assign people to them?
I'll try to come up with a list here. Sorry you are feeling overwhelmed. I'll try to restrict further questions and/or direct others to this issue so it can remain manageable.
No problem - I'm overwhelmed with normal Continuum work anyway. It's life as normal. I just don't want to lose track of things here that I could help with - and I feel bad that I may have already missed a few instances where you all have pinged me.
No problem - I'm overwhelmed with normal Continuum work anyway.
My hope is that conda-forge will in the long run make this less so. I expect everyone will enjoy a few less pings about updates to packages or at least can respond "please submit a PR". :smile:
Admittedly, that is not the case while we are still try to figure out how it should work.
I just don't want to lose track of things here that I could help with...
Absolutely, and I should be doing a better job at compiling these somewhere so we can track them.
...and I feel bad that I may have already missed a few instances where you all have pinged me.
No need to feel bad. There is just a lot going on here.
The todo list feature works really well in gh issues - that is certainly one option @jakirkham.
I use the (free) zenhub chrome plugin. It provides a few additional nice
views as well. I'm cool with any solution aside from @msarahan mentions -
those just aren't nicely searchable/collectable, and I miss them
sometimes. Labels on PRs, or assignment of PRs, or merged PRs without
Windows stuff, plus an issue with a label or an assignment.
I think there are a few labels we would need:
Feel free to add more - I just think these are ones that would help me help
you the most.
On Sun, Mar 20, 2016 at 4:30 AM Phil Elson [email protected] wrote:
The todo list feature works really well in gh issues - that is certainly
one option @jakirkham https://github.com/jakirkham.โ
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/conda-forge/staged-recipes/issues/161#issuecomment-198881674
- Request Continuum recipe update
- Needs Windows fix
- Needs build from source
:+1: to all that. Not sure if we can define label to an organization though. I guess they are defined at project level, right?
I use the (free) zenhub chrome plugin.
Never used, but I am OK for testing something new.
Never used, but I am OK for testing something new.
You don't need to. It's just a way of viewing the same information that Github already has in different ways. It provides todo-lists also, but AFAICT, those are private. Still - give it a shot. I find it improves my ability to track stuff on Github.
I guess they are defined at project level, right?
Yeah... bummer. Seems we're not the first people to think of this:
https://github.com/repo-utils/org-labels
also
Yeah... bummer. Seems we're not the first people to think of this
It is one of those things you never thought you'd use and one day you cannot live without it :grinning:
I was looking at different ways to help us track what is going on as I agree it is kind of busy here and it is hard to tell sometimes (especially with so many repos :wink:).
These appear to be free for open source. This seems like it might help ( https://sprint.ly/ ) the management side. This ( https://zube.io/ ) is pretty similar. Also, this ( https://waffle.io/ ) is ok for single repos like this one. Here is another neat one ( https://codetree.com/ ) that looks potentially good for issue tracking. Another similar one ( http://gitlo.co/ ).
These are interesting, but need to be paid for somehow. This ( https://huboard.com/ ) could be tried, but I can't seem to find a good example of it in use. This ( https://www.blossom.co/ ) is similar to the first two.
BTW, I tried enabling ZenHub for the org. Not sure how this ends up working as I have only begun using it. Will this let us use the same ZenHub or does every user setup their own? Anyways feel free to share feedback. As it is free for open source, we can use this for as long as we like.
I don't see anything set up on ZenHub. I'm not sure if it is just sort of a local tool, storing its information in the plugin, or if it has some sharable content that other people can see. Please do not feel compelled to use it on my behalf. I'm happy to try any other tool - I just want something (anything) that can help unify labels. I feel like if we get the labels stuff done right, then anyone will be able to use any tool to track that metadata appropriately.
I see stuff when I go to the "boards" tab. The information should be synchronized to zenhub's servers--I've used it for collaboration elsewhere.
Yeah, I see stuff on the boards, but only for the two repos I linked myself. Wasn't sure if anyone had created/linked any other repos. Does each new feedstock need to be manually added?
Does each new feedstock need to be manually added?
As a helpful tip, more than a thing you might actually want to do for this:
If you can register with an API it can be batched up with the conda-smithy feedstocks cli (note, I've just found and fixed a bug, so you need a 0.4.* version:
feedstocks apply-cloned echo "foo {} {FEEDSTOCK_NAME}"
You would want to clone all of the feedstocks first though:
feedstocks clone
Bit of a tangent, but could we get a copy of the snappy recipe in here? It's good that we do have a package of snappy available to us, but it would be nice to have the recipe moved over. Alternatively, I can just write a recipe and tweaks can be backported as there is time. However, you would like to do it.
@groutr - can you move the snappy recipe any time soon?
sure. I can do it in the next day or two.
I think we need the Windows build incantation for snappy.
Also, we could probably use those Windows build incantations for OpenCV.
Given that this Windows black magic seems to really be necessary for all multi-Python version projects - would it not make sense for that information to just be provided as an ENV variable? Or maybe a small script that is on the conda path that can be called?
Or maybe a small script that is on the conda path that can be called?
This would be ideal.
So, this is very special. It appears the hdf5 package is in conflict with itself on Windows. Bonus points for anyone with a working Windows dev environment who can figure out this brainteaser ( https://github.com/conda-forge/hdf5-feedstock/issues/1 ). Interestingly this was building before.
If someone has time and a Windows environment up and working, I would appreciate guidance on getting h5py working for Windows in this PR ( https://github.com/conda-forge/h5py-feedstock/pull/1 ) or if you want to submit your own PR feel free.
Could you please make sure this PR ( https://github.com/conda-forge/staged-recipes/pull/194 ) has the right Windows magic for compiler selection, @msarahan?
So, I have been working on protobuf support for C++ ( https://github.com/conda-forge/protobuf-feedstock/pull/4 ). I have what looks so far to be working Windows support. Though I would appreciate feedback from others on anything else this might need.
Could anyone with access please provide the YAML recipe? Appears that we can build it with C++ support, but that is not currently happening. Doesn't even have to be a pretty recipe. We can clean it up here.
Will do tomorrow.
On Mon, Apr 18, 2016, 21:55 jakirkham [email protected] wrote:
Could anyone with access please provide the YAML recipe? Appears that we
can build it with C++ support, but that is not currently happening. Doesn't
even have to be a pretty recipe. We can clean it up here.โ
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/conda-forge/staged-recipes/issues/161#issuecomment-211703915
Actually I started hacking on one just to see how far I could get it to go. Maybe just your feedback should be enough.
Though I was wrong about C++ support. That will require another yaml library. Will have to check and see that they don't clash with the C version.
See this PR ( https://github.com/conda-forge/staged-recipes/pull/397 ) for the yaml recipe that I hacked together. Windows side is at best a guess of what should work. Will see how it goes. If there is anything else I missed or is nice to add, please let me know.
@msarahan, could you please take a look at my yaml recipe in PR ( https://github.com/conda-forge/staged-recipes/pull/397 ) when you have a chance?
I know this working on Mac and Linux. For Windows, I have tried something that appears to be working, but I don't know if I am generating the libraries correctly. It appears they only build a LIB. So, I made a patch against their CMakeLists.txt to build a DLL. It is building this, but I don't know if there is more I need to be doing there. Any pointers would be appreciated.
@msarahan, we have learned that the gcc compiler package in anaconda.org does not contain this range restriction ( https://github.com/conda/conda-recipes/pull/501 ). Could we please get a rebuild to fix this?
I'm now feeling your pain, @msarahan. I woke up with upwards of 70 notifications most from here. My situation as far as searching pings is definitely better than others, but I can't mark my notifications as todos. I have added a "Mac" label. If there is a Mac question you can still ping me if you want, but definitely add the "Mac" label.
Sorry you're feeling my pain, but thanks for the offer of help on mac. I'll certainly take you up on that.
...thanks for the offer of help on mac. I'll certainly take you up on that.
What have I done!? ๐ฑ
Just kidding. Yeah, I'm more than happy to help. Basically, it has been assumed I will at this point, which is totally fine. Just would be nice to know a bit better where that help is needed.
Could you take a look at this PR ( https://github.com/conda-forge/staged-recipes/pull/563 ), @msarahan? This should give us our missing certificates.
Could you please take a look at this PR ( https://github.com/conda-forge/mpfr-feedstock/pull/3 ), @msarahan? We are kind of at a standstill in terms of where stuff lives. It would be good if someone with more Windows and conda expertise combined could weigh in.
Thanks for taking a look at that, @msarahan. ๐
We're seeing downloading issues on CircleCI. I don't know if it is related to conda-build, but it is a bit strange that we start seeing them after a recent release. Here is an example, but there are more.
This PR ( https://github.com/conda-forge/jpeg-feedstock/pull/3 ) installs a development copy of conda-build. This was done to get the jinja/selector fix, which seems to work fine on *NIXes. However, on Windows, we are encountering an error in conda-build develop on all Python versions and architectures. Though I have not tested conda-build on Windows Python 3.4 64-bit because of the current pinning, I expect it will be the same. The traceback I am seeing is below. It appears to be the same in all cases. Do you have any idea what is going on here, @msarahan?
Executing: conda build recipe --quiet
Traceback (most recent call last):
File "C:\conda\lib\argparse.py", line 1758, in parse_known_args
namespace, args = self._parse_known_args(args, namespace)
File "C:\conda\lib\argparse.py", line 1946, in _parse_known_args
positionals_end_index = consume_positionals(start_index)
File "C:\conda\lib\argparse.py", line 1923, in consume_positionals
take_action(action, args)
File "C:\conda\lib\argparse.py", line 1816, in take_action
argument_values = self._get_values(action, argument_strings)
File "C:\conda\lib\argparse.py", line 2267, in _get_values
self._check_value(action, value[0])
File "C:\conda\lib\argparse.py", line 2310, in _check_value
raise ArgumentError(action, msg % args)
argparse.ArgumentError: argument command: invalid choice: 'build' (choose from 'info', 'help', 'list', 'search', 'create', 'install', 'update', 'upgrade', 'remove', 'uninstall', 'run', 'config', 'init', 'clean', 'package', 'bundle')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\conda\scripts\conda-script.py", line 5, in <module>
sys.exit(main())
File "C:\conda\lib\site-packages\conda\cli\main.py", line 127, in main
args = p.parse_args()
File "C:\conda\lib\site-packages\conda\cli\conda_argparse.py", line 166, in parse_args
return super(ArgumentParser, self).parse_args(*args, **kwargs)
File "C:\conda\lib\argparse.py", line 1726, in parse_args
args, argv = self.parse_known_args(args, namespace)
File "C:\conda\lib\argparse.py", line 1765, in parse_known_args
self.error(str(err))
File "C:\conda\lib\site-packages\conda\cli\conda_argparse.py", line 146, in error
p = subprocess.Popen(args)
File "C:\conda\lib\subprocess.py", line 950, in __init__
restore_signals, start_new_session)
File "C:\conda\lib\subprocess.py", line 1220, in _execute_child
startupinfo)
OSError: [WinError 193] %1 is not a valid Win32 application
Command exited with code 1
Looks like none of conda build's CLI/entry point commands are getting installed at all. I don't know why that would happen, but I'll look into it.
Thanks for testing things.
wait, before I go too far, how are you installing conda-build? Remember how I said you have to both conda-install it, _then_ install the development version? the python install does not do the entry points correctly.
the python install does not do the entry points correctly.
Ah! Good point. I keep forgetting about this. Though I can't understand why it worked on the *NIXes. Yeah, I will updated it.
Wait. Don't they get installed. I'm confused. I knew this was needed for bdist_conda, but I didn't think we needed this for the entrypoints.
The scripts are copied, but those are not entry points. They are not executable on Windows.
Ah, ok. I didn't realize Python on Windows did something special here too.
Nearly have a working copy of tk in this PR ( https://github.com/conda-forge/staged-recipes/pull/634 ). Works fine on *NIXes. Builds fine on Windows. However, the tests are failing. If you could take a look at it @msarahan when you have a chance, that would be awesome. I think this is the last dependency of Python that we don't have.
Has anyone ever seen this issue when trying to build on Windows with Cython or other Python package with C extensions. Traceback below. Also, log for more details.
File "C:\conda\envs\_build\lib\distutils\msvc9compiler.py", line 753, in library_dir_option
return "/LIBPATH:" + dir
TypeError: cannot concatenate 'str' and 'NoneType' objects
@jakirkham That is because of a bug in the recipe for that project, here as the LIBDIR config variable doesn't exist on Windows. Even removing that still has issues with finding boost. I think you probably want to use the CMAKE build version from their Github?
Thanks for the feedback, @patricksnape. Will take a closer look at how to get the configuration right.
So, I noticed there was an issue in our build of curl where it got the cert path messed up. I have added a test and fix that seem to work, but would appreciate if some people ( including @msarahan ๐ ) took a look and made sure this seems ok. Here is the PR ( https://github.com/conda-forge/curl-feedstock/pull/3 ).
Once we have curl fixed, our git has basically the same issues with certs. A few of us ( @msarahan, @pelson and myself ) had been looking into fixing this. I think I have put the finishing touches on it and it seems to be doing ok. Though I would appreciate if some people ( including @msarahan ๐, ears burning yet ๐ ) take a closer look at it to make sure it is doing ok. Here is the PR ( https://github.com/conda-forge/git-feedstock/pull/10 ).
Thanks in advance.
Thanks for helping with curl and git, @msarahan.
Trying to package conda as there seemed to be some interest in doing so after our last meeting. However, I was encountering issues when building locally. Maybe some can take a look. Here is the PR ( https://github.com/conda-forge/staged-recipes/pull/656 ).
Could you please take a look at PR ( https://github.com/conda-forge/pillow-feedstock/pull/6 ), @msarahan? We need to bump this pin, but we are having some issues doing so on Windows only. Do you have any ideas? Is this already fixed in a later conda-build?
So, I think we already know the problem here, but there needs to be some action. Basically vs2015_runtime: 14.00.23026.0-1 defaults is breaking Windows Python 3.5 builds. See this issue for some details. ( https://github.com/conda-forge/staged-recipes/issues/666 )
I'm struggling to get a build of HDF5 1.8.15 or 1.8.15.1 working on Windows. They seem to miss all of the CLI tools. Also, they hang for an 1hr when building with VC14. If anyone has any thoughts, please let me know.
We are trying to fix MPIR to have Windows files in the right location in this PR ( https://github.com/conda-forge/mpir-feedstock/pull/3 ). It would be nice if someone can double check that we are doing this right.
@msarahan, could you please double check the aforementioned MPIR PR. It looks ok to me, but I want to double check to ensure we get it right this time.
@msarahan, could you please take a look at two PRs for me? The first one I'm worried we may be hitting a conda-build bug. The second one really I need people with Windows expertise to take a look and make sure it seems ok to them (if you have time @patricksnape your input is always appreciated ๐).
@msarahan, we seem to be between a rock and a hard place. If we use conda-build below 1.21.0, we end up packaging activate, deactivate, and conda links ( https://github.com/conda-forge/hdf5-feedstock/issues/23 ). If we use conda-build 1.21.0, we get our environment variables chopped up ( https://github.com/conda-forge/staged-recipes/pull/751 ). I'm going to try to come up with a workaround, but it would be nice to break ground on one of these. I'm guessing backporting the link handling from 1.21.0 to 1.20.x and doing a release will be the easiest to do.
I'm pretty close to a fix for chopping up variables. More soon.
Thanks @msarahan. I really appreciate it. ๐
@jakirkham - if you want to test master, that would be very helpful.
@jakirkham any feedstocks that show the chopped-up mess of 1.21.0? I think I have 1.21.1 ready, but need to make sure.
PS, I tried to follow #751 back to a recipe, but failed. Otherwise, I'd just use that, if anyone can point me to the actual repo.
So, these are at staged-recipes, but you can try this PR ( https://github.com/conda-forge/staged-recipes/pull/762 ) or ( https://github.com/conda-forge/staged-recipes/pull/751 ). I don't know off hand of a repo having these issues. I think these are the best we can do. Would it be possible to open a "NO MRG" PR on top of those to install a development copy of conda-build or does that not work?
I copied the scripts from #751 manually (copy/paste), locally, and things work fine. I have tagged 1.21.1. Packages should be available soon, but I'll post here when they are.
Did you use the docker container?
Thanks @msarahan. ๐
No, just built the code for Mac. Sorry, tried docker, but I'm using the beta, and volume mounts don't seem to be working. =(
I hope 1.21.1 is a solid release for us.
Appveyor is showing some issues with Windows on the 3.4 stuff (as usual). You should probably expect some failures there. I'm working on it. I hope that the rest of the release will still be useful.
Would you mind taking a look at this PR ( https://github.com/conda-forge/openblas-feedstock/pull/6 ) when you have a chance, @msarahan? I think this is the fastest path forward ATM without winding up in trouble. It appears the libgfortran that is shipped from defaults has a newer ABI than the one pulled from the gcc compiler. So, I have chosen to use that directly on Linux and leave OS X basically unchanged as I have not heard any complaints about the latter.
Friendly nudge for reviewing this PR ( https://github.com/conda-forge/mpc-feedstock/pull/3 ) when you have a chance, @msarahan.
cc @isuruf
Did the symlinks get removed from conda, @msarahan?
cc @gillins
There's not enough context here. Do you mean the activate scripts? They have not been removed AFAIK. Last I dealt with them, I wrote a test to ensure that they exist.
Yeah, the symlinks to activate, conda, and deactivate.
Interesting, well, we mysteriously are not seeing them get packaged even in cases where we have not released the conda-build pinning. For instance, see libtiff version 4.0.6 build number 5 and gdal version 2.1.0 build number 6. Not sure why it is and are trying to figure it out. Even though this is a win, it is a bit disconcerting as we don't understand the reason.
Could you please take a look at the conda recipe build for Windows ( https://github.com/conda-forge/staged-recipes/pull/656 ), @msarahan? Seem to be having some issues with always_include_files.
Could you please double check that the toolchain is now correct w.r.t. to Windows in this PR ( https://github.com/conda-forge/toolchain-feedstock/pull/12 ), @msarahan? It wasn't actually doing anything on Windows previously, but it would be good to make sure it is setup correctly if starts to be needed there.
As you have time @msarahan this PR ( https://github.com/conda-forge/staged-recipes/pull/658 ) needs some TLC probably in the form of a conda-build patch, but it is a bit subtle. Again not a rush or anything just something I think you will have some good insight on.
Got a very strange error when trying to upgrade conda-smithy. Don't think it is related to the changes we made. Please take a look when you have a chance @msarahan. Here is the PR ( https://github.com/conda-forge/conda-smithy-feedstock/pull/29 ).
Could you please review PR ( https://github.com/conda-forge/cmake-feedstock/pull/8 ) when you have a chance, @msarahan?
While you are looking at weird activation issues on Windows, maybe this one will catch your attention, @msarahan?
@msarahan @patricksnape, could either or both of you please take a look at a couple PRs @jjhelmus has added to python. They seem crucial for Windows, but I don't really feel qualified to comment on them. If these are fixing major issues, I'd like to see them get out there soon, but also want to be sure they are working properly. Please see PR ( https://github.com/conda-forge/python-feedstock/pull/52 ) for 3.4 and PR ( https://github.com/conda-forge/python-feedstock/pull/51 ) for 3.5. They are very similar in nature FWICT.
This can probably be done at your own pace (not sure @jjhelmus' timeline). Though it would be nice if some people could take a look at the proposed Python 2.7 recipe ( https://github.com/conda-forge/python-feedstock/pull/62 ) and see if it makes sense or needs changes.
Could you please take a look at this recipe for nsis ( https://github.com/conda-forge/staged-recipes/pull/1089 ), @msarahan? I'm trying to get it to build from source with scons, but seem to be having some issues. We can install the binary from SourceForge, but the fact that it is a binary from SourceForge is making me a little queasy ๐. Especially since it is being used to build installers. Getting it to build from source would make me feel way more comfortable. Any help would be appreciated.
Could you please take a look at issue ( https://github.com/conda-forge/conda-feedstock/issues/1 ) when you have a chance, @msarahan? Seems like the activation scripts are getting messed up in the conda package. Have gone ahead and pulled these until we solve the issue, but the packages can still be inspected if you wish to look at them.
Could you please provide some thoughts on PR ( https://github.com/conda-forge/staged-recipes/pull/1205 ), @msarahan? The OP is working on a tool that allows one to call commands from a conda environment without having to explicitly activate first by having a wrapper that does this for you. There are variants for other things like virtualenv. It would be really neat to figure out how to get this to work best with conda.
Could you please take a look at issue ( https://github.com/conda-forge/openssl-feedstock/issues/3 ), @msarahan? We are totally mystified. It seems that there is some openssl package that has a post-link step going wrong. Though we are pretty sure it isn't ours (despite what the log says) as we don't have a post-link step. Our best guess is it is coming from defaults. Though I'm also having a bit of trouble tracking down with openssl package it is.
Could you please take a look at PR ( https://github.com/conda-forge/staged-recipes/pull/1144 ), @msarahan? We think this is an ok strategy, but would like feedback.
Would appreciate your thoughts on PR ( https://github.com/conda-forge/staged-recipes/pull/1491 ), @msarahan. Especially as there appears to be a defaults package for it yet there appears not to be a recipe in the open.
This error is pretty weird. It appears FWICT that conda-build messed up at packaging Python 2.7 and 3.5 builds for Windows 32-bit (they only have the info directory and not Lib). Would love to understand why that went wrong.
cc @msarahan @pmlandwehr
Could you please take a look at PR ( https://github.com/conda-forge/staged-recipes/pull/1491 ) for libevent, @msarahan? I think this is different than defaults in some ways (though can't check as there is no open source recipe for it AFAICT).
Could you please take a look at PR ( https://github.com/conda-forge/staged-recipes/pull/1678 ), @msarahan and @pelson ? This a continuation of PR ( https://github.com/conda-forge/staged-recipes/pull/617 ), which adds support for VS 2008 64-bit to AppVeyor. We think it is ready to go, but it would be nice if the two of you could give it a quick look through.
Think the story for directing and notifying about issues has gotten a bit better over the years. Partially by increasing reviewer bandwidth. Partially by adding reviewer teams. Partially by people reaching out on Gitter about things that are blocked periodically. Going to close this out. Feel free to reopen if there is something here that still needs to be addressed.
Most helpful comment
I'm pretty close to a fix for chopping up variables. More soon.