Conda-forge.github.io: Feedstock Repo script failing

[jakirkham]

Running into an exception for the last 2 days in the feedstocks_repo build. This appears to be far enough into the script that none of us noticed. This can be seen in the log and down below.

Prefix: /home/travis/conda/tmp_envs/f5607ad6e1b89da01f30
Running command: ['python', '/home/travis/build/conda-forge/conda-forge.github.io/scripts/update_teams.py', './feedstocks_repo/feedstocks']
Traceback (most recent call last):
  File "/home/travis/build/conda-forge/conda-forge.github.io/scripts/update_teams.py", line 91, in <module>
    contents = env.from_string(''.join(fh)).render()
  File "/home/travis/conda/tmp_envs/f5607ad6e1b89da01f30/lib/python3.5/site-packages/jinja2/environment.py", line 989, in render
    return self.environment.handle_exception(exc_info, True)
  File "/home/travis/conda/tmp_envs/f5607ad6e1b89da01f30/lib/python3.5/site-packages/jinja2/environment.py", line 754, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/travis/conda/tmp_envs/f5607ad6e1b89da01f30/lib/python3.5/site-packages/jinja2/_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 2, in top-level template code
jinja2.exceptions.UndefinedError: 'str object' has no attribute 'get'
CalledProcessError: Command '['python', '/home/travis/build/conda-forge/conda-forge.github.io/scripts/update_teams.py', './feedstocks_repo/feedstocks']' returned non-zero exit status 1

cc @pelson

[pelson]

This will be the os rendering issue similar to https://github.com/conda-forge/conda-forge.github.io/pull/137/files. Should be an easy enough fix.

[pelson]

@jakirkham - next time this goes through, would you mind double checking that it has been fixed?

[jakirkham]

next time this goes through, would you mind double checking that it has been fixed?

Absolutely! Thanks for tracking this down, @pelson.

[ericdill]

@pelson seems to be failing on os.environ['PY_VER'] which is in a handful of recipes. Since this code path is only intended to grab the extra/maintainers section out of the meta.yaml, would it make sense to just use the machinery in conda-build to do this? i.e.,

from conda_build.meta import MetaData

meta = MetaData(recipe)
contributors = meta.meta.get('extra', {}).get('recipe-maintainers', [])

The above code snippet does not work in 1.20.2, but it does work in master for at least one of the recipes (expat) that contain os.environ'PY_VER'.

[pelson]

would it make sense to just use the machinery in conda-build to do this

Yes, to an extent it would. I believe there is a get_section method on meta which would make it even more succinct. Only downside is that we need to have defined the annoying CONDA_* variables to drivel 😞

For now, let's stick with the fix that is in place, but if it crops up again, I'm inclined to agree with you on this one. That said, we do need to be able to parse the metadata without the selectors being triggered at some points, so we do need to deal with this in a few places.

Thanks for the ideas @ericdill!

[ericdill]

Only downside is that we need to have defined the annoying CONDA_* variables to drivel

I don't follow. Where are these variables needed?

For now, let's stick with the fix that is in place

It doesn't look like it _is_ fixed since master is failing on travis: link.

[pelson]

I don't follow. Where are these variables needed?

A fair question. It used to be the case that you needed to set CONDA_PY, CONDA_NPY to even parse the metadata. Looks like that has been solved until you need the build_id:

$ cat meta.yaml 
package:
  name: foo
  version: 1

requirements:
  build:
    - python
    - numpy x.x
  run:
    - python
    - numpy x.x

>>> import conda_build.metadata; conda_build.metadata.MetaData('.').build_id()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/pelson/miniconda/lib/python2.7/site-packages/conda_build/metadata.py", line 483, in build_id
    for ms in self.ms_depends():
  File "/Users/pelson/miniconda/lib/python2.7/site-packages/conda_build/metadata.py", line 455, in ms_depends
    ms = handle_config_version(ms, ver)
  File "/Users/pelson/miniconda/lib/python2.7/site-packages/conda_build/metadata.py", line 305, in handle_config_version
    raise RuntimeError("'%s' requires external setting" % ms.spec)
RuntimeError: 'numpy x.x' requires external setting

[jakirkham]

It doesn't look like it _is_ fixed since master is failing on travis: link.

Yeah, I have to agree with @ericdill. Looks like it isn't quite fixed. 😞

[jakirkham]

Also, it appears we are hitting another issue now. Traceback shown below. This can also be seen in the log.

Prefix: /home/travis/conda/tmp_envs/f5607ad6e1b89da01f30
Running command: ['python', '/home/travis/build/conda-forge/conda-forge.github.io/scripts/update_teams.py', './feedstocks_repo/feedstocks']
Traceback (most recent call last):
  File "/home/travis/build/conda-forge/conda-forge.github.io/scripts/update_teams.py", line 91, in <module>
    contents = env.from_string(''.join(fh)).render(os=os, environ=os.environ)
  File "/home/travis/conda/tmp_envs/f5607ad6e1b89da01f30/lib/python3.5/site-packages/jinja2/environment.py", line 989, in render
    return self.environment.handle_exception(exc_info, True)
  File "/home/travis/conda/tmp_envs/f5607ad6e1b89da01f30/lib/python3.5/site-packages/jinja2/environment.py", line 754, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/travis/conda/tmp_envs/f5607ad6e1b89da01f30/lib/python3.5/site-packages/jinja2/_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 27, in top-level template code
jinja2.exceptions.UndefinedError: 'os._Environ object' has no attribute 'PY_VER'
CalledProcessError: Command '['python', '/home/travis/build/conda-forge/conda-forge.github.io/scripts/update_teams.py', './feedstocks_repo/feedstocks']' returned non-zero exit status 

[ericdill]

I am 90% sure that using the MetaData object will fix that too

[jakirkham]

Just as FYI, this more recent problem is actually stopping people from being added to feedstocks. So, the severity of the problem has gone up. If you want to try a PR @ericdill, I'd be more than happy to work with you on it.

[ericdill]

Ok, give me a few minutes. PR incoming

[ericdill]

@jakirkham Lets continue this discussion on #142.

[ericdill]

Can someone confirm that travis jobs are actually _running_ for the conda-forge organization? They all seem to be stalled right now

[jakirkham]

There are builds running. It is just that Travis CI gets really bogged down in general at this time of the day. Let's wait and see how things go.

[ericdill]

Probably want to reopen this until we know it's fixed. My bad for including the github auto close trigger in my PR

[jakirkham]

No worries.

[ericdill]

Pretty sure the problem now is that we need to install conda build master, as I definitely mentioned in this comment and then promptly forgot about https://github.com/conda-forge/conda-forge.github.io/issues/139#issuecomment-219685360

[jakirkham]

Agreed. We should be able to pin it no problem though.

[ericdill]

Oh does it work in 1.20.1 too? I know it works in master

[jakirkham]

Yep. We don't know why, but 1.20.1 does work.

[jakirkham]

So, looks we are now hitting something different. Traceback below. Also, here is the relevant log line. Someone's recipe doesn't have a proper build field. We might be able to track this down by looking for a feedstock failing on every platform it builds on.

Prefix: /home/travis/conda/tmp_envs/4a053950f5cbeca16023

Running command: ['python', '/home/travis/build/conda-forge/conda-forge.github.io/scripts/update_teams.py', './feedstocks_repo/feedstocks']

Traceback (most recent call last):

  File "/home/travis/build/conda-forge/conda-forge.github.io/scripts/update_teams.py", line 78, in <module>

    meta = MetaData(os.path.dirname(recipe))

  File "/home/travis/conda/tmp_envs/4a053950f5cbeca16023/lib/python3.5/site-packages/conda_build/metadata.py", line 334, in __init__

    self.parse_again(permit_undefined_jinja=True)

  File "/home/travis/conda/tmp_envs/4a053950f5cbeca16023/lib/python3.5/site-packages/conda_build/metadata.py", line 345, in parse_again

    self.meta = parse(self._get_contents(permit_undefined_jinja))

  File "/home/travis/conda/tmp_envs/4a053950f5cbeca16023/lib/python3.5/site-packages/conda_build/metadata.py", line 153, in parse

    (field, res[field].__class__.__name__))

RuntimeError: The build field should be a dict, not str

CalledProcessError: Command '['python', '/home/travis/build/conda-forge/conda-forge.github.io/scripts/update_teams.py', './feedstocks_repo/feedstocks']' returned non-zero exit status 1

[ericdill]

Added #146 to print out each recipe's name as it is being processed so we can track this one down

[jakirkham]

So, it goes alphabetically AFAICT and hits auditwheel and then crashes. So, either auditwheel or the next alphabetically (autoconf) is causing the problem somehow. Looking at their build fields doesn't reveal anything obviously wrong. Plus these feedstocks haven't had their recipes touched in a long time. We must be missing something here. 😕

[ericdill]

Ohhhhh I think I figured out the problem. auditwheel has the following build section:

build:
  skip: True  # [not linux]
  skip: True  # [py2k]

Don't both of these skip sections result in the build section being totally empty? I bet that if we added number: 0 to the build section that this recipe would parse just fine

[jakirkham]

I was looking at that section, but hadn't thought of the build number. Let me add that to my PR.

[jakirkham]

BTW, here's the PR ( https://github.com/conda-forge/auditwheel-feedstock/pull/3 ).

[jakirkham]

I restarted Travis after the auditwheel PR was merged. Let's see if that gets things moving.

[jakirkham]

Well, it did pass auditwheel. So, that may very well have been the problem. Now it is libunistring.

[jakirkham]

Added a PR ( https://github.com/conda-forge/libunistring-feedstock/pull/3 ) to hopefully fix libunistring.

[jakirkham]

Alright, restarting Travis after merging the libunistring PR. Let's see how far we get this time.

[jakirkham]

Now it is mingwpy. This one is a bit trickier. Here is the error, which can be seen in this line in the log.

RuntimeError: The source field should be a dict, not str

[jakirkham]

So, I have an idea. I think this PR ( https://github.com/conda-forge/mingwpy-feedstock/pull/5 ) might help. Basically, we just hide the source section completely if it is not Windows.

[jakirkham]

Alright, I'm going to call it. Will check back later.

[jakirkham]

Though I should add there is a clear trend here and I'm wondering if there is a conda-build bug behind it. That being said, many of these are definitely recipes deserving of a cleanup. Though mingwpy doesn't quite feel like one of deserving of cleanup in this way, but I do see it fitting the trend.

[jakirkham]

Restarted Travis while I'm here.

[jakirkham]

Good time to quit too. We are now rate limited. 😆 Honestly, I can't believe that didn't happen faster.

[pelson]

Honestly, I can't believe that didn't happen faster.

Gets reset on the hour. We are out for 50 mins 😄 . Enough time for me to do something about it maybe 😉

[jakirkham]

Gets reset on the hour. We are out for 50 mins 😄 .

Ouch! That long?!

Enough time for me to do something about it maybe 😉

...or maybe do something else interesting that you want to complete. I think we have a good sense of what is going on here now. Given what little time you have, maybe it can be better spent.

[jakirkham]

Turns out that was the last package anyways. It now seems to be running through smoothly. Going to close this out. Thanks everyone for helping us get to the bottom of this.

[ericdill]

Hey nice work on this @jakirkham !

[jakirkham]

Nice work on this yourself, @ericdill. The biggest piece was your code. 😉