Binderhub: R runtime not working/installing even if in runtime.txt

[AppsusUK]

Bug description

Setting up a fresh install of BinderHub (z2bh) using the helm chart binderhub-0.2.0-n361.h6f57706 doesn't work for R environments e.g. using the binder-examples/r doesn't install the R kernel and only allows Python.

Expected behaviour

When using repo with R runtime, JupyterHub should have R kernel and be able to run R code like mybinder.org.

Actual behaviour

If I go the the link http://<my-domain>/v2/gh/binder-examples/r/master?filepath=index.ipynb I get no option for R as seen:

How to reproduce

Follow the Zero to JupyterHub guide for Kubernetes on Google Cloud (GKE) then following Zero to BinderHub but using BinderHub helm chart version binderhub-0.2.0-n361.h6f57706 instead of the older version in the guide.

Your personal set up

Using Google Kubernetes Engine

• Version:
  binderhub-0.2.0-n361.h6f57706
• Configuration:

config:
  BinderHub:
    hub_url: <http://hub-url>
    use_registry: true
    image_prefix: <image/prefix>-
    template_path: /etc/binderhub/custom/templates
    extra_static_path: /etc/binderhub/custom/static
    extra_static_url_prefix: /extra_static/
    template_variables:
        EXTRA_STATIC_URL_PREFIX: "/extra_static/"

initContainers:
  - name: git-clone-templates
    image: alpine/git
    args:
      - clone
      - --single-branch
      - --branch=master
      - --depth=1
      - --
      - https://github.com/<repo>
      - /etc/binderhub/custom
    securityContext:
      runAsUser: 0
    volumeMounts:
      - name: custom-templates
        mountPath: /etc/binderhub/custom
extraVolumes:
  - name: custom-templates
    emptyDir: {}
extraVolumeMounts:
  - name: custom-templates
    mountPath: /etc/binderhub/custom

cors: &cors
  allowOrigin: '*'

jupyterhub:
  custom:
    cors: *cors
  cull:
    timeout: 300
    every: 60

Maybe I'm missing something during setup? Any help would be ver much appreciated. Thank you!

[welcome[bot]]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! :wave:

Welcome to the Jupyter community! :tada:

[betatim]

Thanks for reporting this. Could you post the logs from the build-... Pod that runs when the repo is built to a gist or similar? Instead of copying it from the web interface kubectl logs <name-of-build-pod> should be easier.

As an alternative checkout https://github.com/binder-examples/r-conda I think using conda to install R itself works more reliably than our old runtime.txt based method.

[AppsusUK]

Hi, thank you for the fast response! I've done what you've suggested with the build and jupyter pod logs:

Using binder-examples/r

1. kubectl logs --namespace=binder build-binder-2dexamples-2dr-974603-cac-dc
   https://gist.github.com/AppsusUK/cb88be934ca03b97fbb3f02d310a4254

2. kubectl logs --namespace=binder jupyter-binder-2dexamples-2dr-2dxuvrqwhl
   https://gist.github.com/AppsusUK/ceba4aafc068ed997dc7d974f97e5754

Using binder-examples/r-conda

• Still no R kernel in Jupyter, however if I go to the terminal I can access R

1. kubectl logs --namespace=binder build-binder-2dexamples-2dr-2dconda-ee0ee6-72e-7c
   https://gist.github.com/AppsusUK/59806e7911d1d6115c0b531b6fd1d589

2. kubectl logs --namespace=binder jupyter-binder-2dexamples-2dr-2dconda-2djgq62zt4
   https://gist.github.com/AppsusUK/f2d602cd2bca0ac0c55e2aecfe8c0e85

Using binder-examples/demo-julia

• Fails to build container

Removing intermediate container a753dc0dc3bb
The command '/bin/sh -c JULIA_PROJECT="" julia -e "using Pkg; Pkg.add(\"IJulia\"); using IJulia;installkernel(\"Julia\", \"--project=${REPO_DIR}\");" && julia --project=${REPO_DIR} -e 'using Pkg; Pkg.instantiate(); pkg"precompile"'' returned a non-zero code: 1

1. kubectl logs --namespace=binder build-binder-2dexamples-2ddemo-2djulia-28add9-39a-8d
   https://gist.github.com/AppsusUK/39cd9bbf67be71907a9d8fb883b8b6ea

[betatim]

This line makes me think that for some reason the contents of the build directory doesn't contain files that trigger the right build pack (in this case it should have said something like "R Buildpack triggered". But I have no idea why. The place I'd start digging is to look at kubectl describe build-... to see if there are mounts that hide directories. Or even try and kubectl exec -it build-.. -- /bin/bash into the build pod while it is running to see what you can see.

[AppsusUK]

I've tried both describe (results below) and /bin/bash but no errors are shown. Is there a helm chart version that you know is working I could test?

kubectl describe pods --namespace=binder build-binder-2dexamples-2dr-974603-cac-dc
https://gist.github.com/AppsusUK/84adfde16f48b30e8e6195148f570e2c

[manics]

@AppsusUK BinderHub uses repo2docker under the hood.
The version used by mybinder.org is almost always ahead of the last tagged release:
https://github.com/jupyterhub/mybinder.org-deploy/blob/1bb53c3dea862c6cf18bd0b09a77007837631297/mybinder/values.yaml#L83

In your gist logs it looks like your version of repo2docker is quite old, could you perhaps try a more recent version?
https://gist.github.com/AppsusUK/cb88be934ca03b97fbb3f02d310a4254#file-gistfile1-txt-L175

{"message": "Step 42/46 : LABEL repo2docker.version=\"0.10.0+0.g69c09ae.dirty\"", "phase": "building"}

If you want you can install repo2docker locally to build your repo.

[AppsusUK]

@manics thank you, your suggestion to use a recent version of repo2docker worked!

Here is the updated config for anyone else who faces the same issue:

config:
  BinderHub:
    hub_url: <http://hub-url>
    use_registry: true
    build_image: jupyter/repo2docker:0.11.0-184.g30ef220
    image_prefix: <image/prefix>-
    template_path: /etc/binderhub/custom/templates
    extra_static_path: /etc/binderhub/custom/static
    extra_static_url_prefix: /extra_static/
    template_variables:
        EXTRA_STATIC_URL_PREFIX: "/extra_static/"

initContainers:
  - name: git-clone-templates
    image: alpine/git
    args:
      - clone
      - --single-branch
      - --branch=master
      - --depth=1
      - --
      - https://github.com/<repo>
      - /etc/binderhub/custom
    securityContext:
      runAsUser: 0
    volumeMounts:
      - name: custom-templates
        mountPath: /etc/binderhub/custom
extraVolumes:
  - name: custom-templates
    emptyDir: {}
extraVolumeMounts:
  - name: custom-templates
    mountPath: /etc/binderhub/custom

cors: &cors
  allowOrigin: '*'

jupyterhub:
  custom:
    cors: *cors
  cull:
    timeout: 300
    every: 60