Zero-to-jupyterhub-k8s: Need way to set custom resource limits

[dholt]

Is there a way, or could there be a way, to submit custom resource limits to the notebook pod via the Helm chart? This is needed to specify a number of GPUs to attach; for example:

singleuser:
  image:
    name: dholt/jupyterhub-k8s-gpu
    tag: v0.6
  extraEnv:
    - JUPYTER_ENABLE_LAB: 1
  resources:
    limits:
      nvidia.com/gpu: 1
  nodeSelector:
    gpu: "true"

[clkao]

This can be done with c.KubeSpawner.extra_resource_limits, but there's no z2jh configmap wrapper for that. You'd need to extraConfig for now.

[yuvipanda]

@willingc I'm going to keep this particular one open, since I think we should expose this through the chart.

[willingc]

Sure thing @yuvipanda. To make sure that I understand correctly, support KubeSpawner resource limits, such as GPU, directly in helm chart.

[naeemrashid]

@clkao would you like to elaborate how to achieve this with c.KubeSpawner.extra_resource_limits

[yuvipanda]

For example, you'd set:

hub:
   extraConfig:
      10-gpu: |
          c.KubeSpawner.extra_resource_limits = { 'nvidia.com/gpu': 1 }

[choldgraf]

@yuvipanda do we have this behavior documented somewhere? Would be useful to show people this pattern until it's officially in the chart

[yuvipanda]

@choldgraf https://zero-to-jupyterhub.readthedocs.io/en/latest/advanced.html#arbitrary-extra-code-and-configuration-in-jupyterhub-config-py maybe?

[consideRatio]

I think I've initialized the resource limits to {} if it wasn't already in a kubespawner PR associated with #758 - we could encourage the use of .update

hub:
  extraConfig:
    10-gpu: |
      c.KubeSpawner.extra_resource_limits.update({ 'nvidia.com/gpu': 1 })

[consideRatio]

Fixed by manics in #809!

https://github.com/jupyterhub/zero-to-jupyterhub-k8s/blob/f5dd68cf7e6ddec4579b1c8fa87b56ddb56a59ce/images/hub/jupyterhub_config.py#L142-L143