Zero-to-jupyterhub-k8s: c.KubeSpawner.pre_spawn_hook does not correctly set variables before spawning pod

[albertmichaelj]

I am trying to set both custom volumes and custom environmental variables for a list of users before the pod is spawned, and I'm seeing weird behavior that I can only chalk up to the pre_spawn_hook running asynchronously. The behavior I'm seeing is that my code is ignored for the first pod spawned, and the second pod spawned uses the code from the first (though sometime without resetting some variables that should be reset in the pre_spawn_hook). No matter what I do, I can't get it to behave correctly. I've pasted my pre_spawn_hook function that I've added to the values.yaml file. user_yaml is a variable that is a list of users and groups that they belong to (and they should get volume per group they belong to, and I pass the groups they belong to in to the docker container as an environmental variable).

  hub:
    extraConfig:
      preSpawnHook: |
        async def my_pre_spawn_hook(spawner):
            username = spawner.user.name
            # I have to redo all of the volume mounts everytime because they don't reset after the spawn.
            c.KubeSpawner.volumes = [{
                'name': 'home',
                'persistentVolumeClaim': {
                    'claimName': pvc_claim_name
                }
            }]
            c.KubeSpawner.volume_mounts = [{
                'mountPath': get_config('singleuser.storage.homeMountPath'),
                'name': 'home',
                'subPath': get_config('singleuser.storage.static.subPath')
            }]
            c.KubeSpawner.volumes.extend(get_config('singleuser.storage.extraVolumes', []))
      c.KubeSpawner.volume_mounts.extend(get_config('singleuser.storage.extraVolumeMounts', []))
            if user_yaml[username] and user_yaml[username].get('groups'):
                groups = user_yaml[username].get('groups')
                i = 1
                for group in groups[:5]:
                    c.KubeSpawner.volumes.extend([{
                        'name': 'group' + str(i),
                        'persistentVolumeClaim': {
                            'claimName': 'nfs-persist-group' + str(i)
                        }
                    }])
                    c.KubeSpawner.volume_mounts.extend([{
                        'mountPath': "/mnt/group" + str(i),
                        'name': 'group' + str(i),
                        'subPath': "groups/" + str(group)
                    }])
                    i = i + 1
                c.KubeSpawner.environment['GROUP_FOLDERS'] = ",".join(groups[:5])
            else:
                c.KubeSpawner.environment['GROUP_FOLDERS'] = "" 
            for volume in c.KubeSpawner.volume_mounts:
                if class_name:
                    volume['subPath'] = class_name + "/" + volume['subPath']

        c.KubeSpawner.pre_spawn_hook = my_pre_spawn_hook

(NOTE: the funny looking indented line up there is an artifact of github, not my actual code) Note that I've tried both async def my_pre_spawn_hook(spawner): and def my_pre_spawn_hook(spawner): and there is no change. I print out variables as the pre_spawn_hook is running and all of the variables seem fine, but they are just not being respected in the actual spawning of the pod.

I'm tearing my hair out over this. Any help would be greatly appreciated.

[albertmichaelj]

I figured this out. It turns out that the example I was following used the c.KubeSpawner.environmnent[] pattern in setting things for the pre_spawn_hook, but instead, you should use the pattern spawner.environment[]. This only changes the spawner variables for that spawn, so you don't have to redo things every time. Also, it successfully sets things. The important insight came from coming across @consideRatio jupyterhub code here.

In case anyone finds it useful (I've had a hard time finding examples of per-user customizations), I've posted my fixed code below.

      preSpawnHook: |
        def my_pre_spawn_hook(spawner):
            username = spawner.user.name
            if user_yaml[username] and user_yaml[username].get('groups'):
                groups = user_yaml[username].get('groups')
                print(groups)
                i = 1
                for group in groups[:5]:
                    spawner.volumes.extend([{
                        'name': 'group' + str(i),
                        'persistentVolumeClaim': {
                            'claimName': 'nfs-persist-group' + str(i)
                        }
                    }])
                    spawner.volume_mounts.extend([{
                        'mountPath': "/mnt/group" + str(i),
                        'name': 'group' + str(i),
                        'subPath': "groups/" + str(group)
                    }])
                    i = i + 1
                spawner.environment['GROUP_FOLDERS'] = ",".join(groups[:5])
            for volume in spawner.volume_mounts:
                if class_name:
                    volume['subPath'] = class_name + "/" + volume['subPath']

        c.KubeSpawner.pre_spawn_hook = my_pre_spawn_hook

[consideRatio]

@albertmichaelj thanks for following up with the resolution!