Archivebox: Steps I took to install this to kubernetes

[davralin]

I had some issues deploying this to kubernetes, so to at least try to document my steps for the next person, this is what I did:

1. Add a standard deployment, define your mounts/PVC's as you normally do.
   Add init as an argument to the container, such as:

    containers:
      - name: archivebox
        args: ["init"]
        image: archivebox/archivebox

(If you mount a block-device, using for example Rook, as your PVC, you'll have a Lost+Found-folder in /data - archivebox will not do the init-phase then, I used a sleep-container (davralin/sleep), exec'ed into it and removed lost+found)

1. After the initial run, change init to server, like so:

    containers:
      - name: archivebox
        args: ["server"]
        image: archivebox/archivebox

(dont forget to reapply with kubectl)

1. Exec into the container, and drop to archivebox's dedicated user, like so:

su -l archivebox -s /bin/bash

This allows you to manage the installation, using archivebox add or whatever.
I used archivebox manage createsuperuser to create a user, and then do the rest from the web-page.

This is the steps I did to deploy AB to my kubernetes-cluster, put the docs somewhere you think it's appropriate. :-)

Let me know if I should add the manifests I used for the deployment anywhere.

The init-part was the hardest to be honest, because the container exists immediately when started - some kind of configuration-options (using env) for doing the initial init (and ignoring lost+found) would be much appreciated for the next person.
(Could probably be done by an init-container, but I'm not there yet)

[jdcaballerov]

@davralin Have you tried using kubernetes init containers ?

[davralin]

@davralin Have you tried using kubernetes init containers ?

See my last sentence. :-)

I'm not sure if there are any consequences from doing archivebox init on every startup either, so not sure if that's the correct way.

[jdcaballerov]

@davralin Oh, sorry. archivebox init will only try to apply migrations if any and fix the index (it's safe to run multiple times, but it does use a lot of CPU/disk access unnecessarily if your archive is large).

[davralin]

There's still the issue of lost+found on ext4-volumes, which probably/perhaps should be ignored by archivebox init - regardless of k8s or not.

[pirate]

Fixed @davralin https://github.com/ArchiveBox/ArchiveBox/commit/d9ef3d0bf82854b83a94e5c044b28ecb30152e1b.

[davralin]

Nice, so with those changes in place, this is my relevant manifest:

apiVersion: v1
kind: Namespace
metadata:
  name: archivebox
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  labels:
    app: archivebox
  name: archivebox
  namespace: archivebox
spec:
  storageClassName: rook-ceph-block
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: archivebox
  namespace: archivebox
spec:
  selector:
    matchLabels:
      app: archivebox
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: archivebox
    spec:
      initContainers:
      - name: init-archivebox
        image: archivebox/archivebox
        args: ['init']
        volumeMounts:
          - mountPath: /data
            name: archivebox
      containers:
        - name: archivebox
          args: ["server"]
          image: archivebox/archivebox
          ports:
            - containerPort: 8000
              protocol: TCP
              name: http
          resources:
            requests:
              cpu: 50m
              memory: 32Mi
          volumeMounts:
            - mountPath: /data
              name: archivebox
      restartPolicy: Always
      volumes:
        - name: archivebox
          persistentVolumeClaim:
            claimName: archivebox
---
apiVersion: v1
kind: Service
metadata:
  name: archivebox
  namespace: archivebox
spec:
  ports:
    - name: http
      port: 8000
  selector:
    app: archivebox
  type: LoadBalancer

[pirate]

If you want some Github credit for that manifest you can open a PR to add it to our new Docker repo here: https://github.com/ArchiveBox/docker-archivebox ;)

Otherwise happy to add it myself in a few days.

[davralin]

Now who can pass up free Github credits... :-)