Sentry: `sentry upgrade` fails with ProgrammingError

[nickolaskraus-wf]

Important Details

How are you running Sentry?

• [x] On-Premise docker [Version 9.0] (via stable/sentry)

Description

I am attempting to run Sentry in a Kubernetes cluster locally via Minikube.

Using the stable/sentry chart, I install the Helm Chart:

helm install -f values.yaml --name sentry --wait --timeout 600 .

The Pods come up as expected, however the sentry-web pod fails to run Sentry:

Logs

ProgrammingError: ProgrammingError('relation "sentry_option" does not exist...

Assuming this is due to the fact that sentry upgrade has not been run, I exec into the Pod and run:

sentry upgrade

I receive that same ProgrammingError for "sentry_option" | "sentry_projectkey" does not exist... and the migration fails with error code 137:

Syncing...
Creating tables ...
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)

Synced:
 > django.contrib.admin
 > django.contrib.auth
 > django.contrib.contenttypes
 > django.contrib.messages
 > django.contrib.sessions
 > django.contrib.sites
 > django.contrib.staticfiles
 > crispy_forms
 > debug_toolbar
 > raven.contrib.django.raven_compat
 > rest_framework
 > sentry.plugins.sentry_interface_types
 > sentry.plugins.sentry_mail
 > sentry.plugins.sentry_urls
 > sentry.plugins.sentry_useragents
 > sentry.plugins.sentry_webhooks
 > sudo
 > south
 > sentry_plugins.slack

Not synced (use migrations):
 - sentry
 - sentry.nodestore
 - sentry.search
 - social_auth
 - sentry.tagstore
 - sentry_plugins.jira_ac
 - sentry_plugins.hipchat_ac
(use ./manage.py migrate to migrate these)
Killed
root@sentry-web-5dd94648fc-pnttz:/# echo $?
137

Steps to Reproduce

See above.

What you expected to happen

I expected the migration to instantiated that tables that are required by Sentry.

Possible Solution

Unsure. This is most likely due to the fact that the sentry_option and sentry_projectkey database tables have not been created. I do not know how to create these tables.

[harobed]

To fix ProgrammingError: ProgrammingError('relation "sentry_option" does not exist... I need to execute:

pip uninstall sentry-plugins -y
sentry upgrade --noinput

After first initialization, I can upgrade Sentry with plugins installed:

pip install sentry-plugins -y
sentry upgrade --noinput

[harobed]

The errors message are displayed in this call: https://github.com/getsentry/sentry/blob/28d497fb8af6cc3efbe160e28c1c08f08bd688fc/src/sentry/runner/__init__.py#L125

[nickolaskraus-wf]

The ProgrammingError error I was experiencing was a symptom of a more invidious problem. The migration which is initiated when calling sentry upgrade has a memory leak, which used up the resources on my Pod resulting in it being terminated. This prevented the initial migration from ever being completed, leading to continual ProgrammingError errors.

To remedy this issue, I needed to first allocate more memory to Minikube:

minikube stop && minikube delete && minikube start --cpus 4 --memory 8192

And update the resource requests and limits of the Pod comprising the Sentry web server:

web:
  replicacount: 1
  resources:
    limits:
      cpu: 512m
      memory: 4096Mi
    requests:
      cpu: 256m
      memory: 256Mi
  ...

After these changes, sentry upgrade ran to completion. However, it did consume 50% of the more than 8GB of RAM allocated to Minikube, if your machine does not have ~4GB of memory, the database initialization process is difficult to impossible.

I am closing this issue, but the above description should help anyone who encounters the same issue.