Charts: [stable/sentry] Not able to login to web UI

[vofish]

Hi there,

If I install stable/sentry via helm, everything is working OK, but if I install from yamls that were generated by helm:
helm template --output-dir './yamls' './sentry'
then I'm not able to login to web UI with error The connection was reset
logs from sentry-web pod:

10.8.3.1 - - [15/Jan/2020:13:39:07 +0000] "GET /_health/ HTTP/1.1" 200 200 "-" "kube-probe/1.13+"
10.8.3.1 - - [15/Jan/2020:13:39:09 +0000] "GET /_health/ HTTP/1.1" 200 200 "-" "kube-probe/1.13+"
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 112, in get_response
response = wrapped_callback(request, callback_args, callback_kwargs)
File "/usr/local/lib/python2.7/site-packages/django/views/generic/base.py", line 69, in view
return self.dispatch(request, *args, *kwargs)
File "/usr/local/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 57, in wrapped_view
return view_func(args, *kwargs)
File "/usr/local/lib/python2.7/site-packages/sentry/web/frontend/base.py", line 225, in dispatch
return self.handle(request, args, *kwargs)
File "/usr/local/lib/python2.7/site-packages/django/views/decorators/cache.py", line 52, in _wrapped_view_func
response = view_func(request, args, *kwargs)
File "/usr/local/lib/python2.7/site-packages/django/db/transaction.py", line 371, in inner
return func(args, *kwargs)
File "/usr/local/lib/python2.7/site-packages/sentry/web/frontend/auth_login.py", line 193, in handle
return super(AuthLoginView, self).handle(request, args, *kwargs)
File "/usr/local/lib/python2.7/site-packages/sentry/web/frontend/base.py", line 238, in handle
return super(BaseView, self).dispatch(request, args, *kwargs)
File "/usr/local/lib/python2.7/site-packages/django/views/generic/base.py", line 87, in dispatch
return handler(request, args, *kwargs)
File "/usr/local/lib/python2.7/site-packages/sentry/web/frontend/auth_login.py", line 211, in get
org = Organization.get_default()
File "/usr/local/lib/python2.7/site-packages/sentry/models/organization.py", line 164, in get_default
)[0]
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 132, in __getitem__
return list(qs)[0]
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 96, in __iter__
self._fetch_all()
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 857, in _fetch_all
self._result_cache = list(self.iterator())
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
for row in compiler.results_iter():
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 713, in results_iter
for rows in self.execute_sql(MULTI):
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/sentry_sdk/integrations/django/__init__.py", line 446, in execute
return real_execute(self, sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/sentry/db/postgres/decorators.py", line 80, in inner
raise_the_exception(self.db, e)
File "/usr/local/lib/python2.7/site-packages/sentry/db/postgres/decorators.py", line 78, in inner
return func(self, args, *kwargs)
File "/usr/local/lib/python2.7/site-packages/sentry/db/postgres/decorators.py", line 22, in inner
return func(self, args, *kwargs)
File "/usr/local/lib/python2.7/site-packages/sentry/db/postgres/decorators.py", line 101, in inner
six.reraise(exc_info[0], exc_info0, exc_info[2])
File "/usr/local/lib/python2.7/site-packages/sentry/db/postgres/decorators.py", line 94, in inner
return func(self, sql, args, *kwargs)
File "/usr/local/lib/python2.7/site-packages/sentry/db/postgres/base.py", line 74, in execute
return self.cursor.execute(sql, clean_bad_params(params))
ProgrammingError: ProgrammingError('relation "sentry_organization" does not exist\nLINE 1: ...default_role", "sentry_organization"."flags" FROM "sentry_or...\n ^\n',)
SQL: SELECT "sentry_organization"."id", "sentry_organization"."name", "sentry_organization"."slug", "sentry_organization"."status", "sentry_organization"."date_added", "sentry_organization"."default_role", "sentry_organization"."flags" FROM "sentry_organization" WHERE "sentry_organization"."status" = %s LIMIT 1
13:39:11 [ERROR] django.request: Internal Server Error: /auth/login/ (status_code=500 request=)
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/sentry_sdk/integrations/excepthook.py", line 66, in sentry_sdk_excepthook
hub.capture_event(event, hint=hint)
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/sentry/wsgi.py", line 41, in __call__
response = super(FileWrapperWSGIHandler, self).__call__(environ, start_response)
File "/usr/local/lib/python2.7/site-packages/sentry_sdk/integrations/django/__init__.py", line 108, in sentry_patched_wsgi_handler
return SentryWsgiMiddleware(bound_old_app)(environ, start_response)
File "/usr/local/lib/python2.7/site-packages/sentry_sdk/integrations/wsgi.py", line 120, in __call__
reraise(_capture_exception(hub))
File "/usr/local/lib/python2.7/site-packages/sentry_sdk/integrations/wsgi.py", line 116, in __call__
_sentry_start_response, start_response, span
File "/usr/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 206, in __call__
response = self.get_response(request)
File "/usr/local/lib/python2.7/site-packages/sentry_sdk/integrations/django/__init__.py", line 140, in sentry_patched_get_response
return old_get_response(self, request)
File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 194, in get_response
File "/usr/local/lib/python2.7/site-packages/sentry_sdk/hub.py", line 291, in capture_event
rv = client.capture_event(event, hint, scope)
File "/usr/local/lib/python2.7/site-packages/sentry_sdk/client.py", line 263, in capture_event
event_opt = self._prepare_event(event, hint, scope)
File "/usr/local/lib/python2.7/site-packages/sentry_sdk/client.py", line 137, in _prepare_event
event_ = scope.apply_to_event(event, hint)
File "/usr/local/lib/python2.7/site-packages/sentry_sdk/scope.py", line 61, in wrapper
return fn(self, *args, *kwargs)
File "/usr/local/lib/python2.7/site-packages/sentry_sdk/scope.py", line 322, in apply_to_event
return _drop(event, event_processor, "event processor")
File "/usr/local/lib/python2.7/site-packages/sentry_sdk/scope.py", line 277, in _drop
logger.info("%s (%s) dropped event (%s)", ty, cause, event)
File "/usr/local/lib/python2.7/logging/__init__.py", line 1174, in info
self._log(INFO, msg, args, *kwargs)
File "/usr/local/lib/python2.7/logging/__init__.py", line 1293, in _log
self.handle(record)
File "/usr/local/lib/python2.7/logging/__init__.py", line 1303, in handle
self.callHandlers(record)
File "/usr/local/lib/python2.7/site-packages/sentry_sdk/integrations/logging.py", line 78, in sentry_patched_callhandlers
return old_callhandlers(self, record)
File "/usr/local/lib/python2.7/logging/__init__.py", line 1343, in callHandlers
hdlr.handle(record)
File "/usr/local/lib/python2.7/logging/__init__.py", line 766, in handle
self.emit(record)
File "/usr/local/lib/python2.7/site-packages/sentry/logging/handlers.py", line 88, in emit
logger.log(kwargs)
File "/usr/local/lib/python2.7/site-packages/sentry/../structlog/stdlib.py", line 102, in log
return self._proxy_to_logger(_LEVEL_TO_NAME[level], event, *args, *kw)
File "/usr/local/lib/python2.7/site-packages/sentry/../structlog/stdlib.py", line 119, in _proxy_to_logger
**event_kw)
File "/usr/local/lib/python2.7/site-packages/sentry/../structlog/_base.py", line 176, in _proxy_to_logger
args, kw = self._process_event(method_name, event, event_kw)
File "/usr/local/lib/python2.7/site-packages/sentry/../structlog/_base.py", line 136, in _process_event
event_dict = proc(self._logger, method_name, event_dict)
File "/usr/local/lib/python2.7/site-packages/sentry/../structlog/stdlib.py", line 275, in __call__
event_dict['event'] = event_dict['event'] % args
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 71, in __repr__
data = list(self[:REPR_OUTPUT_SIZE + 1])
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 96, in __iter__
self._fetch_all()
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 857, in _fetch_all
self._result_cache = list(self.iterator())
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
for row in compiler.results_iter():
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 713, in results_iter
for rows in self.execute_sql(MULTI):
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/sentry_sdk/integrations/django/__init__.py", line 446, in execute
return real_execute(self, sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/util.p

Any thoughts?

[dvdliao]

Did db-init job run?

[vofish]

Actually, didn't.
There are two post-upgrade hooks that trigger two jobs: db-init and create-user.
Perhaps you know, how to generate yamls for them?

Thx.

[dvdliao]

I believe helm template will generate the hooks file for you?

[vofish]

My question was how to do that?

[alexandrujieanu]

@dvdliao

In my case the sentry-user-create job isn't created at all.
The sentry-db-init job runs to completion and the rest of the pods are up.
I can access the Sentry UI, but I can't login.

Helm still says: Failed to install app sentry. Error: timed out waiting for the condition

NAME             COMPLETIONS   DURATION   AGE
sentry-db-init   1/1           6m54s      15m

NAME                             READY   STATUS      RESTARTS   AGE
sentry-cron-5c8468fb7b-7fpbq     1/1     Running     0          16m
sentry-db-init-74bdr             0/1     Completed   0          16m
sentry-metrics-78d8fd45f-9d9mq   1/1     Running     0          16m
sentry-sentry-postgresql-0       1/1     Running     0          16m
sentry-sentry-redis-master-0     1/1     Running     0          16m
sentry-sentry-redis-slave-0      1/1     Running     0          16m
sentry-sentry-redis-slave-1      1/1     Running     0          15m
sentry-web-56f8c6b74d-wg7pg      1/1     Running     1          16m
sentry-worker-656d78f7d9-mtdcn   1/1     Running     0          16m
sentry-worker-656d78f7d9-xx8vq   1/1     Running     0          16m

[vofish]

@alexandrujieanu
To enable login you should create a user.
You can do it by running a job or connect to sentry-worker pod:
kubectl exec -it <sentry-worker-pod> bash
and run command:
sentry createuser

[alexandrujieanu]

Thanks, but shouldn't the user be created automatically?

[vofish]

It's created by job sentry-user-create.
If the job doesn't run, the user won't be created

[alexandrujieanu]

Agreed, that's why I reported above that the job isn't created and doesn't run.

[vofish]

Actually, this job starts sentry pod and run the next command within:
command: ["sentry", "createuser", "--no-input", "--email", "[email protected]", "--superuser", "--password", "pass_123"]
that's all

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

[stale[bot]]

This issue is being automatically closed due to inactivity.