Sentry: TypeError: 'NoneType' object has no attribute '__getitem__' while trying to create new project immediately after installation

[ye]

Getting the exception in the docker log.

Sentry 8.3.1 with MySQL 5.5 setup using official Sentry Docker hub recipe and official MySQL docker hub recipe

10.10.10.1 - - [08/Apr/2016:19:28:44 +0000] "GET /organizations/sentry/projects/new/ HTTP/1.1" 200 11371 "http://10.10.10.10:8080/sentry/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36"
[ERROR] 'NoneType' object has no attribute '__getitem__'
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/sentry/utils/logging.py", line 23, in wrapped
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/sentry/event_manager.py", line 493, in save
    **group_kwargs
  File "/usr/local/lib/python2.7/site-packages/sentry/event_manager.py", line 648, in _save_aggregate
    short_id = project.next_short_id()
  File "/usr/local/lib/python2.7/site-packages/sentry/models/project.py", line 109, in next_short_id
    return Counter.increment(self)
  File "/usr/local/lib/python2.7/site-packages/sentry/models/counter.py", line 39, in increment
    return increment_project_counter(project, delta)
  File "/usr/local/lib/python2.7/site-packages/sentry/models/counter.py", line 83, in increment_project_counter
    return cur.fetchone()[0]
TypeError: 'NoneType' object has no attribute '__getitem__'

[mitsuhiko]

This was reported before but it only happened with outdated mysql drivers. Can you share which ones you are using?

[ye]

Sure. I use official Sentry Docker hub recipe https://hub.docker.com/_/sentry/ but linked with https://hub.docker.com/_/mysql/ version 5.5.48.

MySQL-python==1.2.5

Btw, Armin I am a big fan!

[mitsuhiko]

I will look into that but my guess is that this is a bug in mysql python. We have seen another report but it's not clear how to work around it other than changing the client.

[ye]

Ok, great! At the mean time, I will give PyMySQL a shot!

[mitsuhiko]

@ye try using mysqlclient.

[mattrobenolt]

+1 for mysqlclient. It's what's recommended by Django itself. It's a fork of MySQL-python with a bunch of improvements and bug fixes.

[mattrobenolt]

See also https://github.com/getsentry/sentry/pull/3018 https://github.com/getsentry/sentry/issues/3027

[ye]

I've just tried mysqlclient==1.3.7. Still no go. Got similar exception while clicking the New Project button. Web UI doesn't prompt anything though so it seems normal from the user's perspective.

10.10.10.1 - - [08/Apr/2016:21:34:38 +0000] "GET /organizations/sentry/projects/new/ HTTP/1.1" 200 11338 "http://10.10.10.10:8080/sentry/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36"
[ERROR] 'NoneType' object has no attribute '__getitem__'
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/sentry/utils/logging.py", line 23, in wrapped
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/sentry/event_manager.py", line 493, in save
    **group_kwargs
  File "/usr/local/lib/python2.7/site-packages/sentry/event_manager.py", line 648, in _save_aggregate
    short_id = project.next_short_id()
  File "/usr/local/lib/python2.7/site-packages/sentry/models/project.py", line 109, in next_short_id
    return Counter.increment(self)
  File "/usr/local/lib/python2.7/site-packages/sentry/models/counter.py", line 39, in increment
    return increment_project_counter(project, delta)
  File "/usr/local/lib/python2.7/site-packages/sentry/models/counter.py", line 83, in increment_project_counter
    return cur.fetchone()[0]
TypeError: 'NoneType' object has no attribute '__getitem__'

[ye]

Btw, the celery worker container has an infinite amount of similar exceptions plaguing the logs.

[2016-04-08 21:33:40,365: ERROR/Worker-1] 'NoneType' object has no attribute '__getitem__'
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/sentry/utils/logging.py", line 23, in wrapped
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/sentry/event_manager.py", line 493, in save
    **group_kwargs
  File "/usr/local/lib/python2.7/site-packages/sentry/event_manager.py", line 648, in _save_aggregate
    short_id = project.next_short_id()
  File "/usr/local/lib/python2.7/site-packages/sentry/models/project.py", line 109, in next_short_id
    return Counter.increment(self)
  File "/usr/local/lib/python2.7/site-packages/sentry/models/counter.py", line 39, in increment
    return increment_project_counter(project, delta)
  File "/usr/local/lib/python2.7/site-packages/sentry/models/counter.py", line 83, in increment_project_counter
    return cur.fetchone()[0]
TypeError: 'NoneType' object has no attribute '__getitem__'
[2016-04-08 21:33:40,421: ERROR/Worker-2] 'NoneType' object has no attribute '__getitem__'
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/sentry/utils/logging.py", line 23, in wrapped
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/sentry/event_manager.py", line 493, in save
    **group_kwargs
  File "/usr/local/lib/python2.7/site-packages/sentry/event_manager.py", line 648, in _save_aggregate
    short_id = project.next_short_id()
  File "/usr/local/lib/python2.7/site-packages/sentry/models/project.py", line 109, in next_short_id
    return Counter.increment(self)
  File "/usr/local/lib/python2.7/site-packages/sentry/models/counter.py", line 39, in increment
    return increment_project_counter(project, delta)
  File "/usr/local/lib/python2.7/site-packages/sentry/models/counter.py", line 83, in increment_project_counter
    return cur.fetchone()[0]
TypeError: 'NoneType' object has no attribute '__getitem__'

I think this explains why I can't see the events in Sentry Web UI even though I've ping sentry via Raven-Lua and via HTTPie a couple of times.

$ http -v -j  POST http://10.10.10.10:8080/api/2/store/ X-Sentry-Auth:"Sentry sentry_version=7,sentry_client=raven-lua/0.40,sentry_timestamp=2016-04-07T16:58:02Z,sentry_key=public-key,sentry_secret=secret-key" < test.json
POST /api/2/store/ HTTP/1.1
Accept: application/json
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 315
Content-Type: application/json
Host: 10.10.10.10:8080
User-Agent: HTTPie/0.9.3
X-Sentry-Auth: Sentry sentry_version=7,sentry_client=raven-lua/0.40,sentry_timestamp=2016-04-07T16:58:02Z,sentry_key=public-key,sentry_secret=secret-key

{
    "culprit": "my.module.function_name", 
    "event_id": "fc6d8c0c43fc4630ad850ee518f1b9d0", 
    "exception": [
        {
            "module": "__builtins__", 
            "type": "SyntaxError", 
            "value": "Wattttt!"
        }
    ], 
    "message": "SyntaxError: Wattttt!", 
    "tags": {
        "ios_version": "4.0"
    }, 
    "timestamp": "2011-05-02T17:41:36"
}

HTTP/1.1 200 OK
Cache-Control: max-age=0
Content-Language: en
Content-Length: 41
Content-Type: application/json
Expires: Fri, 08 Apr 2016 21:42:28 GMT
Last-Modified: Fri, 08 Apr 2016 21:42:28 GMT
Vary: Accept-Language, Cookie

{
    "id": "fc6d8c0c43fc4630ad850ee518f1b9d0"
}

These pings were all received with a 200 OK in the main sentry app container log.

10.10.10.1 - - [08/Apr/2016:21:32:08 +0000] "POST /api/2/store/ HTTP/1.0" 200 277 "-" "raven-lua/0.4"
10.10.10.1 - - [08/Apr/2016:21:32:08 +0000] "POST /api/2/store/ HTTP/1.0" 200 277 "-" "raven-lua/0.4"
10.10.10.1 - - [08/Apr/2016:21:32:08 +0000] "POST /api/2/store/ HTTP/1.0" 200 277 "-" "raven-lua/0.4"
10.10.10.1 - - [08/Apr/2016:21:42:28 +0000] "POST /api/2/store/ HTTP/1.1" 200 277 "-" "HTTPie/0.9.3"

[mattrobenolt]

Since you're using docker here, could you provide a docker-compose or something similar to test with? We can't reproduce this on our side. :(

[mitsuhiko]

I will try to investigate this in detail. This has come up twice now.

[ye]

I am working on a docker-compose.yml file to integrate thefew messy docker commands thrown together. Should be able to share that today.

[ye]

Sorry about the delay in providing the test setup as I've just been able to put together the test environment setup in a compose file.

Note that I've forked the official docker-sentry repo with a minor tweak on the Dockerfile to accommodate MySQL database backend.

Unfortunately I am still using docker-compose version 1.4.2 as a result I can't tag the image built in the compose file so I have to pre-build Docker image.

All the producible steps and abbreviated result out are in this gist

As soon as all required containers were running, I went to the home page http://10.10.10.10:8080/sentry and start clicking around to create new project and get sentry DSN, the exception started showing up on the celery worker container log which I opened in a background window. Hope these help.

[mitsuhiko]

There is a potential fix for this in PR #3032.

[ye]

Tested bug fix branch in PR #3032. It did resolve this bug. I will be happy to retest the new release once it's out today if necessary. Thanks guys!

[ye]

Verified again that Release 8.3.2 solves the reported issue here and all seems to be working well!

Again, appreciate @mitsuhiko and @mattrobenolt for the bug fix and quick turnaround for making MySQL a viable alternative database backend.

[dcramer]

@ye fwiw MySQL is still not supported by Sentry. We only try to ensure that it still works for existing customers (specifically customers who are paying us for On-Premise support). There _will_ be a day when it's not supported at all.

[ye]

@dcramer thanks for the heads up. I don't think dropping support of MySQL is a good move but I understand your concerns.

Since this is an open source code, I think at the very minimum Sentry should leave the community to help out support on MySQL rather than shutting the door completely, IMMO.

[dcramer]

@ye its open source, but if you actually look at the contributions I wrote 90% of it, and most of the remainder is by people employed by Sentry. The simple truth is Sentry is a complex infrastructure project, and while people may want to keep using simple tools, there will be a day Sentry can no longer support them. You'll see this same trend as Sentry has evolved, where it started off as simply an app that you put into Django's INSTALLED_APPS.