Gunicorn: DatabaseWrapper objects created in a thread can only be used in that same thread

[ace-han]

I ran into this error these days.
gunicorn 18.0 is working okay while 19.0, 19.1.0 & 19.1.1 are not.
My db is MySql.
I was told to try out some pure-python mysql driver implementation like PyMySQL instead of MySQL-python as the app was built on Django.
I've followed the instruction based on this link to setup PyMySQL to work with Django. And still got the same error. Any idea?

Major Error Message:

DatabaseError at /en/admin/xforms_builder/xform/
DatabaseWrapper objects created in a thread can only be used in that same thread. The object with alias 'default' was created in thread id 21846768 and this is thread id 53270800.
Request Method: GET
Request URL:    http://xxx.xxx.org/en/admin/xforms_builder/xform/
Django Version: 1.6.5
Exception Type: DatabaseError
Exception Value:    
DatabaseWrapper objects created in a thread can only be used in that same thread. The object with alias 'default' was created in thread id 21846768 and this is thread id 53270800.
Exception Location: /home/xxx/.pyenv/versions/vvcms/lib/python2.7/site-packages/django/db/backends/__init__.py in validate_thread_sharing, line 499
Python Executable:  /home/xxx/.pyenv/versions/vvcms/bin/python
Python Version: 2.7.8

Error page:

Trigger Command

if [  xx = x$1x ]
then
      echo "Usage: $0 [dev|qa|pp|prod]"
  exit 0
fi

ORIGINAL_DIR=`pwd`
SCRIPT_HOME=$(cd "$(dirname "$0")"; pwd)

PROJECT_NAME=vvcms
export DJANGO_SETTINGS_MODULE=$PROJECT_NAME.settings.$1
echo "Running env $1"

PROJECT_HOME=$SCRIPT_HOME/..

source ~/.bash_profile
pyenv activate $PROJECT_NAME

cd $PROJECT_HOME
gunicorn -D $PROJECT_NAME.wsgi:application -b unix:$SCRIPT_HOME/$PROJECT_NAME.sock --workers 2 --pid $SCRIPT_HOME/$PROJECT_NAME.pid --worker-class gevent --preload --error-logfile $SCRIPT_HOME/$PROJECT_NAME.log && echo "$PROJECT_NAME successfully started"
deactivate
cd $ORIGINAL_DIR

Env:

(vvcms)[xxx@xxx vvcms]$ uname -a
Linux xxx 2.6.32-358.6.2.el6.x86_64 #1 SMP Thu May 16 20:59:36 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
(vvcms)[xxx@xxx vvcms]$ python -V
Python 2.7.8
(vvcms)[xxx@xxx vvcms]$ pip freeze
Django==1.6.5
Pillow==2.4.0
PyMySQL==0.6.2
South==0.8.4
Unidecode==0.04.16
argparse==1.2.1
cmsplugin-filer==0.9.9
dj-database-url==0.3.0
django-appconf==0.6
django-classy-tags==0.5.1
django-cms==3.0.2
django-email-extras==0.2
django-filer==0.9.5
django-forms-builder==0.11.1
django-mptt==0.5.5
django-polymorphic==0.5.5
django-reversion==1.8.0
django-sekizai==0.7
djangocms-admin-style==0.2.2
djangocms-column==1.3
djangocms-file==0.0.1
djangocms-flash==0.0.2
djangocms-googlemap==0.0.5
djangocms-inherit==0.0.1
djangocms-installer==0.5.1
djangocms-link==1.3.4
djangocms-picture==0.0.2
djangocms-snippet==1.0.2
djangocms-style==1.3
djangocms-teaser==0.0.1
djangocms-text-ckeditor==2.1.4
djangocms-video==0.0.1
easy-thumbnails==2.0.1
future==0.9.0
gevent==1.0.1
greenlet==0.4.2
gunicorn==19.1.1
html5lib==0.999
pysqlite==2.6.3
python-gnupg==0.3.6
pytz==2014.3
six==1.6.1
sphinx-me==0.2.1
wsgiref==0.1.2
xlwt==0.7.5
(vvcms)[xxx@xxx vvcms]$

MySql info:

mysql> status
--------------
mysql  Ver 14.14 Distrib 5.6.10, for Linux (x86_64) using  EditLine wrapper

Connection id:          833070
Current database:
Current user:           [email protected]
SSL:                    Not in use
Current pager:          stdout
Using outfile:          ''
Using delimiter:        ;
Server version:         5.6.16-log MySQL Community Server (GPL)
Protocol version:       10
Connection:             146.222.79.181 via TCP/IP
Server characterset:    latin1
Db     characterset:    latin1
Client characterset:    utf8
Conn.  characterset:    utf8
TCP port:               3306
Uptime:                 71 days 18 hours 5 min 12 sec

Threads: 37  Questions: 159493949  Slow queries: 13  Opens: 3296  Flush tables: 1  Open tables: 535  Queries per second avg: 25.726
--------------

mysql>

By the way, while I was installing gunicorn==19.1.1, got below error, too

(vvcms)[xxx@xxx vvcms]$ pip install pymysql gunicorn -U
Downloading/unpacking pymysql
  Downloading PyMySQL-0.6.2-py2.py3-none-any.whl (63kB): 63kB downloaded
Downloading/unpacking gunicorn from https://pypi.python.org/packages/2.7/g/gunicorn/gunicorn-19.1.1-py2.py3-none-any.whl#md5=905b2a090447c82bd66e62ee29b2287c
  Downloading gunicorn-19.1.1-py2.py3-none-any.whl (104kB): 104kB downloaded
Installing collected packages: pymysql, gunicorn
  Found existing installation: gunicorn 18.0
    Uninstalling gunicorn:
      Successfully uninstalled gunicorn
Compiling /home/xxx/.pyenv/versions/vvcms/build/gunicorn/gunicorn/workers/_gaiohttp.py ...
  File "/home/xxx/.pyenv/versions/vvcms/build/gunicorn/gunicorn/workers/_gaiohttp.py", line 64
    yield from self.wsgi.close()
             ^
SyntaxError: invalid syntax

Successfully installed pymysql gunicorn
Cleaning up...
(vvcms)[xxx@xxx vvcms]$

[tilgovi]

The syntax error during install has been reported several times. Don't worry about it.

[tilgovi]

The problem might be --preload. It causes your application to load before the worker fork. At that point, gevent has not loaded and patched the socket module.

Were you using --preload before you upgraded as well?

Try without preload and tell us if it works.

[ace-han]

@tilgovi , I tried. The application without --preload did work without any error as you expected.

However, I'm pretty sure that I've been launching the same code using --preload while using gunicorn=18.0 before upgrading to 19.0.0, 19.1.0 or 19.1.1.

I ran into this error after version upgrade

[tilgovi]

Using gunicorn command always or gunicorn_django? Or runserver? Always with
gevent?
On Sep 11, 2014 7:06 PM, "Ace Han" [email protected] wrote:

@tilgovi https://github.com/tilgovi , I tried. The application without
--preload did work without any error as you expected.

However, I'm pretty sure that I've been launching the same code using
--preload while using gunicorn=18.0 before upgrading to 19.0.0, 19.1.0 or
19.1.1.

I ran into this error after version upgrade

—
Reply to this email directly or view it on GitHub
https://github.com/benoitc/gunicorn/issues/879#issuecomment-55353567.

[ace-han]

Always using gunicorn command only and always with gevent

The trigger command already posted above, I wrote it that way for general purposes.

[tilgovi]

I'm not sure the cause from memory of any changes between 18 and 19. If you have a reliable way to quickly reproduce you could link to a source copy of gunicorn and use git-bisect to try to find the commit that changed this behavior. I understand that might be tricky because it might involve connection pools or some internals of PyMySQL and Gevent.

[pjrobertson]

I just stumbled across this issue. It occured when I switched to using preload_app=True. I've just switched back and everything works fine.
I'm using gunicorn --worker-class gevent as well.

So, by the looks of things, this is an issue with --preload and --worker-class gevent

Edit: Django 1.7, Py2.7.8, Gunicorn 19.1.1

[tilgovi]

I'm going to close this, @ace-han. If you can check that it did work on a previous version (with --preload) then we can re-open to investigate, but I'm not aware of any reason for this behavior to have changed.

[kwist-sgr]

django==1.7
gunicorn==18.0
gevent==1.0.1
greenlet 0.4.3

works with --preload

If upgrade gunicorn to 19.1.1 - does not work.

[acdha]

I can confirm that this used to work with gunicorn 18 but required removing --preload after upgrading to 19.

[logston]

I am also getting this error. My package/version list:

django==1.6.4
gevent==1.0
greenlet==0.4.5
gunicorn==19.1.1

I am not using the --preload flag.

[fletom]

I'm also getting this error (i.e. all requests fail with this traceback) with a completely standard Django setup (1.7.7) after upgrading from 1.8 to 1.9.3. I do not have --preload set anywhere, and adding preload_app = False to my gunicorn_conf.py file didn't make any difference. Will have to downgrade back to 1.8 because of this.

[tilgovi]

This discussion will continue on #927. Thanks for the feedback, everyone. It seems there's a real problem here.