RuntimeError("populate() isn't reentrant") on elastic beanstalk - python

I'm running django on Elastic Beanstalk and after pushing a new version of the app I'm getting a 500 internal server error.
The logs show this:
apps.populate(settings.INSTALLED_APPS)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/apps/registry.py", line 83, in populate
raise RuntimeError("populate() isn't reentrant")
RuntimeError: populate() isn't reentrant
mod_wsgi (pid=21994): Target WSGI script '/opt/python/current/app/wsgi.py' cannot be loaded as Python module.
mod_wsgi (pid=21994): Exception occurred processing WSGI script '/opt/python/current/app/wsgi.py'.
Traceback (most recent call last):
File "/opt/python/current/app/wsgi.py", line 16, in <module>
application = get_wsgi_application()
File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application
django.setup(set_prefix=False)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/__init__.py", line 24, in setup
Any advice on debugging this? My attempts to understand it/google it so far have been unsuccessful.

it normally happens when the interpreter failed to load the source(provided you had it running) or (if you are deploying) it points out to a configuration issue mostly related to the number of threads that try to load the same interpreter.
I would suggest you to do a hard restart of the service that runs the app. This will most likely solve it. If the error persists check your wsgi configuration.

Related

django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet. when I run uwsgi

In my virtual environment, when I use uwsgi --ini uwsgi.ini to run the uWSGI.
I get the bellow error:
mapped 543168 bytes (530 KB) for 20 cores
*** Operational MODE: threaded ***
Traceback (most recent call last):
File "Qiyun02/wsgi.py", line 18, in <module>
from 管理员后台.网站管理.admin_website_chat.views import sio
File "./管理员后台/网站管理/admin_website_chat/views.py", line 2, in <module>
from 管理员后台.用户管理admin_usermanage.models import User
File "./管理员后台/用户管理/admin_usermanage/models.py", line 6, in <module>
from django.contrib.auth.models import AbstractUser
File "/home/lll/repo/myProject/venv_dist/lib/python3.5/site-packages/django/contrib/auth/models.py", line 4, in <module>
from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
File "/home/lll/repo/QmyProjectyun02/venv_dist/lib/python3.5/site-packages/django/contrib/auth/base_user.py", line 52, in <module>
class AbstractBaseUser(models.Model):
File "/home/lll/repo/myProject/venv_dist/lib/python3.5/site-packages/django/db/models/base.py", line 110, in __new__
app_config = apps.get_containing_app_config(module)
File "/home/lll/repo/myProject/venv_dist/lib/python3.5/site-packages/django/apps/registry.py", line 247, in get_containing_app_config
self.check_apps_ready()
File "/home/lll/repo/myProject/venv_dist/lib/python3.5/site-packages/django/apps/registry.py", line 125, in check_apps_ready
raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 32347)
spawned uWSGI worker 1 (pid: 32349, cores: 20)
spawned uWSGI http 1 (pid: 32350)
But in my virtual environment I use the python check, there is no issues.
(venv_dist) [ldl#www myProject]$ python3 manage.py check
System check identified no issues (0 silenced).
I also can python3 manage.py runserver in my virtual environment.
Why there get this AppRegistryNotReady error when I run uwsgi?
Or, does the uwsgi support non-latin characters? I have some Chinese characters there.
Traceback (most recent call last):
File "Qiyun02/wsgi.py", line 18, in <module>
from 管理员后台.网站管理.admin_website_chat.views import sio
...
you should not import things from your view in uwsgi.py, because django has not setup when uwsgi.py is loaded.
and when you run python3 manage.py runserver, it has nothing to do with uwsgi.py.
Because when use the the django has not setup, you should setup it before your code:
import django
django.setup()
... # your code
Hope this help you.

wfastcgi fails import cx_Oracle, but `python -c "import cx_Oracle" succeeds

I have IIS setup with fastCGI, serving a flask app. So far so good. Next I whish to add some database connectivity, so I add the line import cx_Oracle to my app. Now this error is thrown:
Error occurred while reading WSGI handler:
Traceback (most recent call last):
File "D:\Anaconda2\lib\site-packages\wfastcgi.py", line 791, in main
env, handler = read_wsgi_handler(response.physical_path)
File "D:\Anaconda2\lib\site-packages\wfastcgi.py", line 633, in read_wsgi_handler
handler = get_wsgi_handler(os.getenv("WSGI_HANDLER"))
File "D:\Anaconda2\lib\site-packages\wfastcgi.py", line 616, in get_wsgi_handler
raise ValueError('"%s" could not be imported%s' % (handler_name, last_tb))
ValueError: "Bloomberg_server.app" could not be imported:
Traceback (most recent call last):
File "D:\Anaconda2\lib\site-packages\wfastcgi.py", line 600, in get_wsgi_handler
handler = __import__(module_name, fromlist=[name_list[0][0]])
File "D:\website\init__.py", line 6, in import cx_Oracle
ImportError: DLL load failed: The specified module could not be found. StdOut: StdErr:
As the title suggests I fail to reproduce the issue in a controlled environment. The very same import statement works fine in the conda environment and moreover, I can run the Flask debug server just fine with pages that rely on a database connection.
I am at loss. Who has a clue what's going on here? The path/oracle_home variables are pointing to the instant client and I have only one python environment installed.
I am too embarrased to admit how long this has taken me, but I've found the answer.
FastCGI's core business is keeping subprocesses alive so that subsequent calls to the server do not require booting a Python environment. In other words, after installing a python package it is advised to reboot. I solved my first question on SO by rebooting..
The answer to this question got me thinking in the right direction.

Populate() isn't reentrant Django Google App Engine

I'm running google app engine, python 2.7, and importing Django 1.8. After pulling in python-firebase (-e git://github.com/ozgur/python-firebase.git#egg=python-firebase), I get the following error
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/base/data/home/apps/s~trac-us/1.392706776803493304/main.py", line 25, in <module>
application = django.core.wsgi.get_wsgi_application()
File "/base/data/home/apps/s~trac-us/1.392706776803493304/libs/django/core/wsgi.py", line 14, in get_wsgi_application
django.setup()
File "/base/data/home/apps/s~trac-us/1.392706776803493304/libs/django/__init__.py", line 18, in setup
apps.populate(settings.INSTALLED_APPS)
File "/base/data/home/apps/s~trac-us/1.392706776803493304/libs/django/apps/registry.py", line 78, in populate
raise RuntimeError("populate() isn't reentrant")
RuntimeError: populate() isn't reentrant
Many of the other solutions have suggested to alter the wsgi.py file, because django gets installed right before deployment in the requirements.txt file.
Ultimately this leads to the server stating: "Error: Server Error
The server encountered an error and could not complete your request.
Please try again in 30 seconds." How can I resolve this?
I was facing the same issue and removing the database configuration for a sqlite database connection fixed this for me. Issue might've been that the sqlite library that django is looking for underneath was not present and thus, removing the sqlite configuration fixed that for me. pff, I was not using sqlite in that project any way.
However, as per a google groups thread I landed on, the error:
raise RuntimeError("populate() isn't reentrant")
RuntimeError: populate() isn't reentrant
is caused when one of the application that you're trying to load leads to an error within, which leads to this. I'll provide a link to the thread here, and although it's different from your problem(you probably solved it already), you might still find it useful:
Link to groups thread
The answer I was talking about
Hope you find it useful.

Django populate() isn't reentrant

I keep getting this when I try to load my Django application on production . I tried all the stackoverflow answers but nothing has fixed it. Any other ideas. (I'm using Django 1.5.2 and Apache)
Traceback (most recent call last):
File "/var/www/thehomeboard/wwwhome/wsgi.py", line 37, in <module>
application = get_wsgi_application()
File "/usr/local/lib/python2.7/dist-packages/django/core/wsgi.py", line 14, in get_wsgi_application
django.setup()
File "/usr/local/lib/python2.7/dist-packages/django/__init__.py", line 18, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python2.7/dist-packages/django/apps/registry.py", line 78, in populate
raise RuntimeError("populate() isn't reentrant")
RuntimeError: populate() isn't reentrant
This RuntimeError first occured for me after upgrading to Django 1.7 (and still is present with Django 1.8). It is usually caused by an Django application which raises an error, but that error is swallowed somehow.
Here's a workaround which works for me. Add it to your wsgi.py and the real error should be logged:
import os
import time
import traceback
import signal
import sys
from django.core.wsgi import get_wsgi_application
try:
application = get_wsgi_application()
print 'WSGI without exception'
except Exception:
print 'handling WSGI exception'
# Error loading applications
if 'mod_wsgi' in sys.modules:
traceback.print_exc()
os.kill(os.getpid(), signal.SIGINT)
time.sleep(2.5)
See this thread on modwsgi for more details.
In the end the problem that I had was that I tried to run a second Django app and did not have the following defined in my apache config:
WSGIDaemonProcess ...
WSGIProcessGroup ...
Just learned that you can run a single django app without defining them but when its two it produces a conflict.
There will be many reason to causes to populate() isn't reentrant error. If you look at the registry.py in your in django application probably inside this directory /python2.7/site-packages/django/apps
# app_config should be pristine, otherwise the code below won't
# guarantee that the order matches the order in INSTALLED_APPS.
if self.app_configs:
raise RuntimeError("populate() isn't reentrant")
As you see in the comment it says app_config should be pristine. Which means if one of configuration is not correct or required library missing it will rise this populate error. I got this error because I have missed sqlite installation. Even as you see there is no mentioning possible causes in the exception. I installed sqlite by this command on debian
pip install pysqlite
It solved my problem. My exception because of missing pysqlite.Your maybe having missing of another required packages or errors in your settings.py
For those using AWS Lambda (and optionally using zappa) this can happen when the size of code & dependencies zipped into a deployment package exceeds 250MB after decompression.
Typically the zip may only be 50 MB but may decompress to over 250MB so you may need to manually unzip the deployment package to check it isn't too large when uncompressed.
https://docs.aws.amazon.com/lambda/latest/dg/limits.html
Full disclosure - "populate() isn't reentrant" errors can have multiple causes, and checking any recent config or program changes is a very good idea.
However, this error can also occur when Apache is updated, and a module is no longer valid/is corrupted/needs refreshing. This occurred to us on Webfaction after an Apache update (but may happen on any host).
Using the Apache restart script will NOT help this, because the modules remain loaded on the restart. Depending on your system, and whether the mods are cached even when Apache is shut down, this may help.
Fully stop Apache. On Webfaction, that is:
~/webapps/<YOUR WEB APP>/apache2/bin/stop
Wait a few seconds, and then...
~/webapps/<YOUR WEB APP>/apache2/bin/start
That should correct the issue. If your system caches the mods, you may have to flush the cache before starting.
Hope this helps!
Here is the link they gave me (I know the error is different, but we had this happen for the same reason with the populate error as well):
https://statusblog.webfaction.com/2018/05/16/regarding-glibc_private-errors-in-your-python-application/
For me this error was caused, because I hadn't correctly split my INSTALLED_APPS for local and production. Meaning in local I was using django-cors-headers and in production I was not. But I accidentally had left django-cors-headers even though I removed it from my production requirements.txt. After delete cors-headers from the installed apps in production the error went away.

Google app engine + python (django) deployment error: Error loading MySQLdb module

I am creating an app on Google App Engine. I am using Django 1.4 and Python 2.7. Everything works fine on localhost. But after deployment it is not running, I keep getting this on admin logs:
2012-12-15 15:02:41.870
/base/python27_runtime/python27_lib/versions/1/lib/cacerts/urlfetch_cacerts.txt missing; without this urlfetch will not be able to validate SSL certificates.
W 2012-12-15 15:02:41.870
No ssl package found. urlfetch will not be able to validate SSL certificates.
E 2012-12-15 15:02:46.086
Traceback (most recent call last):
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 196, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 266, in _LoadHandler
__import__(cumulative_path)
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/djangoappengine/main/__init__.py", line 28, in <module>
setup_env()
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/djangoappengine/boot.py", line 82, in setup_env
setup_logging()
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/djangoappengine/boot.py", line 130, in setup_logging
if not settings.DEBUG:
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/django/utils/functional.py", line 276, in __getattr__
self._setup()
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/django/conf/__init__.py", line 42, in _setup
self._wrapped = Settings(settings_module)
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/django/conf/__init__.py", line 87, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/settings.py", line 6, in <module>
import django.db.backends.mysql.base
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/django/db/backends/mysql/base.py", line 14, in <module>
raise ImproperlyConfigured("Error loading MySQLdb module: %s" % e)
ImproperlyConfigured: Error loading MySQLdb module: No module named MySQLdb
Any ideas why it is happening?
The 'Django Support' documentation hints at the solution, but doesn't make it explicit:
Since the standard django.db.backends.mysql backend uses MySQLdb internally, app.yaml must reference MySQLdb in the list of libraries.
Adding the following to app.yaml seems to fix the ImportError:
libraries:
- name: MySQLdb
version: "latest"
Note that MySQLdb is not currently included in the list of available third party libraries. I tried it on a whim and it seems to have fixed the issue for me, YMMV.enter link description here
As the Cloud SQL docs clearly state, you should be using ''google.appengine.ext.django.backends.rdbms' as your database engine setting.
If it's not working after adding the following to your app.yaml:
libraries:
- name: MySQLdb
version: "latest"
...then take the following steps:
Ensure MySQLdb is installed pip install mysql-python on Mac
Ensure the GAE devserver is pointing to your virtualenv/pip'd python 2.7 installation vs. Mac/*nix's default version as pip may not have installed it there. I resolved Can't find module: MySQLdb by updating the GAE devserver to use: /usr/local/bin/python2.7 e.g. in GAE devserver preferences for Python Path
Note: the preferences can be accessed through the application menus:
If you're using Windows: look under File > Preferences...
If you're using Mac, look under GoogleAppengineLauncher > Preferences...

Categories