I keep getting this when I try to load my Django application on production . I tried all the stackoverflow answers but nothing has fixed it. Any other ideas. (I'm using Django 1.5.2 and Apache)
Traceback (most recent call last):
File "/var/www/thehomeboard/wwwhome/wsgi.py", line 37, in <module>
application = get_wsgi_application()
File "/usr/local/lib/python2.7/dist-packages/django/core/wsgi.py", line 14, in get_wsgi_application
django.setup()
File "/usr/local/lib/python2.7/dist-packages/django/__init__.py", line 18, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python2.7/dist-packages/django/apps/registry.py", line 78, in populate
raise RuntimeError("populate() isn't reentrant")
RuntimeError: populate() isn't reentrant
This RuntimeError first occured for me after upgrading to Django 1.7 (and still is present with Django 1.8). It is usually caused by an Django application which raises an error, but that error is swallowed somehow.
Here's a workaround which works for me. Add it to your wsgi.py and the real error should be logged:
import os
import time
import traceback
import signal
import sys
from django.core.wsgi import get_wsgi_application
try:
application = get_wsgi_application()
print 'WSGI without exception'
except Exception:
print 'handling WSGI exception'
# Error loading applications
if 'mod_wsgi' in sys.modules:
traceback.print_exc()
os.kill(os.getpid(), signal.SIGINT)
time.sleep(2.5)
See this thread on modwsgi for more details.
In the end the problem that I had was that I tried to run a second Django app and did not have the following defined in my apache config:
WSGIDaemonProcess ...
WSGIProcessGroup ...
Just learned that you can run a single django app without defining them but when its two it produces a conflict.
There will be many reason to causes to populate() isn't reentrant error. If you look at the registry.py in your in django application probably inside this directory /python2.7/site-packages/django/apps
# app_config should be pristine, otherwise the code below won't
# guarantee that the order matches the order in INSTALLED_APPS.
if self.app_configs:
raise RuntimeError("populate() isn't reentrant")
As you see in the comment it says app_config should be pristine. Which means if one of configuration is not correct or required library missing it will rise this populate error. I got this error because I have missed sqlite installation. Even as you see there is no mentioning possible causes in the exception. I installed sqlite by this command on debian
pip install pysqlite
It solved my problem. My exception because of missing pysqlite.Your maybe having missing of another required packages or errors in your settings.py
For those using AWS Lambda (and optionally using zappa) this can happen when the size of code & dependencies zipped into a deployment package exceeds 250MB after decompression.
Typically the zip may only be 50 MB but may decompress to over 250MB so you may need to manually unzip the deployment package to check it isn't too large when uncompressed.
https://docs.aws.amazon.com/lambda/latest/dg/limits.html
Full disclosure - "populate() isn't reentrant" errors can have multiple causes, and checking any recent config or program changes is a very good idea.
However, this error can also occur when Apache is updated, and a module is no longer valid/is corrupted/needs refreshing. This occurred to us on Webfaction after an Apache update (but may happen on any host).
Using the Apache restart script will NOT help this, because the modules remain loaded on the restart. Depending on your system, and whether the mods are cached even when Apache is shut down, this may help.
Fully stop Apache. On Webfaction, that is:
~/webapps/<YOUR WEB APP>/apache2/bin/stop
Wait a few seconds, and then...
~/webapps/<YOUR WEB APP>/apache2/bin/start
That should correct the issue. If your system caches the mods, you may have to flush the cache before starting.
Hope this helps!
Here is the link they gave me (I know the error is different, but we had this happen for the same reason with the populate error as well):
https://statusblog.webfaction.com/2018/05/16/regarding-glibc_private-errors-in-your-python-application/
For me this error was caused, because I hadn't correctly split my INSTALLED_APPS for local and production. Meaning in local I was using django-cors-headers and in production I was not. But I accidentally had left django-cors-headers even though I removed it from my production requirements.txt. After delete cors-headers from the installed apps in production the error went away.
Related
Using flask, I'm trying to send a file to the user on clicking a button in UI using send_from_directory function. It used to work fine. I wanted to change the repo and since changing it, I'm no more able to download the file. On looking at the supervisor log, I see this:
[9617] [ERROR] Error handling request
Traceback (most recent call last):
File "path_to_file/venv/lib/python3.4/site-packages/gunicorn/workers/sync.py", line 182, in handle_request
resp.write_file(respiter)
File "path_to_file/venv/lib/python3.4/site-packages/gunicorn/http/wsgi.py", line 385, in write_file
if not self.sendfile(respiter):
File "path_to_file/venv/lib/python3.4/site-packages/gunicorn/http/wsgi.py", line 375, in sendfile
self.sock.sendfile(respiter.filelike, count=nbytes)
AttributeError: 'socket' object has no attribute 'sendfile'
In the same repo, this works fine locally. But when trying in remote server using the gunicorn + supervisor + nginx setup, I get the above error message. I do get 200 Ok response in the application log file. Spent a lot of time trying to fix but without success.
Also, the notable difference between the working app between the previous repo and the non-working current repo is the python version. Previous: python2.7, Current: python3.4
For me, usually when a script works locally but not when hosted its one (or more) of these possibilities:
Location / path to the files is different
older version of python
older version of the library
Pointing the virtual env to Python 3.6 and upgrading all the relevant libraries including Flask resolved the issue.
I'm moving a web application from a Windows environment to CentOS 5.11 and Apache, and with everything installed, I'm receiving a 500 when I try to load up the site. The error log shows this:
mod_wsgi (pid=5461): Target WSGI script '/usr/local/treehouse/wsgi/index.wsgi' cannot be loaded as Python module.
mod_wsgi (pid=5461): Exception occurred processing WSGI script '/usr/local/treehouse/wsgi/index.wsgi'.
Traceback (most recent call last):
File "/usr/local/treehouse/wsgi/index.wsgi", line 11, in <module>
from django.core.wsgi import get_wsgi_application
ImportError: No module named 'django.core.wsgi'
I found this question (and the related ones), which appears to be similar but none of the answers there fix the issue. I'm not running in a virtualenv, and django appears to be installed correctly, as I can run the offending statement:
>>> from django.core.wsgi import get_wsgi_application
in the interactive Python interpreter and it works just fine. Here's the script causing the error (it's just the default index.wsgi that you see everywhere you look for this stuff):
import os, sys
sys.path.append('/usr/local/treehouse/apple')
sys.path.append('/usr/local/treehouse/apple/apple')
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "apple.settings")
from django.core.wsgi import get_wsgi_application application = get_wsgi_application()
Obviously the last line causes the error, but here's the fun part: the last line remains the problem, even if I add this just above it:
import django.core
import django.core.handlers
Both of the underlying packages import without a hitch (django.core.wsgi is just a thin wrapper around some functionality in django.core.handlers.wsgi). It's only when I try to access the wsgi packages that problems occur. Inspecting sys.path within this script shows all the right directories (/usr/local/lib/python3.4/site-packages, etc.). And again, running the exact same code from the interactive interpreter causes no errors.
I tried an uninstall/reinstall of django (pip install django==1.6.5), then I reset the VM to a totally clean snapshot and rebuilt/reinstalled everything, and still I get the exact same behavior.
What is going on?
Some of the django source files pulled down by pip are saved with Windows line endings: that is, they end with \r\n rather than \n. On *nix systems, this has the effect of breaking imports in affected files, because instead of looking for django.core.wsgi, it tries to import django.core.wsgi\r.
This is why django.core.handlers can still be imported: the __init__.py file is empty, so there are no line endings to corrupt.
To fix it, run dos2unix on affected files. I'm not sure how many files are actually affected (except that it's many of them) so I just hit all of them with a quick Python script from the interpreter:
import os
from os.path import abspath, join
for root, _, files in os.walk('/usr/local/lib/python3.4/site-packages/django'):
for f in files:
os.system('dos2unix %s' % abspath(join(root, f)))
Et voilà, no more import errors!
Storytime
I stumbled on this issue after I began hacking on the django source files in desperation. I edited django/core/__init__.py (normally empty) by adding this:
from . import wsgi
I modified index.wsgi to include this:
import django.core
django.core.wsgi # no-op to see if we can access it
and ended up with a fascinating new error:
Traceback (most recent call last):
File "/usr/local/treehouse/wsgi/index.wsgi", line 11, in <module>
import django.core
File "/usr/local/lib/python3.4/site-packages/django/core/__init__.py", line 1, in <module>
from . import wsgi
File "/usr/local/lib/python3.4/site-packages/django/core/wsgi.py", line 1, in <module>
from django.core.handlers.wsgi import WSGIHandler
ImportError: No module named 'django.core.handlers.wsgi'
So from within django/core/__init__.py, I can access django/core/wsgi.py. But django.core.handlers.wsgi is out of reach. I deleted the changes to reproduce the original error--but the error had changed. Now I was getting the No module named 'django.core.handlers.wsgi' even without the explicit relative import.
This was when it hit me. I opened up django/core/handlers/wsgi.py in gedit, clicked the end of a line, pressed the spacebar, deleted the insertion, and saved the file. It should have been a no-op. But when I restarted the server, suddenly the import error had moved on to a different django import. The only logical explanation was that the line endings were wrong, and by re-saving the file in gedit, I was silently fixing them.
I've tried everything that I could find to fix this issue, and I'm starting to tear my hair out a little.
I'm getting this error:
django.core.exceptions.ImproperlyConfigured: Requested setting CACHES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
My scripts run fine when I do:
python3 ./manage.py runserver
However, whenever I try to run tests, I get the above error...
I use a VirtualEnv, that inherits nothing globally, everything is installed (with the correct version), and within my manage.py I have set:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "<appname>.settings")
I'm using PyCharm Proffesional to develop, and I've tried running the tests in both the IDE, and in the shell.
Within the shell I'm using :
python3 manage.py test
The shell is finding no tests. The test is basic, and I'm not really all that bothered about the content of it currently as it's the environment I'm struggling with. UPDATE: I have solved the issue with the shell. Tests must be defined by:
def test_<name>():
However this hasn't solved my issue with PyCharm.
I have also called:
settings.configure()
Which told me that it was already configured.
Please note that I am not using any database with Django, and I have commented the appropriate things out of the settings.
The full error is:
Traceback (most recent call last):
File "/root/kiloenv/lib/python3.4/site-packages/django/conf/__init__.py", line 38, in _setup
settings_module = os.environ[ENVIRONMENT_VARIABLE]
File "/usr/lib/python3.4/os.py", line 631, in __getitem__
raise KeyError(key) from None
KeyError: 'DJANGO_SETTINGS_MODULE'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/<username>/Documents/<dirname>/<appname>/tests.py", line 1, in <module>
from django.test.utils import setup_test_environment
File "/root/virtualenv/lib/python3.4/site-packages/django/test/__init__.py", line 5, in <module>
from django.test.client import Client, RequestFactory
File "/root/virtualenv/lib/python3.4/site-packages/django/test/client.py", line 11, in <module>
from django.contrib.auth import authenticate, login, logout, get_user_model
File "/root/virtualenv/lib/python3.4/site-packages/django/contrib/auth/__init__.py", line 6, in <module>
from django.middleware.csrf import rotate_token
File "/root/virtualenv/lib/python3.4/site-packages/django/middleware/csrf.py", line 14, in <module>
from django.utils.cache import patch_vary_headers
File "/root/virtualenv/lib/python3.4/site-packages/django/utils/cache.py", line 26, in <module>
from django.core.cache import get_cache
File "/root/virtualenv/lib/python3.4/site-packages/django/core/cache/__init__.py", line 69, in <module>
if DEFAULT_CACHE_ALIAS not in settings.CACHES:
File "/root/virtualenv/lib/python3.4/site-packages/django/conf/__init__.py", line 54, in __getattr__
self._setup(name)
File "/root/virtualenv/lib/python3.4/site-packages/django/conf/__init__.py", line 47, in _setup
% (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting CACHES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
I have ran PyCharm under sudo to ensure that the issue wasn't permissions as I'm storing the env under root.
EDIT: I've just discovered that my tests which do not use Django are running fine, but Pycharm is still throwing several failures. These failures aren't the individual tests, they're just the error which I have mentioned here (there's 341 tests that aren't Django related). I only have one test which uses Django, which will not get past initializing and throwing the mentioned error.
Hope I've been explanitory
Use this
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'
instead of
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "<appname>.settings")
In your python script, you are trying to access Django models before setting the environment try it in this order :
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "<appname>.`settings`")
from <appname>.models import Class1, Class2,...
If you are using PyCharm Pro, you can either test your app by action 'Run Django Console...'. After you click 'test', it will prompt you for the app you want to test.
or
Create a Django test in Run/Debug configuration.
since, I don't want to add this in every file, I created
a file called usercustomize.py and added this content:
import os
os.environ['DJANGO_SETTINGS_MODULE']='foosite.settings'
The file path in my case:
src/foosite/usercustomize.py
I installed foosite with "pip install -e". I am unsure if this works for normal installs.
In my case I needed to use
python3 manage.py check --deploy
instead of
django-admin check --deploy
fill "working directory" correctly with your working path
then write the copy down in it's place
I got this error when I imported TestCase from unittest2. I fixed it by importing TestCase by:
from django.test import TestCase
My suspicions were raised when the test was labelled "Unittest" rather than "Test". I had to run the whole suite before Pycharm realised I'd made this change...
If you're using PyCharm you may have forgotten to choose your debug configuration here:
I was trying to deploy django on a shared webhost which is not django specific. Host offers old version of python installed, but since I have ssh access ability, I've managed to extend python installation with modules I need (including django) by installing them locally in my home folder.
Ok, so I've created django project, did tweaks that needed to do (setting up PYTHONPATH and PATH globals etc.), made django.fcgi script which starts django and did ./django.fcgi from shell.
This is the response:
WSGIServer: missing FastCGI param REQUEST_METHOD required by WSGI!
WSGIServer: missing FastCGI param SERVER_NAME required by WSGI!
WSGIServer: missing FastCGI param SERVER_PORT required by WSGI!
WSGIServer: missing FastCGI param SERVER_PROTOCOL required by WSGI!
Traceback (most recent call last):
File "/home/tentacle/lib/python2.4/site-packages/flup-1.0.3.dev_20110405-py2.4.egg/flup/server/fcgi_base.py", line 574, in run
protocolStatus, appStatus = self.server.handler(self)
File "/home/tentacle/lib/python2.4/site-packages/flup-1.0.3.dev_20110405-py2.4.egg/flup/server/fcgi_base.py", line 1159, in handler
result = self.application(environ, start_response)
File "/home/tentacle/lib/python2.4/site-packages/django/core/handlers/wsgi.py", line 272, in __call__
response = self.get_response(request)
File "/home/tentacle/lib/python2.4/site-packages/django/core/handlers/base.py", line 169, in get_response
response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
File "/home/tentacle/lib/python2.4/site-packages/django/core/handlers/base.py", line 202, in handle_uncaught_exception
from django.views import debug
File "/home/tentacle/lib/python2.4/site-packages/django/views/debug.py", line 9, in <module>
from django.template import (Template, Context, TemplateDoesNotExist,
File "/home/tentacle/lib/python2.4/site-packages/django/template/__init__.py", line 53, in <module>
from django.template.base import (ALLOWED_VARIABLE_CHARS, BLOCK_TAG_END,
MemoryError
Status: 500 Internal Server Error
Content-Type: text/html
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>Unhandled Exception</title>
</head><body>
<h1>Unhandled Exception</h1>
<p>An unhandled exception was thrown by the application.</p>
</body></html>
So, I'm aware that this issue is about limited memory for my user (wrong?), but is it so low that I cannot run bare django?
Just for things to be worse, a month ago the same provider gave me a test account, and a was able not just to run django, but to install locally newer version of python and run it nice.
I did asked my web host support for help, but I didn't got any usable answer. Also since they are resellers I'm not very sure how much they help.
Is there a way to overcome this problem? Any suggestion is welcome.
Thanks in advance
-----------------------------------Update--------------------------------------------------
#!/home/<username>/bin/python
import os, sys
sys.path.insert(0, "/home/<username>/djangoprojects")
sys.path.insert(0, "/home/<username>/djangoprojects/testproject")
os.environ['PATH']= "/home/<username>/bin:"+os.environ['PATH']
os.environ['DJANGO_SETTINGS_MODULE'] = 'testproject.settings'
from django.core.servers.fastcgi import runfastcgi
runfastcgi(method="threaded", daemonize="false")
This is how look like my configuration. I must admit that there was a little error (interpreter path), but 'settings' file path was good.
After correcting the interpreter path to right one, First I've got StringError (some triple quoted doc string in one of djangos files was not closed - hm, hardly), than in next run MemoryError, and again MemoryError and so on. After a while (an hour) I tried to execute script again (no further changes made) and constantly ending up with Segmentation fault (core dumped).
Any suggestions?
When running it from the command line, you need to specify the Django parameters explicitly. Assuming you are running the script from the same folder as your settings.py is in:
import os, sys
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
# ... rest of your startup script...
Also, check if your .htaccess file still has a correct RewriteRule to your .fcgi script.
Here's the setup:
Django (1.2) app on mod_wsgi that imports ctypes
Python 2.6.5
Apache 2.2.3
SELinux disabled
RedHat EL 5 64bit
some of the file system is mounted over nfs
Occasionally, when I restart apache I get an import error when it's trying to import ctypes. Every incoming request fails with a 500 error. If I restart apache usually everything just starts working again.
Here's the stack trace of the error:
Traceback (most recent call last):
File "/home/appfirst/django/django/core/handlers/base.py", line 80, in get_response
response = middleware_method(request)
-------------- A BUNCH OF DJANGO MIDDLEWARE STUFF HERE -------------
File "/home/appfirst/django/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/home/appfirst/backend/backend/streamer/views.py", line 6, in <module>
import appfirst.main.models as FEmodels
File "/home/appfirst/frontend/appfirst/main/models.py", line 27, in <module>
import numpy, math, mpmath
File "/usr/lib64/python2.6/site-packages/numpy/__init__.py", line 43, in <module>
import ctypeslib
File "/usr/lib64/python2.6/site-packages/numpy/ctypeslib.py", line 9, in <module>
import ctypes
File "/usr/lib64/python2.6/ctypes/__init__.py", line 546, in <module>
CFUNCTYPE(c_int)(lambda: None)
MemoryError
I thought it might be related to this bug, but I have SELinux turned off which I thought would mean this case could never occur:
http://bugs.python.org/issue5504
Any suggestions on how to reproduce it consistently and/or fix it? This is really stumping me!
I've run into this bug too. In my case it occurs when I exec a Python script from within a PHP script running under Apache on a 64-bit Linux system. [The Python code being run is the front-end to a pypy sandbox.] The same bit of code works fine on a 32-bit system and even works fine when the PHP script is executed directly from the command line. My "fix" has been simply to comment out that line "CFUNCTYPE(c_int)(lambda: None)" in ctypes/init.py. It's the last line of the file and is preceded by the following comment, showing that the programmer doesn't understand what's going on, either!
# XXX for whatever reasons, creating the first instance of a callback
# function is needed for the unittests on Win64 to succeed. This MAY
# be a compiler bug, since the problem occurs only when _ctypes is
# compiled with the MS SDK compiler. Or an uninitialized variable?
CFUNCTYPE(c_int)(lambda: None)
Clearly there's a deeper problem somewhere in cpython, but the fix works for me.
In case others happen to run into this issue, my fix was to upgrade from Python 3.7 to 3.8.
This offending line: https://github.com/python/cpython/blob/3.7/Lib/ctypes/__init__.py#L273
Was removed in 3.8: https://github.com/python/cpython/blob/3.8/Lib/ctypes/__init__.py#L261-L270
Hopefully, it can save someone the headache of trying to monkey patch ctype/__init__.py which was unsuccessful for me due to the way conda pack handles standard Python libraries.
Hope that helps 🤞
Consider turning SELinux off.
It should solve the problem.