Django is constantly causing our application to crash. After deployment the application is running fine, but once the initial instance is restarted/shutdown it often fails to start with an error similar to the following:
Traceback (most recent call last): File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 266, in Handle
result = handler(dict(self._environ), self._StartResponse)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/django-1.5/django/core/handlers/wsgi.py", line 236, in call
self.load_middleware()
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/django-1.5/django/core/handlers/base.py", line 53, in load_middleware
raise exceptions.ImproperlyConfigured('Error importing middleware %s: "%s"' % (mw_module, e))
ImproperlyConfigured: Error importing middleware myfolder.middleware: "No module named myfolder.middleware"
Our file structure is similar to this:
|- app.yaml
|- _ _ init _ _.py
|- settings.py
|- myfolder |
| |- _ _ init _ _.py
| |- middleware.py
| |- ...
|-...
|
Our app.yaml:
application: XXXXX
module: app
version: master
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /api/(login|logout|passwd|master.|banners.)
script: app.handler
secure: always
...
builtins:
- django_wsgi: on
libraries:
- name: django
version: 1.5
env_variables:
DJANGO_SETTINGS_MODULE: 'settings'
We have 2 modules in our application and they both exhibit this behaviour (they have similar configurations). Sometimes the modules will stay up for a whole day before crashing again. After they fail to load, all subsequent requests fail with he same error. Deploying one more time always solves the problem temporarily.
We are using plain django with CloudSql. The problem is not reproducible in the development server. After deployment everything in both modules works fine. All middleware, ndb, memcache, cloudsql, taskqueue, etc, including all the modules inside the "myfolder" and every other library xcopied.
The following attempts at solving this problem haven't worked:
We have tried using the appengine_config.py to force django to reload the settings with from django.conf import settings\nsettings._target = None\n
Originally we had shared settings inside "myfolder" and were importing them with "from myfolder.shared_settings import *" inside the root settings.py but django could not load the module myfolder.shared_settings either (similar problem)
using a custom mysettings.py and defining the DJANGO_SETTINGS_MODULE in the app.yaml or in python
The system is not live yet but will be soon and we are running out of options.
Other traces of similarly failing configurations:
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 266, in Handle
result = handler(dict(self._environ), self._StartResponse)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/django-1.5/django/core/handlers/wsgi.py", line 236, in __call__
self.load_middleware()
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/django-1.5/django/core/handlers/base.py", line 45, in load_middleware
for middleware_path in settings.MIDDLEWARE_CLASSES:
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/django-1.5/django/conf/__init__.py", line 53, in __getattr__
self._setup(name)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/django-1.5/django/conf/__init__.py", line 48, in _setup
self._wrapped = Settings(settings_module)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/django-1.5/django/conf/__init__.py", line 134, in __init__
raise ImportError("Could not import settings '%s' (Is it on sys.path?): %s" % (self.SETTINGS_MODULE, e))
ImportError: Could not import settings 'settings' (Is it on sys.path?): No module named myfolder.settings
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 239, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/lib_config.py", line 353, in __getattr__
self._update_configs()
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/lib_config.py", line 289, in _update_configs
self._registry.initialize()
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/lib_config.py", line 164, in initialize
import_func(self._modname)
File "/base/data/home/apps/s~blue-myapp/app:master.375531077560785947/appengine_config.py", line 17, in
settings._target = None
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/django-1.5/django/utils/functional.py", line 227, in __setattr__
self._setup()
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/django-1.5/django/conf/__init__.py", line 48, in _setup
self._wrapped = Settings(settings_module)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/django-1.5/django/conf/__init__.py", line 134, in __init__
raise ImportError("Could not import settings '%s' (Is it on sys.path?): %s" % (self.SETTINGS_MODULE, e))
ImportError: Could not import settings 'settings' (Is it on sys.path?): No module named myfolder.settings
This is our current appengine_config.py:
import sys
import logging
logging.debug(",\n".join(sys.path))
# Since Google App Engine's webapp framework uses Django templates, Django will half-initialize when webapp is loaded.
# This causes the initialization of the rest of Django's setting to be skipped. If you are getting this error, you need
# to explicitly force Django to reload your settings:
from django.conf import settings
settings._target = None
Logging sys.path from appengine_config.py does not change between a successful instance start and a failed instance start (apart from the XXXXXXXXXXX bit of course):
/base/data/home/apps/s~blue-persomi/app:master.3759720XXXXXXXXXXX,
/base/data/home/runtimes/python27/python27_dist/lib/python27.zip,
/base/data/home/runtimes/python27/python27_dist/lib/python2.7,
/base/data/home/runtimes/python27/python27_dist/lib/python2.7/plat-linux2,
/base/data/home/runtimes/python27/python27_dist/lib/python2.7/lib-tk,
/base/data/home/runtimes/python27/python27_dist/lib/python2.7/lib-old,
/base/data/home/runtimes/python27/python27_dist/lib/python2.7/lib-dynload,
/base/data/home/runtimes/python27/python27_dist/lib/python2.7/site-packages,
/base/data/home/runtimes/python27/python27_lib/versions/1,
/base/data/home/runtimes/python27/python27_lib/versions/third_party/MySQLdb-1.2.4b4,
/base/data/home/runtimes/python27/python27_lib/versions/third_party/django-1.5,
/base/data/home/runtimes/python27/python27_lib/versions/third_party/protorpc-1.0,
/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2,
/base/data/home/runtimes/python27/python27_lib/versions/third_party/webob-1.1.1,
/base/data/home/runtimes/python27/python27_lib/versions/third_party/yaml-3.10
It seems to be a path related issue as people have mentioned in your question's comments
Possible short-sighted solution add everything to your path manually - look at the top answer here:
How to import modules in Google App Engine?
At the very least, this will help narrow the problem to path related.
what the docs say: https://developers.google.com/appengine/docs/python/
The Python module include path includes your application's root
directory (the directory containing the app.yaml file). Modules you
create in your application's root directory are available using a path
from the root. Don't forget to create init.py files in
sub-directories, so Python will recognize the sub-directories as
packages.
so from what I can tell, b/c everything is at or below the app.yaml file in your question the path should already be correct.
double check all your __init__.py file files are in place and spelled correctly.
try deleting all of your *.pyc files and letting them be regenerated.
try importing from the container's folder name FOLDER_CONTAINING_YAML.myfolder.middleware
Add the following lines to your app.yaml
libraries:
- name: MySQLdb
version: "latest"
It is in the documentation here.
https://cloud.google.com/appengine/docs/python/cloud-sql/
Ideally it should have been documented in the GCM DJANGO guide. Would have saved me a lot of time.
Praveen
Related
I have a project in Django 1.1.4 and I am trying to set up this project on production with mod_wsgi but I am getting some errors :
My wsgi file code :
import os
import sys
import site
# Add the site-packages of the chosen virtualenv to work with
site.addsitedir('path_to_\site-packages')
# Add the app's directory to the PYTHONPATH
sys.path.append('path_to_dir')
sys.path.append('path_to_dir')
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
I have tried to debug the error like when I print something in my settings It doesn't get print so I guess my settings is not being called.
My wsgi and settings file are in the same level and I have used two ways to mention settings in my wsgi file like os.environ['DJANGO_SETTINGS_MODULE'] = 'settings' and os.environ['DJANGO_SETTINGS_MODULE'] = 'project.settings' but both fails.
.
error - log :
File "C:\\Python27\\lib\\site-packages\\django\\core\\handlers\\wsgi.py", line 230, in __call__
self.load_middleware()
File "C:\\Python27\\lib\\site-packages\\django\\core\\handlers\\base.py", line 33, in load_middleware
for middleware_path in settings.MIDDLEWARE_CLASSES:
File "C:\\Python27\\lib\\site-packages\\django\\utils\\functional.py", line 272, in __getattr__
self._setup()
File "C:\\Python27\\lib\\site-packages\\django\\conf\\__init__.py", line 40, in _setup
self._wrapped = Settings(settings_module)
File "C:\\Python27\\lib\\site-packages\\django\\conf\\__init__.py", line 75, in __init__
raise ImportError, "Could not import settings '%s' (Is it on sys.path? Does it have syntax errors?): %s" % (self.SETTINGS_MODULE, e)
ImportError: Could not import settings 'cbc_website.settings' (Is it on sys.path? Does it have syntax errors?): No module named cbc_website.settings
Python is interpreting your '\' in the path definitions as an escape character. https://docs.python.org/2/reference/lexical_analysis.html#string-literals
Use r'path' when specifying a literal path with backslashes. So instead of:
sys.path.append('C:\\\something')
You specify:
sys.path.append(r'C:\\\something')
in your wsgi.py file.
Have you tried to import yourprojectname.settings instead of just settings ?
This is what I remenber doing, and what is specified in the link https://github.com/django/django/blob/1.1.4/docs/howto/deployment/modwsgi.txt provided in comments.
I once wrote an django app, named superlists, and settings was in superlists.settings. It was for tutorial, and I was changing differnt settings, little realizing what I was doing.
And now when i starting new django project - it tells me it could not import superlists.settings
I reinstalled Pycharm, django, python (both versions 2.7 and 3.3), deleted all virtual envs, deleted everything that might be connected to it several times... IT IS STILL THERE! on a new, clean version of python, which even dont have django - there is DJANGO_SETTINGS_MODULE in sys.path pointing to superlists.settings. Each new project on a clean, with default settings, version of pycharm, tells me that it cannot import superlists.settings.
I delete DJANGO_SETTINGS_MODULE pointing to that settings from sys.path from cmd - exit then watch again - its there.
Where it takes it? I got no idea.
here is traceback:
Traceback (most recent call last):
File "D:/Python27/Lib/site-packages/django/bin/django-admin.py", line 5, in <module>
management.execute_from_command_line()
File "D:\Python27\Lib\site-packages\django\core\management\__init__.py", line 399, in execute_from_command_line
utility.execute()
File "D:\Python27\Lib\site-packages\django\core\management\__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "D:\Python27\Lib\site-packages\django\core\management\__init__.py", line 261, in fetch_command
commands = get_commands()
File "D:\Python27\Lib\site-packages\django\core\management\__init__.py", line 107, in get_commands
apps = settings.INSTALLED_APPS
File "D:\Python27\Lib\site-packages\django\conf\__init__.py", line 54, in __getattr__
self._setup(name)
File "D:\Python27\Lib\site-packages\django\conf\__init__.py", line 49, in _setup
self._wrapped = Settings(settings_module)
File "D:\Python27\Lib\site-packages\django\conf\__init__.py", line 132, in __init__
% (self.SETTINGS_MODULE, e)
ImportError: Could not import settings 'superlists\settings.py' (Is it on sys.path? Is there an import error in the settings file?): Import by filename is not supported.
It turned out that I created new environment system variable DJANGO_SETTINGS_MODULE. So evident, but it almost cost me sanity.
Change the environment variable from
DJANGO_SETTINGS_MODULE=superlist.superlist.settings
To DJANGO_SETTINGS_MODULE=superlist.settings
or use an __init__.py in superlist
I have a Django "project" that I have inherited, which I am developing in Eclipse. On my OS (windows 7 32 bit), I have Python 2.7.4 installed, likewise for my virtualenv. However, on my project (extracted from SVN) the Python version is 2.7 only.
This causes a conflict when trying to create another superuser (I do not know the original superuser name/password) where I get the message:
cannot import maxrepeat
How do I upgrade the python version located at:
c:\users\"username"\workspace\"project"\scripts
from 2.7 to 2.7.4?
Apologies if I have omitted some important details, or if I am asking the wrong question as I am newbie to Django/python development.
EDIT
Having spoken to a friend before referring back to these responses (thanks btw), he advised me to copy over the contents of the 'scripts' folder within my virtualenv to the folder:
c:\users\"username"\workspace\"project"\scripts
I did that, so in theory, they are both now running from python version 2.7.4.
However, when I run the script
python manage.py createsuperuser
I get the following FULL Traceback:
Traceback (most recent call last):
File "manage.py", line 10, in
execute_from_command_line(sys.argv)
File "C:\users\alecc\workspace\hub\lib\site-packages\django\core\management\__init__.py"
, line 453, in execute_from_command_line
utility.execute()
File "C:\users\alecc\workspace\hub\lib\site-packages\django\core\management\__init__.py"
, line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\users\alecc\workspace\hub\lib\site-packages\django\core\management\__init__.py"
, line 263, in fetch_command
app_name = get_commands()[subcommand]
File "C:\users\alecc\workspace\hub\lib\site-packages\django\core\management\__init__.py"
, line 109, in get_commands
apps = settings.INSTALLED_APPS
File "C:\users\alecc\workspace\hub\lib\site-packages\django\conf\__init__.py", line 53,
in __getattr__
self._setup(name)
File "C:\users\alecc\workspace\hub\lib\site-packages\django\conf\__init__.py", line 48,
in _setup
self._wrapped = Settings(settings_module)
File "C:\users\alecc\workspace\hub\lib\site-packages\django\conf\__init__.py", line 134,
in __init__
raise ImportError("Could not import settings '%s' (Is it on sys.path?): %s" % (self.SE
TTINGS_MODULE, e))
ImportError: Could not import settings 'hub.settings' (Is it on sys.path?): No module name
d hub.settings
I have checked the system variables and I'm sure my project is on the Python system path. I have also checked 'django.contrib.auth' is enabled in my INSTALLED_APPS in the settings.pyfile.
Edit 2
Many other posts suggest it's a cross over of Python versions. However when I check the version number using the command:
$scripts\python.exe --version
I get Python 2.7.4 for each installation (Project & virtualenv)
Based on this information:
File "C:\users\alecc\workspace\hub\lib\site-packages\django\conf\__init__.py", line 134,
in __init__
raise ImportError("Could not import settings '%s' (Is it on sys.path?): %s" % (self.SE
TTINGS_MODULE, e))
ImportError: Could not import settings 'hub.settings' (Is it on sys.path?): No module name
d hub.settings
It looks like your application is called hub? The problem here is that it's trying to import hub.settings - but it can't find it. So for some reason, your settings.py is not on the path.
You can check this by editing C:\users\alecc\workspace\hub\lib\site-packages\django\conf\__init__.py, and somewhere before line 134 you can put import sys; print(sys.path). Then check to see that the path where your settings.py file lives is available. If it is, something else weird is happening.
If not, just go ahead and pull those lines from the __init__, and try running manage from the same directory your settings file is in.
I am creating an app on Google App Engine. I am using Django 1.4 and Python 2.7. Everything works fine on localhost. But after deployment it is not running, I keep getting this on admin logs:
2012-12-15 15:02:41.870
/base/python27_runtime/python27_lib/versions/1/lib/cacerts/urlfetch_cacerts.txt missing; without this urlfetch will not be able to validate SSL certificates.
W 2012-12-15 15:02:41.870
No ssl package found. urlfetch will not be able to validate SSL certificates.
E 2012-12-15 15:02:46.086
Traceback (most recent call last):
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 196, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 266, in _LoadHandler
__import__(cumulative_path)
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/djangoappengine/main/__init__.py", line 28, in <module>
setup_env()
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/djangoappengine/boot.py", line 82, in setup_env
setup_logging()
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/djangoappengine/boot.py", line 130, in setup_logging
if not settings.DEBUG:
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/django/utils/functional.py", line 276, in __getattr__
self._setup()
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/django/conf/__init__.py", line 42, in _setup
self._wrapped = Settings(settings_module)
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/django/conf/__init__.py", line 87, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/settings.py", line 6, in <module>
import django.db.backends.mysql.base
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/django/db/backends/mysql/base.py", line 14, in <module>
raise ImproperlyConfigured("Error loading MySQLdb module: %s" % e)
ImproperlyConfigured: Error loading MySQLdb module: No module named MySQLdb
Any ideas why it is happening?
The 'Django Support' documentation hints at the solution, but doesn't make it explicit:
Since the standard django.db.backends.mysql backend uses MySQLdb internally, app.yaml must reference MySQLdb in the list of libraries.
Adding the following to app.yaml seems to fix the ImportError:
libraries:
- name: MySQLdb
version: "latest"
Note that MySQLdb is not currently included in the list of available third party libraries. I tried it on a whim and it seems to have fixed the issue for me, YMMV.enter link description here
As the Cloud SQL docs clearly state, you should be using ''google.appengine.ext.django.backends.rdbms' as your database engine setting.
If it's not working after adding the following to your app.yaml:
libraries:
- name: MySQLdb
version: "latest"
...then take the following steps:
Ensure MySQLdb is installed pip install mysql-python on Mac
Ensure the GAE devserver is pointing to your virtualenv/pip'd python 2.7 installation vs. Mac/*nix's default version as pip may not have installed it there. I resolved Can't find module: MySQLdb by updating the GAE devserver to use: /usr/local/bin/python2.7 e.g. in GAE devserver preferences for Python Path
Note: the preferences can be accessed through the application menus:
If you're using Windows: look under File > Preferences...
If you're using Mac, look under GoogleAppengineLauncher > Preferences...
So, I know you guys have been asked this question a million times, believe me I think I have read them all. I am getting the DJANGO_SETTINGS_MODULE is undefined error when trying to convert the Django polls app (the one made in the django tutorial) to run on the google app engine using the google cloud SQL.
Here is the actual stack trace given to me by the google app engine logs:
Traceback (most recent call last):
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 223, in Handle
result = handler(dict(self._environ), self._StartResponse)
File "/base/python27_runtime/python27_lib/versions/third_party/django-1.4/django/core/handlers/wsgi.py", line 219, in __call__
self.load_middleware()
File "/base/python27_runtime/python27_lib/versions/third_party/django-1.4/django/core/handlers/base.py", line 39, in load_middleware
for middleware_path in settings.MIDDLEWARE_CLASSES:
File "/base/python27_runtime/python27_lib/versions/third_party/django-1.4/django/utils/functional.py", line 184, in inner
self._setup()
File "/base/python27_runtime/python27_lib/versions/third_party/django-1.4/django/conf/__init__.py", line 40, in _setup
raise ImportError("Settings cannot be imported, because environment variable %s is undefined." % ENVIRONMENT_VARIABLE)
ImportError: Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.
I was getting the same error when trying to run it on dev_appserver.py provided with the google app engine but managed to remedy that by adding
env_variables:
DJANGO_SETTINGS_MODULE: 'DjangoTut.settings'
to the app.yaml file.
my project directory looks like this:
DjangoTut
DjangoTut
init.py
settings.py
urls.py
wsgi.py
polls
init.py
admin.py
models.py
tests.py
urls.py
views.py
app.yaml
manage.py
and my app.yaml file looks like this
application: hochotime
version: 10
runtime: python27
api_version: 1
threadsafe: true
libraries:
- name: django
version: "1.4"
builtins:
- django_wsgi: on
env_variables:
DJANGO_SETTINGS_MODULE: 'settings'
every other file is identical to those suggested here https://docs.djangoproject.com/en/1.4/intro/tutorial01/
I have tried adding
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'DjangoTut.settings'
to the start of just about every single python file but to no avail. I have tried moving the settings file into the project directory rather than the app directory and changing all the os.environ['DJANGO_SETTINGS_MODULE'] to point to just 'settings'. I have also tried configuring the settings directly as shown here https://docs.djangoproject.com/en/dev/topics/settings/#using-settings-without-setting-django-settings-module but still, I get the same error whenever I try to run it on the google servers.
Any help would be greatly appreciated.
Are you using Django-nonrel? If not, take a look at this appengine ariticle.
I would try moving settings.py out to the same directory as app.yaml. The remove the env_variables from app.yaml, you shouldn't need those for it to work. However you will need handlers defined if you don't already (not sure if you listed your whole app.yaml or not):
handlers:
- url: /static
static_dir: static
- url: /.*
script: main.py
The environment setting should be in the wsgi handler you setup (main.py in this example) as shown in the google article above.
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
Hope this gets you going down the right path.