I'm having difficulty calling a django script. Here is what I'm currently doing in my root directory:
>>> import os
>>> os.environ['DJANGO_SETTINGS_MODULE'] = 'settings.py'
>>> from django.conf import settings
>>> settings.configure()
>>> settings.DATABASES
{}
The settings.DATABASES should not be empty, so I know I haven't initialized the project correct. How would I do this in django2.1? I used to be able to do this easily using import settings; setup_environ(settings), but not anymore.
Note: I'm looking to be able to run the above from any directory. Here is an example from trying to import my project from tmp:
(V) david$ cd /tmp && python
>>> import django
>>> from django.conf import settings
>>> settings.configure()
>>> django.setup()
>>> from users.models import *
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'users'
The 2.1 docs state that you need to use setup() now. Relevant snippet:
import django
from django.conf import settings
settings.configure()
django.setup()
# Now this script or any imported module can use any part of Django it needs.
from myapp import models
Here is the full documentation.
You don't need to run settings.configure(). To properly initialize, you could do something like:
if __name__ == '__main__':
import sys, os
sys.path.append(django_root)
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
from django.conf import settings
print (settings.DATABASES)
I have had this problem and my solution (script) is the result of hundreds of Django sites over the course of a decade.
Some solutions can inhibit the use of settings and this is the best working version of a stand alone script for running Django that I have been able to compile. This is faster than the Django shell generally (no need to exit and re-enter).
Above suggestions will work 98% of the time. Still, consider reviewing (and commenting) on this for a more robust way to run Django scripts. I have hit the "Exceptions" about 1-2x every year for a while until I got this script fully debugged via many projects.
"""
scratch.py
----->note the no .py-----v
clear; echo 'import scratch' | python manage.py shell
"""
from django.conf import settings
from django.core.wsgi import get_wsgi_application # turns on app, access to DB
print(f'- - - - - - - - - - - - - - - - - - - - ')
print(f'settings: {settings}')
print('settings.DEBUG: {}'.format(settings.DEBUG))
# settings.DEBUG = True
# etc
if not settings.DEBUG:
# keeps this off prod, usually on git ignore as well
print('SETTINGS NOT IN DEBUG, set to:')
print(settings.DEBUG)
raise(RuntimeError('Can not be run on production or when debug is False'))
application = get_wsgi_application()
print('READY!')
print(f'- - - - - - - - - - - - - - - - - - - - ')
# App ready, code below. Add imports, calls, etc below here.
Additionally this script can be run in an infinite loop, such as for monitoring, debugging, and other day to day Django operations. At the ned of the file just add the following, possibly with a sleep() and logging.
while True:
pass
Call this script scratch.py from the same directory as manage.py. Call using the following syntax:
echo 'import scratch' | python manage.py shell
If you change the file name, then the import will need to be edited. This syntax loads the settings without needing to call settings.configure(). We leverage manage.py and that seems to solve the edge cases.
Note: This will NOT auto-reload on with changes.
Related
When running dev_appserver.py ., I get the following error when trying to access http://localhost:8080:
Traceback (most recent call last):
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 86, in run
self.finish_response()
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 128, in finish_response
self.write(data)
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions 2.7/lib/python2.7/wsgiref/handlers.py", line 204, in write
assert type(data) is StringType,"write() argument must be string"
AssertionError: write() argument must be string
I've searched and it seems to come back to my app.yaml file, per these links:
SO Question for GAE Assertion Error
Russian Question site with
same information
I'm just not sure how to go about debugging it. Below is my app.yaml file and my main.py file. I'm super new to the GAE platform and any help would be appreciated.
app.yaml file:
application: gqtimer
version: 1-56
runtime: python27
api_version: 1
threadsafe: false
handlers:
- url: /favicon.ico
static_files: static/images/favicon.ico
upload: static/images/favicon.ico
- url: /_ah/login_required
script: main.py
- url: /static
static_dir: static
- url: /player.*
script: main.py
login: required
- url: /stat.*
script: main.py
login: required
- url: .*
script: main.py
libraries:
- name: django
version: "1.11"
main.py file:
#!/usr/bin/env python
#
import config
import os
import sys
# Force sys.path to have our own directory first, so we can import from it.
sys.path.insert(0, config.APP_ROOT_DIR)
sys.path.insert(1, os.path.join(config.APP_ROOT_DIR, 'externals'))
os.environ["DJANGO_SETTINGS_MODULE"] = "settings"
import wsgiref.handlers
from google.appengine.ext import webapp
from google.appengine.dist import use_library
use_library('django', '1.2')
from handlers import error, timer, do_openid_login
def main():
application = webapp.WSGIApplication([('/', timer.ExportHandler),
('/_ah/login_required', do_openid_login.OpenIdLoginHandler),
('/player/([-\w]+)', timer.PlayerHandler),
('/player/([-\w]+)/archives', timer.ArchivesHandler),
('/stat/([-\w]+)', timer.StatHandler),
('/stat/([-\w]+)/delete', timer.StatDeleteHandler),
# If we make it this far then the page we are looking
# for does not exist
('/.*', error.Error404Handler),
],
debug=True)
wsgiref.handlers.CGIHandler().run(application)
if __name__ == '__main__':
main()
Indeed, your app.yaml file might not be properly mapping your application code. You need to:
take the application variable outside of the main() function so that it becomes a global variable in the main.py module (and maybe rename it to app as well - just to stay inline with the official convention and documentation examples)
replace the script: main.py statements from your app.yaml's handlers with main.application (or main.app if renamed as mentioned above) - this is the reference to the above-mentioned global variable. From the script row in the Handlers element table:
A script: directive must be a python import path, for example,
package.module.app that points to a WSGI application. The last component of a script: directive using a Python module path is
the name of a global variable in the module: that variable must be a
WSGI app, and is usually called app by convention.
I'd also recommend explicitly passing the app.yaml as argument to dev_appserver.py instead of the app's directory (. in your case) - occasionally the auto-detection doesn't behave as expected. It's also the only way to run multiple services and/or using a dispatch.yaml file for routing, so it's a good habit.
When i run the same code in django shell it works fine for me. but when I fire up the Python interpreter (Python 2) to check some things, I get the an error when I try importing - from django.contrib.auth.models import User
import os
import django
django.setup()
import smtplib
import sys
sys.path.insert(0, "/edx/app/edxapp/edx-platform/lms/djangoapps/courseware")
import dateutil.relativedelta
from django.conf import settings
from django.contrib.auth.models import User
from opaque_keys.edx.keys import CourseKey
from courseware.models import StudentModule
from datetime import datetime
def PCSurvey_email():
for student in StudentModule.objects.filter(module_type="PCSurvey"):
two_months_date = datetime.now().date() - dateutil.relativedelta.relativedelta(months=2)
created = student.created.date()
if created == two_months_date :
user_id = student.student_id
user_email = User.objects.get(id = user_id).email
FROM = "user#example.com"
TO = [user_email] # must be a list
SUBJECT = "Hello!"
TEXT = "This message was sent with Python's smtplib."
# Prepare actual message
message = """\
From: %s
To: %s
Subject: %s
%s
""" % (FROM, ", ".join(TO), SUBJECT, TEXT)
# Send the mail
server = smtplib.SMTP('outlook.net')
server.sendmail(FROM, TO, message)
server.quit()
#deleting the module
smod = StudentModule.objects.get(module_type="PCSurvey", course_id = student.course_id, student_id= student.student_id)
smod.delete()
The error that I get is
Traceback (most recent call last):
File "/edx/app/edxapp/edx-platform/lms/djangoapps/courseware/PCSurvey_email.py", line 4, in <module>
django.setup()
File "/usr/local/lib/python2.7/dist-packages/django/__init__.py", line 22, in setup
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 53, in __getattr__
self._setup(name)
File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 39, in _setup
% (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting LOGGING_CONFIG, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
Any help on this is very much appreciated.
Django needs to be setup, in order to run fine, when you run it through manage.py shell, manage.py takes care or setting it up for you, but doing it via a python script, needs manual setup.
You also seem to have used your defined models, to be able to import them into your python script, you need to add the path to the root folder of your project to the current python path.
Finally, you need to tell django where your settings file is (before setting up your django), in your manage.py file, you should have something like this :
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.settings")
Go make it a constant, name it * DEFAULT_SETTINGS_MODULE *, so you now have:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", DEFAULT_SETTINGS_MODULE)
Now you need to import the constant into your script and tell django (By setting an env var) where it should look for the settings file.
So in all:
import sys, os
sys.path.insert(0, "/path/to/parent/of/courseware") # /home/projects/my-djproj
from manage import DEFAULT_SETTINGS_MODULE
os.environ.setdefault("DJANGO_SETTINGS_MODULE", DEFAULT_SETTINGS_MODULE)
import django
django.setup()
This way you're setup up just fine.
For new users coming on this question.
If you don't want to setup django settings file manually as described wonderfully by #SpiXel
You can run python manage.py shell from the project directory which will basically link the settings file that django needs automatically.
When you open a shell, django automatically sets itself up.
With Python interpreter you have to do this manually:
import django
django.setup()
Just a simple solution all can try!
I tried to run my custom script inside shell only by using the following way:
python manage.py shell < /path_to_script
I have a Django app with a common directory structure:
project
---manage.py
---app
---__init__.py
---settings.py
---settings_secret.py
---a bunch of other files for the app itself
That settings_secret.py file contains variables from secrets.py which I do not want to send to github. For some reason, I cannot seem to import it into settings.py. First 5 lines of settings.py:
# Django settings for project.
DEBUG = True
TEMPLATE_DEBUG = DEBUG
import os
from settings_secret import *
Which fails with the partial stacktrace:
File "/foo/bar/project/app/settings.py", line 5, in <module> from settings_secret import *
ImportError: No module named 'settings_secret'
To debug, I created a test file inside /project/ like so:
from settings_secret import *
print(VARIABLE_FROM_SETTINGS_SECRET)
Which worked like a charm. So clearly, settings.py isn't looking in the right place for settings_secret. So where is it looking?
In settings.py, you should do: from .settings_secret import *
It works with the . because the proper syntax is supposed to be:
from app.settings_secret import *
Removing the app is shorthand coding, but the same principle applies. You need to tell Django to look for the directory (app), and then you are specifying which file in that directory to look for.
If you just did, from settings_secret import *, you are telling Django to look for the directory settings_secret, which doesn't exist.
Does that make sense to you?
I've been stuck all day on what seems to be a very silly import problem. From my Django project directory, I can import a module and run a function just fime:
(msg-gw)slashingweapon:~/msg-gw/www$ python
>>> import snpp
>>> snpp.config_url('snpp://server.name.com:1234?user=me&pass=whatever')
{'host': 'server.name.com', 'pass': 'whatever', 'port': 1234, 'user': 'me'}
But when I try to run my app, either through manage.py or by gunicorn, I get an attribute error:
(msg-gw)slashingweapon:~/msg-gw/www$ python manage.py runserver 8000
File "/home/slashingweapon/msg-gw/www/project/settings.py", line 26, in <module>
SNPP = snpp.config_url('snpp://server.name.com:1234?user=me&pass=whatever')
AttributeError: 'module' object has no attribute 'config_url'
The two relevant lines in my settings.py file are exactly what you would expect. Notice that I can import the module just fine, but the config_url() function isn't found.
import snpp
SNPP = snpp.config_url('snpp://server.name.com:1234?user=me&pass=whatever')
The directory layout is exactly what you would expect:
www
|
+-project
| +-__init__.py
| +-settings.py
| +-urls.py
| +-views.py
| +-wsgi.py
|
+-snpp
+-__init__.py
+-protocol.py
+-views.py
+-urls.py
The config_url() function is defined inside snpp/__init__.py
I have tried all kinds of things:
from snpp import config_url
move config_url to the file snpp/config and then import with
import snpp.confg
from snpp.config import config_url
from snpp import config and then invoke through config.config_url()
The __init__.py file is nothing special. It just lets you encode some server information as a string, so you can stick your SNPP config into the environment:
import urlparse
def config_url(urlStr):
config = {
'host':None,
'port':444,
'user':None,
'pass':None,
}
url = urlparse.urlparse(urlStr)
if url.scheme == 'snpp':
locParts = url.netloc.split(":")
config['host'] = locParts[0]
if len(locParts) > 1:
port = int(locParts[1])
if port > 0:
config['port'] = port
args = urlparse.parse_qs(url.query)
config['user'] = args.get('user', [None])[0]
config['pass'] = args.get('pass', [None])[0]
return config
I am using Python 2.7, django 1.5.1, and virtualenv.
Other parts of my project work well. When I print out the path in my browser, it looks correct. Importing snpp should not be a problem, since snpp is in the www directory:
/home/slashingweapon/msg-gw/www
/home/slashingweapon/msg-gw/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg
/home/slashingweapon/msg-gw/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg
/home/slashingweapon/msg-gw/lib/python2.7/site-packages/django_json_rpc-0.6.2-py2.7.egg
/home/slashingweapon/msg-gw/lib/python27.zip
/home/slashingweapon/msg-gw/lib/python2.7
... etc ...
It doesn't matter if the snpp module is in my INSTALLED_APPS list or not. I get the same result.
Solved
With the help of SO denizens, I found the problem.
I had refactored my application by moving some reusable code pieces from the project directory to the new snpp directory. When I did that, I neglected to move or delete the *.pyc files.
The value of snpp.__file__ was:
/home/slashingweapon/msg-gw/www/project/snpp.pyc
instead of the expected:
/home/slashingweapon/msg-gw/www/snpp/__init__.pyc
During the import process, Python was looking in project/ before snpp/ and finding an old snpp.pyc file. It would import the old pyc file and be satisfied, thus ignoring the entire snpp/ dir.
Had I been a little sharper (or a little more experienced with Python) I might have noticed that I was getting some strange import behavior in general whenever I tried to import anything from snpp/. It should have occurred to me that the whole module was wonky, and not just the one function I was trying to use at the moment.
Check what exactly is being imported by using
snpp.__file__
right after import snpp statement.
Actually import might not be from the path you are expecting to see.
OK, I have the following directory structure (it's a django project):
-> project
--> app
and within the app folder, there is a scraper.py file which needs to reference a class defined within models.py
I'm trying to do the following:
import urllib2
import os
import sys
import time
import datetime
import re
import BeautifulSoup
sys.path.append('/home/userspace/Development/')
os.environ['DJANGO_SETTINGS_MODULE'] = 'project.settings'
from project.app.models import ClassName
and this code just isn't working. I get an error of:
Traceback (most recent call last):
File "scraper.py", line 14, in
from project.app.models import ClassName
ImportError: No module named project.app.models
This code above used to work, but broke somewhere along the line and I'm extremely confused as to why I'm having problems. On SnowLeopard using python2.5.
import sys
sys.path.append ('/path/to/the/project')
from django.core.management import setup_environ
import settings
setup_environ(settings)
from app.models import MyModel
Whoa whoa whoa. You should never ever have to put your project name in any of your app code. You should be able to reuse app code across multiple projects with no changes. Pinax does this really well and I highly recommend checking it out for a lot of django best practices.
The worst thing you could do here is to hard code your absolute path into your app or settings. You shouldn't do this because it will break during deployment unless you do some import local_settings hacking.
If you have to access the project root directory, try what pinax has in settings.py...
import os.path
PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))
The thing is that it looks like you are trying to access the models module within the same app and this is waaay easier.
To import models.py inside scraper.py in the same directory just use import models or import models as app_models if you already have something named models in scraper.py (django.db.models for instance). Are you familiar with Python module conventions?
However, the best way is probably to stick with the django idiom, from ... import ... statement:
from app import models
If this doesn't work automatically, then something is wrong in your settings.py.
You don't indicate if project is located in /home/userspace/Development/. I'll assume that it is.
Make sure there's an (empty by default) file named __init__.py in project and another one in app.
EDIT: Next thing to try: Fire up the Python command line in the script's directory and try the following:
import project
import project.app as app
import project.app.models as models
models.__dict__.keys()
Do they all work? If so, what is the last line's output? If not, which dies first?