I'm try to populate data into my db and I'd like to use the django API to do so. I've written a python module that allows me to this very easily.
However, within the django shell (manage.py), it doesn't seem possible to run my "populate.py" file.
Is it possible to run my populate.py from the django shell? Or alternatively, should I instead be running a regular python terminal and importing the necessary django components? Either way, I would greatly appreciate some pointers.
You can also consider using the loaddata command or create your own command
Here is my standard wrapper for running django stuff from the command line. This is also useful if you have cron scripts you want to run.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os, sys
# If this script lives in a child directory of the main project directory
# uncomment the following 2 lines:
#dir_parent = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
#sys.path.insert(0, dir_parent)
from django.core.management import setup_environ
import settings
setup_environ(settings)
from django.db import transaction
# import whatever you like from app.models or whatever
# do random stuff.
# If you are *changing* database records,
# wrap your database code either in:
#transaction.commit_manually
def foo():
... stuff
transaction.commit()
# OR end you script with:
transaction.commit_unless_managed()
Update for comment:
The variable mentioned above is not file, it is __file__ (i.e. 2 underscores file and 2 more underscores). This is always set by the Python interpreter to the full path of the file it occurs in. As the comment says, if this script lives in a child directory of the main directory, the directory-of-the-directory of the filename gives you the directory name of the main project directory, and adding that to sys.path will make sure that all of your imports work correctly. If the script lives in the main directory and you execute it from there, then you don't need to do that.
Try django-extensions it has a runscript command you put in a scripts directory.
Oddly the command seems to be missing from the main documentation but if you google django-extensions runscript you will find examples and documentation.
Related
I am trying to execute an external snippet for debugging and terminal-like purposes in the environment the Django console uses so it can connect to the db, etc.
Basically, I am just using it for the same reason one would fiddle with the console but I am using longer snippets to output some formatted information so it is handy to have that code in an actual file manipulated with an IDE.
An answer said you could do that by executing python manage.py shell < snippet.py but I did not see a successfull result. And although no errors are reported, I am not getting the excepted output, but only a series of >>> prompts.
So how can I do this?
By the way, I am using PyCharm, in case this IDE has a shorthand way of doing this or any special tool.
I would say creating a new Custom management command is the best way to achieve this goal.
But you can run your script in a django environment. I use this sometimes to run a oneoff script or some simple tests.
You have to set the environment variable DJANGO_SETTINGS_MODULE to your settings module and then you have to call django.setup()
I copied these lines from the manage.py script, you have to set the correct settings module!
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings.local")
django.setup()
Here is a simple template script which I use sometimes:
# -*- coding: utf-8 -*-
import os
import django
# you have to set the correct path to you settings module
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings.local")
django.setup()
from project.apps.bla.models import MyModel
def run():
# do the work
m = MyModel.objects.get(pk=1)
if __name__ == '__main__':
run()
It is important to note that all project imports must be placed after calling django.setup().
Many times you need scripts to play with db or more. But you need to the the django way i.e. the ORM and play with every thing that your project has.
You can checkout https://github.com/django-extensions/django-extensions
You create a scripts folder in your project home and write down a method run . the app provide you a way to run those scripts easily.
How to do it.
Step 1 - Install the package.
Step 2 - Create a scripts folder with init.py in project root, and add 'django_extensions' in your applications
Step 3- Create a file send_email_to_users.py in scripts folder. Simple Example.
from project.models import MyModel
from django.contrib.auth.models import User
from project.utils import send_email
def run():
results = MyModel.objects.all()
for res in results:
send_mail(res.email)
Now from command line you can run
python manage.py runscript send_mail_to_users
If its just a one off script,
import django
django.setup()
from myapp.models.import MyModel
You need to have your environmental variable set up, so its easiest to run it from the IDE (make it part of the same project, right click on teh file and there should be a run option).
If you are looking for something to run on a production environment I would create a management command as suggested by DanEEStar.
Use case:
I'm attempting to create a migration script that will create a table (which will make a many to many relationship) and then populate that table with foreign keys from the database.
To do this I'm attempting to load my flask applications ORM models so that I may use them in my migration script.
directory structure
/home/alord/git/my_project/ # project directory
/home/alord/git/my_project/alembic_testing # migration directory
/home/alord/git/my_project/my_project #requirement of flask modules
/home/alord/git/my_project/runserver.py # script that starts development server
/home/alord/git/my_project/alembic.ini # alembic configuration
/home/alord/git/my_project/development_config.py # development server configuration
if I run alembic upgrade +1
import os
print os.getcwd()
print os.path.dirname(os.path.realpath(__file__))
in
/home/alord/git/my_project/alembic_testing/env.py
I get
/home/alord/git/my_project
/home/alord/git/my_project/alembic_testing
What I'm doing
I'm placing
import os.path
sys.path.append(
os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir)))
import my_project
In my upgrade script and then running the command
alembic upgrade +1
and I'm recieving a stack trace with the error
ImportError: No module named my_project
Without this attempted import the table upgrade and downgrade functions run without error.
What I expect
I would like to be able to import the package, and more importantly *my_project.models* so that I can use the ORM to populate my new table.
Note:
I can't use relative package inclusion because I'm running alembic rather than calling python. As such, the python -m argument isn't useful to me.
Fix your script_location in alembic.ini to point to the right place, then just run alembic from the root of your repository (/home/alord/git/), and you shouldn't need any sys.path hijinks to be able to import your app.
Short answer:
trying to inport from a revision file in alembic puts you two directories down instead of one.
Long answer:
So I ended up printing off my sys.path to see what directories were actually included. When I noticed that the path that was included was
/home/alord/git/my_project/alembic_testing
instead of
/home/alord/git/my_project/alembic_testing
I made my path.append go up one more directory using,
import sys
import os.path
#hacky solution to get to the root application directory.
sys.path.append(
os.path.abspath(
os.path.join(
os.path.join(
os.path.dirname(__file__),
os.path.pardir),
os.path.pardir)))
In my home directory, I have a folder called local. In it, there are files called __init__.py and local_settings.py. My django app is in a completely different directory. When the app is NOT running in DEBUG mode, I want it to load the local_settings.py file. How can this be acheived? I read the below:
Import a module from a relative path
Importing files from different folder in Python
http://docs.python.org/tutorial/modules.html
Basically, those tutorials are allowing to import from another directory, but what about a completely different working tree? I don't want to keep doing .., .., .. etc. Is there a way to goto the home directory?
I tried the following code:
import os, sys
os.chdir(os.path.join(os.getenv("HOME"), 'local'))
from local_settings import *
But i keep seeing errors in my apache error.log for it...
os.chdir just affects the current working directory, which has nothing whatsoever to do with where Python imports modules from.
What you need to do is to add the the local directory to the Pythonpath. You can either do this from the shell by modifying PYTHONPATH, or from inside Python by modifying sys.path:
import sys
import os
sys.path.append(os.path.expanduser("~/local"))
import local_settings
In response to your concerns about source control, you can just set the source control to ignore that file, and/or have a symlink installed as part of your deploy script to link the file on the os into another. I do both , though not in django. but it's a trivial task.
your deploy layout could look like this:
/current ( symlink to /releases/v3 )
/settings/local_settings.py
/releases/v1
/releases/v2
/releases/v3
and a task runs as part of your deploy:
cd /current
ln -s /settings/local_settings.py local_settings.py
if you're deploying with fab or capistrano, it's a few lines of configuration. i'm sure you could do this with puppet/chef simply too.
I have a settings.py file and a dev_settings.py file that I use to override some values for dev purposes. Everytime I run the ./manage.py command, I have to specify --settings=whatever.local_settings. This becomes very tedious to do every time and I am trying to find a way to force manage.py to load my dev_settings.py file every by default so that I don't have to type that long argument every time I want to run a command.
I have tried setting DJANGO_SETTINGS_MODULE, however, it appears that manage.py overrides this option.
Is it possible to make this happen or am I doomed to always specify that argument?
manage.py sets path to settings for you, that's why it's ignoring DJANGO_SETTINGS_MODULE (it's basically just script that wraps around django-admin.py).
There are 2 easy ways to fix your problem:
set DJANGO_SETTINGS_MODULE and use django-admin.py to run all commands instead of manage.py. This is even better if you use vitualenv.
copy manage.py and name it local.py (that's the name in my case) and rename all settings mentions to dev_settings.
For example:
#!/usr/bin/env python
from django.core.management import execute_manager
import imp
try:
import settings_local
except ImportError:
import sys
sys.stderr.write("Error: Can't find the file 'settings_local.py' in the directory containing %r. It appears you've customized things.\nYou'll have to run django-admin.py, passing it your settings module.\n" % __file__)
sys.exit(1)
if __name__ == "__main__":
execute_manager(settings_local)
You can run all commands by ./local.py now.
The way this is typically done is you have settings.py with all settings that are common between environments (things like INSTALLED_APPS, etc.). Then, you have something like settings_local.py, that defines settings particular to the environment in context. You then import settings_local.py in settings.py.
# settings.py
from settings_local import *
settings.py gets added to your source code repository, but settings_local.py does not. (However, you would normally add something like settings_local.py.example to the repo.)
When you first move your app over to production, for example, you pull down the code base from your repo. You then copy settings_local.py.example to settings_local.py and make any necessary environment specific changes.
You then have separate settings_local.py files in each environment, and it all just works.
You can make a bash alias by adding these lines to your .bash_profile file:
mymanage()
{
python manage.py $1 --settings=settings_debug
}
alias mng=mymanage
Then when you run this command:
mng runserver
settings_debug.py file will be used for settings.
You can use django-admin.py with that environment variable. Commands are interchangeable, only django-admin.py doesn't override the variable you're trying to use.
If a settings file is common to all installation, you can just import it e.g.
from settings_local import *
but usually settings_local are changed and tweaked per installation and as my installation script directly copy files to target sites (without worrying what is local what is not), which mean settings_local may get overwritten, to avoid that I just keep settings_local in parent folder of the installation target and manually import it in settings.py e.g.
local_settings_file = os.path.join(prevFolder, "settings_local.py")
if os.path.exists(local_settings_file):
execfile(local_settings_file)
I created a sub-directory of my Django project called bin where I want to put all command-line run Python scripts. Some of these scripts need to import my Django project settings.py file that is in a parent directory of bin.
How can I import the settings.py file from a sub-directory of the project?
The code that I use in my command-line script to set into the "Django context" of the project is:
from django.core.management import setup_environ
import settings
setup_environ(settings)
This works fine if the script is in the root directory of my project.
I tried the following two hacks to import the settings.py file and then setup the project:
import os
os.chdir("..")
import sys
sys.path = [str(sys.path[0]) + "/../"] + sys.path
The cruel hack can import settings.py, but then I get the error:
project_module = __import__(project_name, {}, {}, [''])
ValueError: Empty module name
I think your approach may be over-complicating something that Django 1.x provides for you. As long as your project is in your python path, you can set the environment variable DJANGO_SETTINGS_MODULE at the top of your script like so:
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.settings'
In your command line script where you need to read your settings, simply import the settings module from 'django.conf' as you would do in your application code:
from django.conf import settings
And presto, you have your settings and a Django-enabled environment for your script.
I personally prefer to set my DJANGO_SETTINGS_MODULE using '/usr/bin/env' in a bash script called 'proj_env' so I don't have to repeat it
#!/bin/bash
proj_env="DJANGO_SETTINGS_MODULE=myproject.settings"
/usr/bin/env $proj_env ${*}
With this, now I can run any python script with my Django application in context:
proj_env python -m 'myproject.bin.myscript'
If you use virtualenv, this also gives you a good place to source the activate script.
etc. etc.
This is going one level up from your question, but probably the best solution here is to implement your scripts as custom manage.py (django-admin.py) commands. This gives you all of Django's functionality (including settings) for free with no ugly path-hacking, as well as command-line niceties like options parsing. I've never seen a good reason to write Django-related command-line scripts any other way.
Add the parent directory to your path:
import sys
sys.path.append('../')
import settings
Update from comments:
Don't forget the __init__.py file in
the directory that has your
settings.py – S.Lott
Let's say your project directory is /opt/someProject/`
This has files like:
manage.py
Now you have you may have your sub directory anywhere, does not matter.
Eg. subdirectory could be like:
/opt/someproject/dir1/dir2
Now for you to import your project settings inside /opt/someProject/dir1/dir2
You need to set your PYTHONPATH variable
export PYTHONPATH=/opt/someproject/
Now to import modules from bin
from someproject import bin