We Keep Coding

How to execute external script in the Django environment

I am trying to execute an external snippet for debugging and terminal-like purposes in the environment the Django console uses so it can connect to the db, etc.
Basically, I am just using it for the same reason one would fiddle with the console but I am using longer snippets to output some formatted information so it is handy to have that code in an actual file manipulated with an IDE.
An answer said you could do that by executing python manage.py shell < snippet.py but I did not see a successfull result. And although no errors are reported, I am not getting the excepted output, but only a series of >>> prompts.
So how can I do this?
By the way, I am using PyCharm, in case this IDE has a shorthand way of doing this or any special tool.

I would say creating a new Custom management command is the best way to achieve this goal.
But you can run your script in a django environment. I use this sometimes to run a oneoff script or some simple tests.
You have to set the environment variable DJANGO_SETTINGS_MODULE to your settings module and then you have to call django.setup()
I copied these lines from the manage.py script, you have to set the correct settings module!
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings.local")
django.setup()
Here is a simple template script which I use sometimes:
# -*- coding: utf-8 -*-
import os
import django
# you have to set the correct path to you settings module
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings.local")
django.setup()
from project.apps.bla.models import MyModel
def run():
# do the work
m = MyModel.objects.get(pk=1)
if __name__ == '__main__':
run()
It is important to note that all project imports must be placed after calling django.setup().

Many times you need scripts to play with db or more. But you need to the the django way i.e. the ORM and play with every thing that your project has.
You can checkout https://github.com/django-extensions/django-extensions
You create a scripts folder in your project home and write down a method run . the app provide you a way to run those scripts easily.
How to do it.
Step 1 - Install the package.
Step 2 - Create a scripts folder with init.py in project root, and add 'django_extensions' in your applications
Step 3- Create a file send_email_to_users.py in scripts folder. Simple Example.
from project.models import MyModel
from django.contrib.auth.models import User
from project.utils import send_email
def run():
results = MyModel.objects.all()
for res in results:
send_mail(res.email)
Now from command line you can run
python manage.py runscript send_mail_to_users

If its just a one off script,
import django
django.setup()
from myapp.models.import MyModel
You need to have your environmental variable set up, so its easiest to run it from the IDE (make it part of the same project, right click on teh file and there should be a run option).
If you are looking for something to run on a production environment I would create a management command as suggested by DanEEStar.

django import local settings from the server

In my home directory, I have a folder called local. In it, there are files called __init__.py and local_settings.py. My django app is in a completely different directory. When the app is NOT running in DEBUG mode, I want it to load the local_settings.py file. How can this be acheived? I read the below:
Import a module from a relative path
Importing files from different folder in Python
http://docs.python.org/tutorial/modules.html
Basically, those tutorials are allowing to import from another directory, but what about a completely different working tree? I don't want to keep doing .., .., .. etc. Is there a way to goto the home directory?
I tried the following code:
import os, sys
os.chdir(os.path.join(os.getenv("HOME"), 'local'))
from local_settings import *
But i keep seeing errors in my apache error.log for it...

os.chdir just affects the current working directory, which has nothing whatsoever to do with where Python imports modules from.
What you need to do is to add the the local directory to the Pythonpath. You can either do this from the shell by modifying PYTHONPATH, or from inside Python by modifying sys.path:
import sys
import os
sys.path.append(os.path.expanduser("~/local"))
import local_settings

In response to your concerns about source control, you can just set the source control to ignore that file, and/or have a symlink installed as part of your deploy script to link the file on the os into another. I do both , though not in django. but it's a trivial task.
your deploy layout could look like this:
/current ( symlink to /releases/v3 )
/settings/local_settings.py
/releases/v1
/releases/v2
/releases/v3
and a task runs as part of your deploy:
cd /current
ln -s /settings/local_settings.py local_settings.py
if you're deploying with fab or capistrano, it's a few lines of configuration. i'm sure you could do this with puppet/chef simply too.

running python module from django shell (manage.py)

I'm try to populate data into my db and I'd like to use the django API to do so. I've written a python module that allows me to this very easily.
However, within the django shell (manage.py), it doesn't seem possible to run my "populate.py" file.
Is it possible to run my populate.py from the django shell? Or alternatively, should I instead be running a regular python terminal and importing the necessary django components? Either way, I would greatly appreciate some pointers.

You can also consider using the loaddata command or create your own command

Here is my standard wrapper for running django stuff from the command line. This is also useful if you have cron scripts you want to run.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os, sys
# If this script lives in a child directory of the main project directory
# uncomment the following 2 lines:
#dir_parent = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
#sys.path.insert(0, dir_parent)
from django.core.management import setup_environ
import settings
setup_environ(settings)
from django.db import transaction
# import whatever you like from app.models or whatever
# do random stuff.
# If you are *changing* database records,
# wrap your database code either in:
#transaction.commit_manually
def foo():
... stuff
transaction.commit()
# OR end you script with:
transaction.commit_unless_managed()
Update for comment:
The variable mentioned above is not file, it is __file__ (i.e. 2 underscores file and 2 more underscores). This is always set by the Python interpreter to the full path of the file it occurs in. As the comment says, if this script lives in a child directory of the main directory, the directory-of-the-directory of the filename gives you the directory name of the main project directory, and adding that to sys.path will make sure that all of your imports work correctly. If the script lives in the main directory and you execute it from there, then you don't need to do that.

Try django-extensions it has a runscript command you put in a scripts directory.
Oddly the command seems to be missing from the main documentation but if you google django-extensions runscript you will find examples and documentation.

Sys.path modification or more complex issue?

I have problems with importing correctly a module on appengine. My app generally uses django with app-engine-patch, but this part is task queues using only the webapp framework.
I need to import django settings for the app to work properly.
My script starts with:
import os
import sys
sys.path.append('common/')
# Force Django to reload its settings.
from django.conf import settings
settings._target = None
# Must set this env var before importing any part of Django
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
I always get this error, or something related:
<type 'exceptions.ImportError'>: No module named ragendja.settings_pre
because the settings.py file starts with
from ragendja.settings_pre import *
I think I need to add ragendja to sys.path again but I had several tries that didn't work.
Here is my directory:
project/
app.yaml
setting.py
common/
appenginepatch/
ragendja/
setting_pre.py
myapp/
script.py
Is it only a sys.path problem and how do I need to modify it with the correct syntax?
Thanks

App engine patch manipulates sys.path internally. Background tasks bypass that code, so your path will not be ready for Django calls. You have two choices:
Fix the paths manually. The app engine documentation (see the sub-section called "Handling import path manipulation") suggests factoring the path manipulation code into a module that can be imported by your task script.
Eliminate dependencies on django code, if possible. If you can write your task to be pure python and/or google api calls, you're good to go. In your case, this might mean refactoring your settings code.

Why not:
sys.path.append('common/appenginepatch')
since the ragendja is in this directory?

How to import a Django project settings.py Python file from a sub-directory?

I created a sub-directory of my Django project called bin where I want to put all command-line run Python scripts. Some of these scripts need to import my Django project settings.py file that is in a parent directory of bin.
How can I import the settings.py file from a sub-directory of the project?
The code that I use in my command-line script to set into the "Django context" of the project is:
from django.core.management import setup_environ
import settings
setup_environ(settings)
This works fine if the script is in the root directory of my project.
I tried the following two hacks to import the settings.py file and then setup the project:
import os
os.chdir("..")
import sys
sys.path = [str(sys.path[0]) + "/../"] + sys.path
The cruel hack can import settings.py, but then I get the error:
project_module = __import__(project_name, {}, {}, [''])
ValueError: Empty module name

I think your approach may be over-complicating something that Django 1.x provides for you. As long as your project is in your python path, you can set the environment variable DJANGO_SETTINGS_MODULE at the top of your script like so:
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.settings'
In your command line script where you need to read your settings, simply import the settings module from 'django.conf' as you would do in your application code:
from django.conf import settings
And presto, you have your settings and a Django-enabled environment for your script.
I personally prefer to set my DJANGO_SETTINGS_MODULE using '/usr/bin/env' in a bash script called 'proj_env' so I don't have to repeat it
#!/bin/bash
proj_env="DJANGO_SETTINGS_MODULE=myproject.settings"
/usr/bin/env $proj_env ${*}
With this, now I can run any python script with my Django application in context:
proj_env python -m 'myproject.bin.myscript'
If you use virtualenv, this also gives you a good place to source the activate script.
etc. etc.

This is going one level up from your question, but probably the best solution here is to implement your scripts as custom manage.py (django-admin.py) commands. This gives you all of Django's functionality (including settings) for free with no ugly path-hacking, as well as command-line niceties like options parsing. I've never seen a good reason to write Django-related command-line scripts any other way.

Add the parent directory to your path:
import sys
sys.path.append('../')
import settings
Update from comments:
Don't forget the __init__.py file in
the directory that has your
settings.py – S.Lott

Let's say your project directory is /opt/someProject/`
This has files like:
manage.py
Now you have you may have your sub directory anywhere, does not matter.
Eg. subdirectory could be like:
/opt/someproject/dir1/dir2
Now for you to import your project settings inside /opt/someProject/dir1/dir2
You need to set your PYTHONPATH variable
export PYTHONPATH=/opt/someproject/
Now to import modules from bin
from someproject import bin