We Keep Coding

When working with a settings directory, where should I import it?

I have recently started working with a settings directory as described in the Two Scoops of Django book. It contains the following files
local.py
staging.py
production.py
test.py
__init__.py
To be able to use the different setting files on the server I have adapted my django.fcgi script to import the settings module. It works very smoothly.
How do I do the same on my local machine on which I use runserver, however? I have set the DJANGO_SETTINGS_MODULE and I have adapted the manage.py file to
#!/usr/bin/env python
from django.core.management import execute_manager
import imp
import sys
sys.path.insert(0, '/home/user/don.joey/projects/a_project/a_project_site/settings')
import settings
try:
imp.find_module('settings') # Assumed to be in the same directory.
except ImportError:
import sys
sys.stderr.write("Error: Can't find the file 'settings.py' in the directory containing %r. It appears you've customized things.\nYou'll have to run django-admin.py, passing it your settings module.\n" % __file__)
sys.exit(1)
if __name__ == "__main__":
execute_manager(settings)
It works fine.
How can I make django-admin.py find these settings? I do not want to manually edit django-admin.py because it is part of my virtualenv and it will does thus regularly be updated.
Update
I have set the following: export DJANGO_SETTINGS_MODULE=settings.local.

You need to set two things
DJANGO_SETTINGS_MODULE and
PYTHONPATH so that the settings module can be found
The way Two Scoops of Django suggests setting up a project named blah you would have the following directory structure:
- blah_project/
- blah/
- manage.py
- blah/
- ...
- settings/
- __init__.py
- local.py
- production.py
- ...
Run the following (assuming a bash environment):
export DJANGO_SETTINGS_MODULE=blah.settings.local
export PYTHONPATH=/full/path/to/blah_project/blah
As long as django-admin.py is on your path (and it should be if django is installed and activated within your venv), you should be able to run:
django-admin.py runserver

How does django manage.py put a project package on sys.path

I read the django doc and some SO posts to know the differences between manage.py and django-admin.py.
They all say:
manage.py is automatically created in each Django project. manage.py
is a thin wrapper around django-admin.py that takes care of two things
for you before delegating to django-admin.py:
It puts your project’s package on sys.path.
It sets the DJANGO_SETTINGS_MODULE environment variable so that it points to
your project’s settings.py file.
So I checked the scource code of these 2 files(latest version, so it the doc).
Then I am confused. manage.py does the second thing: sets the DJANGO_SETTINGS_MODULE environment variable. Besides that, I really can not find any differences between these 2 scripts.
[django-admin.py]
#!/usr/bin/env python
from django.core import management
if __name__ == "__main__":
management.execute_from_command_line()
[manage.py]
#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "{{ project_name }}.settings")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
Why? Is the django documentation out of date? Or I missed something here? And where is the code that puts the project’s package on sys.path?

The sys.path is updated here using handle_default_options(options) statement located here. The execution path is as follows:
execute_from_command_line(argv) (your manage.py)
utility.execute() here
handle_default_options(options) here
The same method is used by Command class used as base class for management commands.

From what I can see it's not ./manage.py who "puts your project’s package on sys.path."
The doc says:
sys.path is initialized from these locations:
The directory containing the input script (or the current directory when no file is specified).
PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
The installation-dependent default.
The installation-dependent default must be what site.py adds. But it can be disabled with -S switch. Now then when I run this script (1.py):
import sys
print(sys.path)
I get:
$ python -S ../1.py # to show that it's not current directory that is added
['/path/to/the/script/directory', '/usr/lib/python35.zip', '/usr/lib/python3.5/', \
'/usr/lib/python3.5/plat-linux', '/usr/lib/python3.5/lib-dynload']
So, when you run django-admin, /path/to/env/bin/django-admin will be the first on the sys.path. And when ./manage.py, /path/to/project.
As such, one can probably say that ./manage.py "puts your project’s package on sys.path," but indirectly, by being at the root of your project.

How do I get all files for my django app into one folder?

I currently have a Django app named reserve. In the folder named "reserve" I have most of the content of my app (views.py, urls.py, models.py, templates folder). However, I have a folder outside of "reserve" named "booking" that has only my settings.py. I tried consolidating by putting the settings.py in "booking" into "reserve" but I seem to be getting an error. Any advice on how to have only one folder with all contents?
The error I get is: ImportError: Could not import settings 'booking.settings' (Is it on sys.path?): No module named booking.settings
overall project folder
booking folder
settings.py
init.py
reserve folder (the app)
views.py
admin.py
models.py
...

what version of django are you using? what command are you running ? assuming that you are just trying to run a command using manage.py you could just chnage manage.py to reflect this if you are going to permanently keep this project structure
#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "your_project.settings")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
you should change os.environ.setdefault to the correct directory for your settings.py

I'm assuming you have manage.py at the overal project directory? I'm also assuming you want to just runserver.
It seems you moved the settings.py file outside the overal project directory. If you do this, you'll have to specify the settings file path when you do runserver, or you can also export the django settings module to point to your new settings path (using dot notation). In this case, I think it would be (in the command line):
export DJANGO_SETTINGS_MODULE=<overal-project-folder-name>.booking.settings
python manage.py runserver

django manage.py settings default

I have a settings.py file and a dev_settings.py file that I use to override some values for dev purposes. Everytime I run the ./manage.py command, I have to specify --settings=whatever.local_settings. This becomes very tedious to do every time and I am trying to find a way to force manage.py to load my dev_settings.py file every by default so that I don't have to type that long argument every time I want to run a command.
I have tried setting DJANGO_SETTINGS_MODULE, however, it appears that manage.py overrides this option.
Is it possible to make this happen or am I doomed to always specify that argument?

manage.py sets path to settings for you, that's why it's ignoring DJANGO_SETTINGS_MODULE (it's basically just script that wraps around django-admin.py).
There are 2 easy ways to fix your problem:
set DJANGO_SETTINGS_MODULE and use django-admin.py to run all commands instead of manage.py. This is even better if you use vitualenv.
copy manage.py and name it local.py (that's the name in my case) and rename all settings mentions to dev_settings.
For example:
#!/usr/bin/env python
from django.core.management import execute_manager
import imp
try:
import settings_local
except ImportError:
import sys
sys.stderr.write("Error: Can't find the file 'settings_local.py' in the directory containing %r. It appears you've customized things.\nYou'll have to run django-admin.py, passing it your settings module.\n" % __file__)
sys.exit(1)
if __name__ == "__main__":
execute_manager(settings_local)
You can run all commands by ./local.py now.

The way this is typically done is you have settings.py with all settings that are common between environments (things like INSTALLED_APPS, etc.). Then, you have something like settings_local.py, that defines settings particular to the environment in context. You then import settings_local.py in settings.py.
# settings.py
from settings_local import *
settings.py gets added to your source code repository, but settings_local.py does not. (However, you would normally add something like settings_local.py.example to the repo.)
When you first move your app over to production, for example, you pull down the code base from your repo. You then copy settings_local.py.example to settings_local.py and make any necessary environment specific changes.
You then have separate settings_local.py files in each environment, and it all just works.

You can make a bash alias by adding these lines to your .bash_profile file:
mymanage()
{
python manage.py $1 --settings=settings_debug
}
alias mng=mymanage
Then when you run this command:
mng runserver
settings_debug.py file will be used for settings.

You can use django-admin.py with that environment variable. Commands are interchangeable, only django-admin.py doesn't override the variable you're trying to use.

If a settings file is common to all installation, you can just import it e.g.
from settings_local import *
but usually settings_local are changed and tweaked per installation and as my installation script directly copy files to target sites (without worrying what is local what is not), which mean settings_local may get overwritten, to avoid that I just keep settings_local in parent folder of the installation target and manually import it in settings.py e.g.
local_settings_file = os.path.join(prevFolder, "settings_local.py")
if os.path.exists(local_settings_file):
execfile(local_settings_file)

Python - packages and settings file

I have a python package that needs to pull in settings from my project directory, here is how my project is currently structured:
~/Project/bin/mypackage
- package files
~/Project/myproject/
- project files
- start.py
- settings.py
I guess it's similar to how Django is structured, you have a settings.py file in your project directory that is somehow referenced by the Django system package in your Python directory.
So, if I am running start.py like so:
python ~/Project/myproject/start.py
..and start.py imports and utilizes the mypackage package, is there any way I can reference the settings.py file local to start.py from within the package? Would I have to load the settings file in start.py and store the values in a global? Does anyone know how this is possible?

The way I see it you have several options:
look for settings and import them either from the current working directory or as determined from environment variables. This is the "django way" using DJANGO_SETTINGS_MODULE and PYTHONPATH. This is nice and magical when it works and inconvenient when it doesn't such as in your case when you are running from a different directory.
rely on module search path which will include the directory of the calling package. Nice and simple but the settings will vary based on the caller. For example all you need in mypackage is:
import settings
pass in settings as a variable

The directory containing the script that was used to invoke the python interpreter is added to the PYTHONPATH. It is available at sys.path[0]. See http://docs.python.org/library/sys.html#sys.path
This means that settings should be available from mypackage.mymodule simply by import settings.
However, I would consider handling the loading of settings in start.py and structuring your app so that a settings object (perhaps just a dict) is passed to it.