I have problems with importing correctly a module on appengine. My app generally uses django with app-engine-patch, but this part is task queues using only the webapp framework.
I need to import django settings for the app to work properly.
My script starts with:
import os
import sys
sys.path.append('common/')
# Force Django to reload its settings.
from django.conf import settings
settings._target = None
# Must set this env var before importing any part of Django
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
I always get this error, or something related:
<type 'exceptions.ImportError'>: No module named ragendja.settings_pre
because the settings.py file starts with
from ragendja.settings_pre import *
I think I need to add ragendja to sys.path again but I had several tries that didn't work.
Here is my directory:
project/
app.yaml
setting.py
common/
appenginepatch/
ragendja/
setting_pre.py
myapp/
script.py
Is it only a sys.path problem and how do I need to modify it with the correct syntax?
Thanks
App engine patch manipulates sys.path internally. Background tasks bypass that code, so your path will not be ready for Django calls. You have two choices:
Fix the paths manually. The app engine documentation (see the sub-section called "Handling import path manipulation") suggests factoring the path manipulation code into a module that can be imported by your task script.
Eliminate dependencies on django code, if possible. If you can write your task to be pure python and/or google api calls, you're good to go. In your case, this might mean refactoring your settings code.
Why not:
sys.path.append('common/appenginepatch')
since the ragendja is in this directory?
Related
Recently I discovered Dynaconf which is a nice configuration management package that integrates nicely with Flask and Django. The Django app is running wonderfully with Dynaconf. However the Celery app that my App depends on to run background tasks is not.
Here is the code for the configuration of the Celery app that was working before using Dynaconf:
from celery import Celery
app = Celery('KillerApp')
app.config_from_object('django.conf:settings', namespace='CELERY')
It seems that I need to change 'django.conf:settings' to something else. Any ideas?
You can pass in a string representing a module to import, or just pass in the configuration object directly; see the Celery.config_from_object() method documentation.
You'll have a module that sets up the Dynaconf() instance, e.g. if you have a package named acme_project with a config.py file in it with
from dynaconf import Dynaconf
settings = Dynaconf(
settings_files=['settings.toml', '.secrets.toml'],
)
then you can import acme_project.config and find the settings object there. You can either import that object or let Celery do that by using 'acme_project.config:settings' as the value you pass to app.config_from_object(). The namespace argument tells Celery to expect all settings to be prefixed with CELERY_, exactly like the way this works with Django. Use this if you plan to use the Dynaconf-managed settings to configure multiple components, not just Celery.
E.g., if you used:
app.config_from_object('acme_project.config:settings', namespace='CELERY')
then your settings.toml or settings.yaml or whatever file format you picked would need to use CELERY_ as a prefix for all the settings.
If you are using the Django plugin for Dynaconf then you are able to just use the django.conf:settings directly as Dynaconf patches the django settings object.
If you still have problems I recommend opening an issue on dynaconf repo and try using the settings of your application directly.
Example, if you have an app called foo your DJANGO_SETTINGS_MODULE might be foo.settings then you can use for celery:
app.config_from_object('foo.settings:settings')
I have read numerous SO questions and blogs. I am trying to structure my flask application. The current structure of my application is the following:
application
run_server.py
/config
__init__.py
production.py
staging.py
development.py
/app
__init__.py
/site
__init__.py
views.py
Now, inside app/__init__.py I want to access the config based on my environment( dev, staging, production).
from ..config import config
I am getting this error:
ValueError: Attempted relative import beyond toplevel package
I have tried using -m switch.
I have also tried to set PYTHONPATH as my root directory to tell interpreter what is top level package.
I think I am missing some fundamental in relative imports.
Try using absolute import. IMHO it makes things easier to grok
from __future__ import absolute_import
from application.config import production
This is absolute because you are specifying the exact path you are importing from which reduces ambiguity.
Also, you are missing __init__.py in the application folder
If you are running your application through run_server.py then there is no need of relative import in app/__init__.py. You can simply say,
from config import <production/staging/development>
This is because, when your interpreter interprets run_server.py, at the line say, from app import <something>, it will fetch app/__init__.py content and try to execute them at toplevel i.e. from directory application.
Now, assume you are running from ..config import config at toplevel. Obviously, it will throw an error.
Assume you have configin application/config/__init__.py.
You also need __init__.py under application directory, if not, then the application/app is your top level package, you can not access application/config.
I have a django application structured like this...
app_foo
__init__.py
urls.py
views.py
models.py
bar_app
__init__.py
...
bar_app...
By using distutils, I can get the application to install into the python path under the "app_foo" module name.
However, any of the code inside of the "bar_app" python files which refers to things inside the django app relatively does not work when executed from the python path. For example,
from bar_app.views import stuff
I know that I can go through the app and change all the references to be absolute. For example,
from app_foo.bar_app.views import stuff
My question:
Is there anyway I can get all of the apps inside "app_foo" to also be on the python path?
Conceptually this would be similar to saying from app_foo import * for the entire path.
You can do
from .bar_app.views import stuff
http://docs.python.org/whatsnew/2.5.html#pep-328-absolute-and-relative-imports
I'm starting on my first large django project and have realized that the default directory structure isn't going to work well.
I saw this question and answer and decided to implement something similar.
Large Django application layout
What do I need to do to make something like this work? How can I change where django looks for apps/modules?
Python works automatically with deep directory structures. That's porbably you didn't find any instructions on how to do it.Here are some instructions on how to get classes and models to work.
If you want to have a module in folder yourproject/apps/firstapp you can just add it to INSTALLED_APPS by adding line 'apps.firstapp',. You will have to add a __init__.py file to each of the directories so that they are recognized as python packages.
When you import classes you can simply use from yourproject.apps.firstapp.filename import yourclass.
You should also make sure all template directories are listed in TEMPLATE_DIRS.
I have two methods to handle this; one for production and one for development.
In development:
Append the path to your apps in your settings.py. In my case, I have my main project in ~/hg/django/project/ and my apps are in ~/hg/django/apps/. So I use:
if DEVEL:
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'apps'))
I also use the production method on my development box. This method has a knock on effect of letting me roll back to the most recent production version by simply commenting out the line path insertion in the settings.py.
In production:
I install the apps using distutils so that I can control deployment per server, and multiple projects running on each server can all access them. You can read about the distutils setup scripts here. Then, to install, you simply:
./setup.py install
if you add:
import os
PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))
to your settings .py and the following to your manage.py:
sys.path.insert(0, join(settings.PROJECT_ROOT, "apps"))
sys.path.insert(0, join(settings.PROJECT_ROOT, "lib"))
then you can include them in your installed apps just as you would if they were in your project root:
INSTALLED_APPS = (
#local apps
'myapp',
#local libs
'piston_dev',
)
this allows you a bit more freedom to move apps around without having to redeclare imports and such.
I created a sub-directory of my Django project called bin where I want to put all command-line run Python scripts. Some of these scripts need to import my Django project settings.py file that is in a parent directory of bin.
How can I import the settings.py file from a sub-directory of the project?
The code that I use in my command-line script to set into the "Django context" of the project is:
from django.core.management import setup_environ
import settings
setup_environ(settings)
This works fine if the script is in the root directory of my project.
I tried the following two hacks to import the settings.py file and then setup the project:
import os
os.chdir("..")
import sys
sys.path = [str(sys.path[0]) + "/../"] + sys.path
The cruel hack can import settings.py, but then I get the error:
project_module = __import__(project_name, {}, {}, [''])
ValueError: Empty module name
I think your approach may be over-complicating something that Django 1.x provides for you. As long as your project is in your python path, you can set the environment variable DJANGO_SETTINGS_MODULE at the top of your script like so:
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.settings'
In your command line script where you need to read your settings, simply import the settings module from 'django.conf' as you would do in your application code:
from django.conf import settings
And presto, you have your settings and a Django-enabled environment for your script.
I personally prefer to set my DJANGO_SETTINGS_MODULE using '/usr/bin/env' in a bash script called 'proj_env' so I don't have to repeat it
#!/bin/bash
proj_env="DJANGO_SETTINGS_MODULE=myproject.settings"
/usr/bin/env $proj_env ${*}
With this, now I can run any python script with my Django application in context:
proj_env python -m 'myproject.bin.myscript'
If you use virtualenv, this also gives you a good place to source the activate script.
etc. etc.
This is going one level up from your question, but probably the best solution here is to implement your scripts as custom manage.py (django-admin.py) commands. This gives you all of Django's functionality (including settings) for free with no ugly path-hacking, as well as command-line niceties like options parsing. I've never seen a good reason to write Django-related command-line scripts any other way.
Add the parent directory to your path:
import sys
sys.path.append('../')
import settings
Update from comments:
Don't forget the __init__.py file in
the directory that has your
settings.py – S.Lott
Let's say your project directory is /opt/someProject/`
This has files like:
manage.py
Now you have you may have your sub directory anywhere, does not matter.
Eg. subdirectory could be like:
/opt/someproject/dir1/dir2
Now for you to import your project settings inside /opt/someProject/dir1/dir2
You need to set your PYTHONPATH variable
export PYTHONPATH=/opt/someproject/
Now to import modules from bin
from someproject import bin