coming from php, the namespaces are always well defined in php, typically using psr-4 from composer
https://getcomposer.org/doc/04-schema.md#psr-4
e.g. you have a folder
src/Foo/Bar/Baz
you then define in the composer.json file that
src/Foo is where the namespace Foo starts
thereafter, all subfolders are by convention, a new sub namespace e.g.
src/Foo/Bar/Baz/MyClass.php turns into
Foo.Bar.Baz.MyClass
Say I want to drop a utility python class into my django project, that I want to use in all django "apps"
Where would I drop it, and how to properly define the namespace? what is the transparent way to understand namespaces in python?
From this document:
https://docs.djangoproject.com/en/2.2/ref/applications/#projects-and-applications
Is this snippet:
A project’s root directory (the one that contains manage.py) is usually the container for all of a project’s applications which aren’t installed separately.
You can create whatever hierarchy you chose under that directory for utilities, which can conform to python's package system:
https://docs.python.org/3/tutorial/modules.html#packages
In Python, there is a search path for modules. It can be initialized from the command line with the PYTHONPATH environment variable and accessed programmatically via sys.path.
By default, the directory of the script you use to start the Python interpreter is the first entry in the search path.
Any module or package on the python path "starts a new namespace", to use your wording. Let's assume your project called mysite has an app called polls and has this structure:
mysite
├── manage.py
├── mysite
│ ├── __init__.py
│ ├── settings.py
│ ├── urls.py
│ ├── utils.py
│ └── wsgi.py
├── polls
│ ├── __init__.py
│ ├── models.py
│ ├── utils.py
...
If you start it with python manage.py runserver, the packages mysite and polls are available.
I'd say a good place for project-wide utils module is the project package mysite. So you do import mysite.utils in any of your apps. If it grows beyond a single file, you can turn it into a subpackage, so instead of a utils.py, you have a utils directory containing an __init__.py file.
The above structure clutters the global namespace a bit. So some people prefer a structure where all your apps are subpackages of your project package, like this:
mysite
├── manage.py
├── mysite
│ ├── __init__.py
│ ├── settings.py
│ ├── urls.py
│ ├── utils.py
│ ├── wsgi.py
│ └── apps
│ └── polls
│ ├── __init__.py
│ ├── models.py
│ ├── utils.py
...
Related
I would like to import modules from within the following folder structure by referencing the sub-module app.
├── src
│ ├── __init__.py
│ └── app
│ ├── __init__.py
│ ├── getters
│ │ ├── company_1.py
│ │ └── company_2.py
│ ├── parsers
│ │ ├── company_1.py
│ │ └── company_2.py
│ └── writers
│ ├── company_1.py
│ └── company_2.py
Here is an example of what I want to do:
# src/app/writers/company_1.py
import app.parsers.company_1 as parser_1
def write_data(event, context):
...
but I get the error ModuleNotFoundError: No module named 'app'. I don't understand why this shouldn't work.
NOTE
Why am I including the seemingly redundant sub-folder app? The side story here is that the functions writers/company_*.py will be implemented as Lambda functions on AWS, and in order to get all the necessary dependencies, I need to upload the contents of the src folder to AWS but need to have a top-level folder with a known name so that I can use it for absolute imports. Without app the contents of src are uploaded to a folder with an autogenerated name which I cannot reference.
I first tried relative imports, but received an attempted relative import with no known parent package error, so now I am going for absolute imports, inspo from here.
Did you try having in __init__.py inside your app folder ?
module
|
|--module
| |
| |--__init__.py
| |--module.py
You can add this in the code ,
import os, sys
sys.path.append(os.path.join(os.path.dirname(__file__), "source"))
Just replace source with the folder where you have saved your modules.
In this case replace source with app
I'd like to figure out the cleanest and preferably self contained way to use my packages in scripts that are in a different directory to the package itself.
The example problem is as follows:
The modules in lib need to both be imported, and run as a script.
My project directory is as below and I'm having two issues:
in lib/api.py, I want to read in data_files/key.txt correctly when api.py is called or imported
in testing_script.py I want to import and use lib/get_data.py
I can't seem to find a clean way to do this, does this mean my project is structured in a non-pythonic way?
Thanks for the help.
my-project-git
├── LICENSE
├── README.md
├─── my_project
│ ├── data_files
│ │ ├── key.txt
│ │ ├── mappings.csv
│ ├── lib
│ │ ├── __init__.py
│ │ ├── api.py
│ │ └── get_data.py
│ └── test
│ ├── __init__.py
│ └── testing_script.py
├── requirements.txt
└── setup.py
As far as I know, there's isn't a pythonic way to structure your project.
This is what Kenneth Reitz recommended in 2013 and it's how I use it: https://www.kennethreitz.org/essays/repository-structure-and-python.
README.rst
LICENSE
setup.py
requirements.txt
sample/__init__.py
sample/core.py
sample/helpers.py
docs/conf.py
docs/index.rst
tests/test_basic.py
tests/test_advanced.py
Inside sample (my_project in your case) you can separate into categories as you like. E.g. Utils (common functions), Database (read, write), View (user commands), etc. It depends on your project.
As for calling the modules at the same level, you should define them in the __init__ file of the top hierarchy module which is sample in this case.
For example:
__init__ in _my_project
from sample.core import a_Class
from sample.core import a_function
from sample.core import anything
then from /test/test_basic.py you do:
from sample import a_Class
# or import sample
a = a_Class() # use the class from core.py
# or a = sample.a_Class()
Take a look at the sample module repository: https://github.com/navdeep-G/samplemod
In order to manage all my python paths for my project and have them available as soon as I start python interpreter, I created a project.pth in the project home directory having relative paths in it,
Actually, to be read, I need to do a site.addsitedir(my_project_home_dir) each time I start the interpreter.
I tried setting PYTHONPATH or create a .pth in site-packages pointing to my project home directory, but project.pth is still not read automatically when I start the interpreter.
The only thing that works is to put my project.pth in site-packages, but by doing that, I have to transform my project relative paths to absolute paths.
So it there a master .pth file where I can specify my project home directory so I can have my project.pth located in that directory to be read automatically ?
While I know this doesn't specifically answer your question, might I suggest a different approach? The use of virtual environments (venv).
Given a project (I used pyscaffold) to create.
- Create a virtual environment at the highest level (python -m venv venv).
- Add PYTHONPATH to your venv\Scripts\activate.bat file.
- PYTHONPATH should contain at least three entries (path\to\MyTestProject; pat\to\MyTestProject\src; path\to\MyTestProject\tests). Activate your virtual environment from cmd by "venv\Scripts\activate.bat". - From within your modules, import from src (import src.mytestproject.package1.module1)
You will have your paths available to you each time you activate your virtual environment - and, you will have all the goodness associated with a virtual environment as a bonus.
MyTestProject
├── AUTHORS.rst
├── CHANGELOG.rst
├── docs
│ ├── authors.rst
│ ├── changelog.rst
│ ├── conf.py
│ ├── index.rst
│ ├── license.rst
│ ├── Makefile
│ └── _static
├── LICENSE.txt
├── README.rst
├── requirements.txt
├── setup.cfg
├── setup.py
├── src
│ └── mytestproject
│ ├── __init__.py
│ ├── package1
│ │ ├── __init__.py
│ │ └── module1.py
│ ├── package2
│ │ ├── __init__.py
│ │ └── module2.py
│ └── skeleton.py
└── tests
├── conftest.py
└── test_skeleton.py
Finally I found 2 ways :
1) First, note that .pth files also accept lines beginning with 'import' and execute them. So the solution was to create project.pth inside site-packages importing a module in my project home dir that do actually the site.addsitedir()
2) Other possibility : use the environment variable PYTHONSTARTUP : it will execute any module you want at python startup. You can specify a module that do the site.addsitedir() or even directly add paths into sys.path
Here is my project folder structure.
<pre>
front
├── __init__.py
├── __init__.pyc
├── manage.py
├── middleware.py
├── settings.py
├── news
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── admin.py
│ ├── admin_views.py
│ ├── models.py
│ ├── search_indexes.py
│ ├── search_sites.py
│ ├── utils.py
│ ├── views.py
</pre>
After I run
./manage.py runserver
then I visit http://127.0.0.1:8000/
It gives me error:
No module named front
This is caused by the following line in file views.py under news folder.
from front import settings
So the front folder is one level up to the views.py file. How I import the settings from one level up folder?
Thanks!
Either way this is wrong, whenever you're importing your own settings you should be importing from django.conf
from django.conf import settings
This isn't a module, its an object that does magic to import any settings you have set in your DJANGO_SETTINGS_MODULE
From the docs:
Also note that your code should not import from either global_settings or your own settings file. django.conf.settings abstracts the concepts of default settings and site-specific settings; it presents a single interface. It also decouples the code that uses settings from the location of your settings.
I'm working through the official Django tutorial. At the end of part two they have you customize the admin template just a bit (changing some heading text). I think I've done everything right, but apparently I haven't because the admin site looks exactly the same after running syncdb and restarting the server. My project directory (excluding the virtualenv part) looks like this:
mysite
├── manage.py
├── mysite
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── settings.py
│ ├── settings.pyc
│ ├── urls.py
│ ├── urls.pyc
│ ├── wsgi.py
│ └── wsgi.pyc
├── polls
│ ├── admin.py
│ ├── admin.pyc
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── models.py
│ ├── models.pyc
│ ├── tests.py
│ └── views.py
└── templates
└── admin
└── base_site.html
In settings.py I have these lines added to make the project use the template in templates/admin:
import os
PROJECT_PATH = os.path.realpath(os.path.dirname(__file__))
later:
TEMPLATE_DIRS = (os.path.join(PROJECT_PATH, 'templates'),)
I've tried several combinations of slashes and variations on assigning the value to PROJECT_PATH and TEMPLATE_DIRS, but haven't gotten anything to work. Each time I verified that it was actually making a filepath string (with a simple print statement).
To try to see what was going on I edited the original Django base_site.html it its admin folder and that changed my site, so it's obviously still using that. I've read several SO posts, and I don't really know what to do anymore.
Oh, and if it's relevant: Python 2.7.3 and Django 1.4.3
First thing I would try is deleting your *.pyc files they often contain stale information which sometimes can cause issues likes this.
If that isn't the case I would then double check your PROJECT_PATH which I believe is the issue here.
dirname gets you the containing directory. so if that line is in the settings.py it will return /path/to/inner/mysite the one inside the main mysite and since there is not templates directory in your inner mysite it wont work.
what you need to do is this.
PROJECT_PATH = os.path.realpath(os.path.dirname(os.path.dirname(__file__)))
Which will return you the outer mysite path when combined with templates it will return you the correct path.
Everything should then work.