I have root dir with following structure:
.
|-- myproject
| |-- a
| | |-- foo.c
| | |-- foo.h
| | `-- __init__.py
| |-- b
| | |-- bar.c
| | |-- bar.h
| | `-- __init__.py
| |-- c
| | |-- __init__.py
| | `-- qwe.py
| `-- __init__.py
`-- setup.py
I want to import the external modules as:
from myproject import a
from myproject import b
or:
import myproject.a
And the question is how the ....
My setup.py is like:
from setuptools import *
setup(
name = 'myproject',
.
.
.
ext_package = 'myproject',
ext_modules = [Extension("myproject.a", ["myproject/a/foo.c"])]
All my init.py are empty.
When:
python setup.py install
The package is installed but cannot refer to a, b and c.
Any idea how to do that?
==========================
Actually i can refer to foo and bar, but cannot see or use any method....???
Related
I am currently working on my first 'bigger' project where I want to separate some code. For me as a python newbie, it's pretty hard to get the imports right. I tried to use relative and absolute imports, but I can't get around all the different import errors. The project is currently structured like:
|- scripts
| |-- etl
| | |-- etl.py
| |-- model
| | |--model.py
| |-- ps
| | |--ps.py
| |-- shared
| | |-- classes.py
| | |-- enums.py
| | |-- functions.py
| |-- main.py
How can I import
classes.py and enums.py within the shared package
classes.py, enums.py and functions.py into etl.py, model.py, ps.py and main.py
classes from etl.py, model.py, ps.py into main.py
I searched the whole internet and could not make it work, but sadly could not make it so far (it could be because of the project structure as well. If there is a better structure which will make it work, I will change the structure!)
Thanks for your help!
I am having some trouble trying to run a script after building an EGG file for my pkgs using Python3/setuptools. I am able to run the script OK if I use zip_safe=False in setup.py. But I get a ModuleNotFoundError otherwise. What am I doing wrong? I would prefer not to use zip_safe=False option.
My project is structured as follows:
.
|-- python_pkgs
| |-- __init__.py
| |-- acap
| | |-- __init__.py
| | |-- acap.py
| | |-- cfg
| | |-- pch_acap.pl
| | |-- spf.py
| | `-- tvpv.py
| |-- parsers
| | |-- STIL1999ModelBuilderSemantics.py
| | |-- STIL1999Parser.py
| | |-- __init__.py
| | |-- grammars
| | `-- test
| `-- utils
| |-- __init__.py
| |-- misc.py
| |-- nbq.py
| |-- splitter.py
| `-- util.py
|-- run_acap.py
`-- setup.py
The contents of run_acap.py:
#!/nfs/sc/disks/mpe_tvpv_032/sw_tools/anaconda3/bin/python3.7 -B
import sys
print(sys.path)
from python_pkgs.acap.acap import main
if __name__ == "__main__":
main()
The contents of setup.py file:
from setuptools import setup, find_namespace_packages
setup(
name='python_pkgs',
version='0.1',
packages=find_namespace_packages(include=['python_pkgs.*'],
exclude=['python_pkgs.parsers',
'python_pkgs.parsers.*']),
scripts=['run_acap.py']
)
To install I say: python setup.py install --prefix a path
The install directory contents look like this:
$ tree
.
|-- bin
| `-- run_acap.py
`-- lib
`-- python3.7
`-- site-packages
|-- easy-install.pth
|-- python_pkgs-0.1-py3.7.egg
`-- site.py
I notice the contents of bin/run_acap.py is a bit different:
#!/nfs/sc/disks/mpe_tvpv_032/sw_tools/anaconda3/bin/python -B
# EASY-INSTALL-SCRIPT: 'python-pkgs==0.1','run_acap.py'
__requires__ = 'python-pkgs==0.1'
__import__('pkg_resources').run_script('python-pkgs==0.1', 'run_acap.py')
Now to run this script, I say:
setenv PYTHONPATH <a path>/lib/python3.7/site-packages
<a path>/bin/run_acap.py --help
$ /nfs/sc/disks/mpe_tvpv_032/pgupta6/releases/acap/ww49.2/bin/run_acap.py --help
['/nfs/sc/disks/mpe_tvpv_032/pgupta6/releases/acap/ww49.2/bin', '/nfs/sc/disks/mpe_tvpv_032/pgupta6/releases/acap/ww49.2/lib/python3.7/site-packages', '/nfs/sc/disks/mpe_tvpv_032/pgupta6/releases/acap/ww49.2/lib/python3.7/site-packages/python_pkgs-0.1-py3.7.egg', '/nfs/sc/disks/mpe_tvpv_032/sw_tools/anaconda3/lib/python37.zip', '/nfs/sc/disks/mpe_tvpv_032/sw_tools/anaconda3/lib/python3.7', '/nfs/sc/disks/mpe_tvpv_032/sw_tools/anaconda3/lib/python3.7/lib-dynload', '/nfs/sc/disks/mpe_tvpv_032/sw_tools/anaconda3/lib/python3.7/site-packages']
Traceback (most recent call last):
File "/nfs/sc/disks/mpe_tvpv_032/pgupta6/releases/acap/ww49.2/bin/run_acap.py", line 4, in <module>
__import__('pkg_resources').run_script('python-pkgs==0.1', 'run_acap.py')
File "/nfs/sc/disks/mpe_tvpv_032/sw_tools/anaconda3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 666, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/nfs/sc/disks/mpe_tvpv_032/sw_tools/anaconda3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1469, in run_script
exec(script_code, namespace, namespace)
File "/nfs/sc/disks/mpe_tvpv_032/pgupta6/releases/acap/ww49.2/lib/python3.7/site-packages/python_pkgs-0.1-py3.7.egg/EGG-INFO/scripts/run_acap.py", line 11, in <module>
ModuleNotFoundError: No module named 'python_pkgs'
But if I open up a python interpreter and just type import python_pkgs the import works OK. I am not sure what I am doing wrong in my setup.py file (or somewhere else). How to fix this error? Thanks.
I have a django project with srtucture like this:
main_project
----main_project
<------libs
<<---------exceptions.py
----project_a
----project_b
In the views.py of project_a I am trying to import a folder named libs of main_project and a file from libs called exceptions.py, but I am getting the error
ImportError: No module named libs.exceptions
My code is :
from main_project.libs.exceptions import (
APIException400,
APIException405,
APIException403,
exception_handler_dispatcher,
)
Can someone tell me what am I missing here?
With reference to https://stackoverflow.com/a/31407131/5080347 answer I even tried :
from main_project.main_project.libs.exceptions import (
APIException400,
APIException405,
APIException403,
exception_handler_dispatcher,
)
but doesn't work.
It seems like you forgot to add __init__.py to libs directory.
The __init__.py is used to initialize Python packages. Check the documentation to better understand how things are working.
Your structure should looks as follow:
project/
|
|-- __init__.py
|
|-- module_a/
| |-- __init__.py
| |
| |-- file1.py
| |
| |-- file2.py
|
|-- module_b/
| |
| |-- __init__.py
| |
| |-- file1.py
| |
| |-- submodule/
| | |--__init__.py
| | |
| | |-- file1.py
When you import using from main_project.libs.exceptions, python expects that main_project is the package and libs and exceptions are submodules. So there must be a file named __init__.py in those directories. The init.py files are required to make Python treat the directories as containing packages. For further reading please refer to here.
The question of what the directory structure of a Python project has been asked a number of times on Stack Overflow (e.g. here, here and here)
And many answers are given. But one thing that doesn't seem to be clear in any of those answers is why some projects have repetitive directories. For example, in this article which is often cited, the suggested layout is:
<root>/
|-- Twisted/
| |-- __init__.py
| |-- README
| |-- setup.py
| |-- twisted/
| | |-- __init__.py
| | |-- main.py
| | |-- test/
| | | |-- __init__.py
| | | |-- test_main.py
| | | |-- test_other.py
| | |-- bin/
| | | |-- myprogram
In this example, /Twisted/twisted/main.py is the main file
But then on the other hand you have advice like this:
Many developers are structuring their repositories poorly due to the new bundled application templates.
<root>/
|-- samplesite/
| |-- manage.py
| |-- samplesite/
| | |-- settings.py
| | |-- wsgi.py
| | |-- sampleapp/
| | |-- models.py
Dont do this.
Repetitive paths are confusing for both your tools and your developers. Unnecessary nesting doesnt help anybody. Let's do it properly:
<root>/
|-- manage.py
|-- samplesite/
| |-- settings.py
| |-- wsgi.py
| |-- sampleapp/
| |-- models.py
My question is not necessarily "which way is better?", since there may be pros or cons to each way.
Instead, my question is, if I go with the more simplified second style, what will I lose? Is there a good reason to have a /<root>/Twisted/twisted/main.py directory structure rather than just /<root>/twisted/main.py ? Does it make it easier somehow to share my application or make the import process smoother? Something else?
I believe the most common layout of python projects is something like this:
project/
|-- setup.py
|-- bin/
|-- docs/ ...
|-- examples/ ...
|-- package/
|-- __init__.py
|-- module1.py
|-- module2.py
|-- subpackage/ ...
|-- tests/ ...
Where the project is the name of the project and the package is the name of the top level import, for example scikits-learn and sklearn. The package has everything that python should be able to import, and you import using the package name. For example from package import thing or from package.module1 import thing. The project has the package and any supporting things like docs, examples and installation scripts. Notice that there is typically no __init__.py in project because project is not python importable. It is common for the project and package to have the same name, but not required.
Those two documents are closer than you think. Both Interesting Things, Largely Python and Twisted Related (your first example) and the django-admin startproject docs assume you are outside of the project repository while Structuring Your Project (your second example) assumes you are inside the repository. To quote, "Well, they go to their bare and fresh repository and run the following...".
The django docs state that if you run
django-admin.py start-project samplesite
both the project directory and project package will be named and the project directory will be created in the current
working directory
The command creates the project directory for you, so you certainly shouldn't be inside of an already-created project directory when you run it. The docs go on to say
django-admin startproject myproject /Users/jezdez/Code/myproject_repo
If the optional destination is provided, Django will use that existing
directory as the project directory
Now, suppose you were already in /Users/jezdez/Code/myproject_repo. Then you would do
django-admin startproject myproject .
to create the project package in the current directory. Voila, you've got the second author's example! The author was really just telling you to avoid the first form if you are creating your repo before running the command.
So, lets redraw your directory structure. In the first example, <root> is the directory where you hold your dev repos. Twisted is the directory with your repo. (As an aside, that directory shouldn't have an __init__.py because its not a package directory). In the final example, <root> is the repo directory itself. Supposing I named that directory DjangoExample, then the structure would be
<root>
|-- Twisted/
| |-- __init__.py
| |-- README
| |-- setup.py
| |-- twisted/
| | |-- __init__.py
| | |-- main.py
| | |-- test/
| | | |-- __init__.py
| | | |-- test_main.py
| | | |-- test_other.py
| | |-- bin/
| | | |-- myprogram
|
|-- DjangoExample/
| |-- manage.py
| |-- samplesite/
| | |-- settings.py
| | |-- wsgi.py
| | |-- sampleapp/
| | |-- models.py
As for other differences, the django app has to follow the django framework rules whereas twised follows the more generic python package rules.
I have written an application with python (2.7). The structure looks like:
kent$ tree myApp
myApp
|-- foo.py
|-- gui
| |-- g1.py
| |-- g2.py
| |-- g3.py
| `-- __init__.py
|-- icons
| |-- a.png
| `-- e.png
|-- logic
| |-- __init__.py
| |-- l1
| | |-- __init__.py
| | |-- la.py
| | `-- lc.py
| |-- l2
| | |-- __init__.py
| | |-- ld.py
| | `-- lf.py
| |-- logic1.py
| |-- logic2.py
| `-- logic3.py
|-- myApp.py
`-- resources
|-- x.data
`-- z.data
Now I am about to write a setup.py to distribute my application. I am new to this. After reading the py doc and doing some testing. a few questions come up:
how can I (or should I) package my root package (myApp) under /lib/python/site-package ?
since in my py file, I reference resources/icons by relative path. for example, in foo.py there could be icons/a.png and in gui/g1.py there could be ../icons/e.png and so on
how can I package icons and resources directory?
It seems that package_data and data_files won't copy the two directories to right place.
is this the right way?
packages = [''],
package_dir = {'': ''},
package_data= {'': ['icons/*.*', 'resources/*.*']},
after install, my files will be:
/usr/lib/python2.7/site-packages/icons/*.png
/usr/lib/python2.7/site-packages/resources/*.data
/usr/lib/python2.7/site-packages/gui/...
/usr/lib/python2.7/site-packages/logic/...
Is there problem of my application structure?
should those resources/icons/whatever files go to certain python package, not under the project root? so that in setup.py I can use package_data to copy them to right place.
from ez_setup import use_setuptools
use_setuptools()
from setuptools import setup
setup(name="somename",
version="1.0",
description="description string",
long_description="""\
long description
""",
author="Foo",
author_email="bar#gmail.com",
url="http://nowhere.com",
include_package_data=True,
license="MIT",
packages=["gui", "logic"],
package_dir={
"gui": "myApp/gui",
"logic": "myApp/logic",
},
classifiers=[
"Development Status :: 5 - Production/Stable",
"Topic :: Utilities",
"License :: OSI Approved :: MIT License"
],
data_files=[
('/path/to/resources', ['resources/x.data', 'resources/y.data']),
('/path/to/icons', ['myApp/icons/a.ico', 'myApp/icons/e.ico'])
]
)