I have a project called build-your-own-computer with a structure that looks like this:
build-your-own-computer
├── computer
│ ├── arithmetic
│ ├── __init__.py
│ ├── logic
│ └── memory
├── README.md
├── setup.py
├── setup.py~
├── solutions
│ ├── arithmetic
│ │ ├── half_adder.py
│ │ ├── __init__.py
│ │ └── __init__.py~
│ ├── __init__.py
│ ├── __init__.py~
│ ├── logic
│ │ ├── _and.py
│ │ ├── __init__.py
│ │ ├── __init__.py~
│ │ ├── _not.py
│ │ ├── _or.py
│ │ └── xor.py
│ └── memory
│ └── __init__.py
└── tests
├── arithmetic
│ └── test_half_adder.py
├── logic
│ ├── test_and.py
│ ├── test_not.py
│ ├── test_or.py
│ └── test_xor.py
└── memory
My goal is to be able to install this project/package using pip and then to use it from anywhere on my system. I want to import the packages/modules like this:
from byoc.solutions.logic import _and
from byoc.computer.arithmetic import half_adder
As I understand it, build-your-own-computer itself can be considered a package, build-your-own-computer\computer a subpackage, and build-your-own-computer\computer\logic\ a sub-sub-package. All .py files besides init's and setup.py are modules. Is this correct, and does the import scheme above correspond with this project structure?
All of the __init__.py files are empty.
setup.py contains the following:
from setuptools import setup, find_packages
setup(
name='byoc',
packages=find_packages()
)
When I install this using pip and then try importing the submodules, I run into problems:
>>> from byoc.solutions.logic import *
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'byoc.solutions'
I can import byoc without errors.
I'm obviously doing something wrong, but what?
You can check here the suggested layout by setuptools. Also here is a good guide.
Here is my suggestion on your layout:
build-your-own-computer
├── README.md
├── setup.py
├── setup.cfg
├── src
│ └── byoc
│ ├── __init__.py
│ ├── computer
│ │ ├── arithmetic
│ │ ├── __init__.py
│ │ ├── logic
│ │ └── memory
│ └── solutions
│ ├── arithmetic
│ │ ├── half_adder.py
│ │ └── __init__.py
│ ├── __init__.py
│ ├── logic
│ │ ├── _and.py
│ │ ├── __init__.py
│ │ ├── __init__.py~
│ │ ├── _not.py
│ │ ├── _or.py
│ │ └── xor.py
│ └── memory
│ └── __init__.py
└── tests
├── arithmetic
│ └── test_half_adder.py
├── logic
│ ├── test_and.py
│ ├── test_not.py
│ ├── test_or.py
│ └── test_xor.py
└── memory
And then your setup.py should be:
from setuptools import setup, find_packages
setup(
name='byoc',
packages=find_packages('src/')
package_dir={'': 'src/'},
)
This way you should then be able to import it anywhere, e.g.:
from byoc.solutions import logic
Related
I have a project like this:
├── CHANGES.txt
├── LICENSE
├── MANIFEST.in
...
├── docs
│ └── index.rst
├── negar
│ ├── Negar.py
│ ├── Virastar.py
│ ├── Virastar.pyc
│ ├── __init__.py
│ ├── data
│ │ ├── __init__.py
│ │ └── untouchable.dat
│ ├── gui.py
│ ├── gui.pyc
│ ├── i18n
│ │ ├── fa_IR.qm
│ │ └── fa_IR.ts
│ └── negar.pro
├── setup.py
...
and inside that my file Virastar.py need some data from data.untouchable.dat. it works fine until I install the project with this setup.py:
setup(
...
include_package_data=True,
packages = find_packages() + ['negar'],
package_dir={'negar': 'negar'},
package_data={'negar': ['data/*.dat']},
entry_points={
'console_scripts': [
'negar = negar.Negar:main',
],
},
...
)
after that when I start my program and when it needs that data file it return this error:
IOError: [Errno 2] No such file or directory: 'data/untochable.dat'
even in my egg-info sources I can't find any data file:
...
negar/Negar.py
negar/Virastar.py
negar/__init__.py
negar/gui.py
negar/data/__init__.py
have I missed something here?
Thank you all.
EDIT: Do I have to add any special thing in init.py?
and I have to add this: I used untouchable.dat just like this:
f = codecs.open('data/untouchable.dat', encoding="utf-8")
I used data_files
data_files = [('', ['negar/data/untouchable.dat'])],
The first problem is that I didn't import my data file into the package with MANIFEST.in file. I imported it like this:
include negar/data/*.dat
After that my data file already imported with my package install. but because I had mistakes in open my data files, python couldn't find it. this question helped me to find the right way
Python Access Data in Package Subdirectory and now I use something like this:
import os
this_dir, this_filename = os.path.split(__file__)
DATA_PATH = os.path.join(this_dir, "data", "data.txt")
print open(DATA_PATH).read()
Maybe try:
package_data={'negar/data': ['data/*.dat']},
A solution that does not require any of:
MANIFEST.in
include_package_data=True
package_dir={}
and neither the __init__.py file in the folder!
Having the project like:
├── CHANGES.txt
├── LICENSE
...
├── negar
│ ├── Negar.py
│ ├── Virastar.py
│ ├── Virastar.pyc
│ ├── __init__.py
│ ├── data
│ │ ├── __init__.py
│ │ └── untouchable.dat
│ ├── subfolder
│ │ ├── __init__.py
│ │ ├── data_NOT_included
│ │ │ └── garbage.toml
│ │ └── data_with_no_init_file
│ │ ├── config.ini
│ │ └── options.yml
│ ├── gui.py
│ ├── gui.pyc
│ ├── i18n
│ │ ├── fa_IR.qm
│ │ └── fa_IR.ts
│ └── negar.pro
├── setup.py
...
The setup file:
setup(
...
packages = find_packages()
package_data= {
# all .dat files at any package depth
'': ['**/*.dat'],
# into the data folder (being into a module) but w/o the init file
'negar.subfolder': [ '**/*.ini', '**/*.yml', ]
},
entry_points={
'console_scripts': [
'negar = negar.Negar:main',
],
},
...
)
This worked great in my case and I avoided maintaining an additional MANIFEST.in file.
caveat: the "negar.subfolder" has to be a python module.
To the access the file:
from importlib.resources import files
config = files('negar.subfolder').joinpath('data_with_no_init_file').joinpath('config.ini').read_text()
from accessing-data-files-at-runtime
I have this structure for my project:
├── Dockerfile
├── app
│ ├── __init__.py
│ ├── __pycache__
│ ├── config
│ ├── database
│ ├── logging.py
│ ├── main.py
│ ├── routers
│ ├── services
│ ├── static
│ ├── templates
│ ├── utils
│ └── worker
├── k6.js
├── poetry.lock
├── prestart.sh
├── pyproject.toml
├── pytest.ini
└── run.py
Inside app, I have this worker folder that I also open as a kind of separate project.
├── __init__.py
├── database
│ ├── __init__.py
│ └── conn.py
├── engine
│ ├── __init__.py
│ ├── core
│ ├── data
│ ├── main.py
│ └── utils
├── main.py
├── poetry.lock
├── pyproject.toml
└── run.sh
The issue that I have when I open worker project which uses code from upper directory, pylance gives me an error of an import that could not be resolved. However, this code runs fine and perfect.
I created .vscode/settings.json for the worker project and add these options:
"python.analysis.extraPaths": ["../../app"],
"python.autoComplete.extraPaths": ["../../app"]
But I am still getting these errors! How can I fix this?
These paths fixed my issue:
"python.analysis.extraPaths": ["${workspaceFolder}\\..\\.."],
"python.autoComplete.extraPaths": ["${workspaceFolder}\\..\\.."]
I'm trying to resolve this problem:
Have a tree structure like this:
── geny16
│ ├── gen26xgeny16
│ │ ├── sustrato270
│ │ │ ├── sustrato270_data01.dat
│ │ │ ├── sustrato270_data02.dat
│ │ │ └── sustrato270_data03.dat
│ │ ├── sustrato90
│ │ │ ├── sustrato90_data01.dat
│ │ │ ├── sustrato90_data02.dat
│ │ │ └── sustrato90_data03.dat
│ │ ├── tentata0
│ │ │ ├── tentata0_data01.dat
│ │ │ ├── tentata0_data02.dat
│ │ │ └── tentata0_data03.dat
│ │ └── tenteta90
│ │ │ ├── tentata90_data01.dat
│ │ │ ├── tentata90_data02.dat
│ │ │ └── tentata90_data03.dat
│ └── gen40xgeny16
│ ├── sustrato270
│ ├── sustrato90
│ ├── tenteta0
│ └── tenteta90
└── geny9
├── gen16xgeny9
│ ├── sustrato270
├── sustrato90
│ ├── tenteta0
│ └── tenteta90
├── gen26xgeny9
│ ├── sustrato270
│ ├── sustrato90
│ ├── tenteta0
│ └── tenteta90
└── gen40xgen9y
├── sustrato270
├── sustrato90
├── tenteta0
└── tenteta90
in each deeper folder there are some .dat files that I will manipulate and plot with matplotlib. When I use a python script inside the deeper folder, my work is done: read each data file, normalize a column, superspose and plot data; but as it's the same task for each deep folder, I would like to leave the script in the root folder and that goes through the subfolders to manipulate and plot data.
I can list the name of each element in the tree with this code:
rootDir = '.'
for dirName, subdirList, fileList in os.walk(rootDir):
print('Directorio encontrado: %s' % subdirList)
for fname in fileList:
print('\t%s' % fname)
But it's only a list of names, how can I use this list to navigate and execute my script to manipulate and plot data in each deeper folders?
Thanks in advance for your comments and suggestions.
Gustavo.
You can use pathlib
from pathlib import Path
root_dir = '.' # __file__
d = Path(root_dir).glob('*/*/*/*.dat') #*==any folder/file
for i in d:
print(i.name, i.parent, i.parts[-2])
print(i.resolve())
print(i.stem)
# code to manipulate these files here
# e.g read, visualise and store data
See what you get. You can the do whatever you wish with the files
How do import my class from a specific directory (in my case root directory i want to keep it).
So, i have following directory map, now i need to load the class parsePresets from myglobal.py file, which is located in root directory: /var/tmp/mypython directory.
but, i want to import that from class/methods from my new module: /var/tmp/mypython/media/test.py with:
from myglobal import parsePresets
but i am getting:
from myglobal import parsePresets
ImportError: No module named myglobal
i also have init.py in root directory and in the media directory.
$ cd /var/tmp/mypython; tree
.
├── arduino
│ ├── arduino.diest.c
│ ├── arduino.gent.c
│ ├── arduino.lalouvier.c
│ ├── arduino.makenoise.c
│ ├── arduino.servo.c
│ ├── arduino.string.c
│ ├── arduino.tcpserver.c
│ ├── arduino.tcpserver.c~
│ ├── arduino.test.sh
├── bash
│ ├── all.sh
│ ├── alsa-info.sh
│ ├── asound.conf
│ ├── autoreboot.sh
│ ├── diskfix.sh
│ ├── kernelfix.sh
│ ├── update.sh
│ └── usbformat.sh
├── chrome.py
├── download.py
├── download.sh
├── gui.py
├── image
│ ├── a.png
│ ├── b.gif
│ ├── cross_new.png
│ ├── e150
│ │ ├── 1.png
│ │ ├── de.png
│ │ ├── en.png
│ │ ├── __init__.py
│ │ └── nl.png
│ ├── __init__.py
│ ├── logo.png
│ ├── menu.jpg
│ └── slider_btn.png
├── __init__.py
├── INSTALL
├── ip.py
├── loading.py
├── logout.py
├── media
│ ├── __init__.py
│ ├── test.py
├── menu.py
├── myglass.py
├── myglass.pyc
├── myglobal.py
├── myglobal.pyc
├── rightclick.py
├── runme.sh
├── server.py
├── server.pyc
├── src.nja
├── test
│ ├── Button.py
│ ├── json.py
│ ├── json.pyc
│ ├── keyboard.py
│ ├── loop.sh
│ ├── mytimer.py
│ ├── qtclick.py
│ ├── qtmouse.py
│ ├── qt.py
│ ├── qtwindows7.py
│ ├── shape.py
│ ├── skeleton.py
│ ├── slider.py
│ ├── testpreview.py
│ ├── test.py
│ ├── Text.py
│ ├── transparent.py
│ ├── transparentwindow.py
│ └── Vscale.py
├── test.py
├── unavailable.py
├── upload.sh
└── internet
├── backup
├── protocol.txt
└── server.py
You can add sys.path.append(/var/tmp/mypython/media/) to your script.
EDIT:
$ cat >> /var/tmp/mypython/stackoverflow.py <<\EOF
import sys
sys.path.append("/var/tmp/mypython/")
from myglobal import parsePresets
EOF
$ python /var/tmp/mypython/stackoverflow.py
or with NINJA-IDE
Running: /var/tmp/mypython/media/stackoverflow.py (Wed Dec 11 13:37:25 2013)
Execution Successful!
Use relative imports
from ..myglobal import parsePresets
The extra periods . take you "out" a level in your hierarchy.
I have a project like this:
├── CHANGES.txt
├── LICENSE
├── MANIFEST.in
...
├── docs
│ └── index.rst
├── negar
│ ├── Negar.py
│ ├── Virastar.py
│ ├── Virastar.pyc
│ ├── __init__.py
│ ├── data
│ │ ├── __init__.py
│ │ └── untouchable.dat
│ ├── gui.py
│ ├── gui.pyc
│ ├── i18n
│ │ ├── fa_IR.qm
│ │ └── fa_IR.ts
│ └── negar.pro
├── setup.py
...
and inside that my file Virastar.py need some data from data.untouchable.dat. it works fine until I install the project with this setup.py:
setup(
...
include_package_data=True,
packages = find_packages() + ['negar'],
package_dir={'negar': 'negar'},
package_data={'negar': ['data/*.dat']},
entry_points={
'console_scripts': [
'negar = negar.Negar:main',
],
},
...
)
after that when I start my program and when it needs that data file it return this error:
IOError: [Errno 2] No such file or directory: 'data/untochable.dat'
even in my egg-info sources I can't find any data file:
...
negar/Negar.py
negar/Virastar.py
negar/__init__.py
negar/gui.py
negar/data/__init__.py
have I missed something here?
Thank you all.
EDIT: Do I have to add any special thing in init.py?
and I have to add this: I used untouchable.dat just like this:
f = codecs.open('data/untouchable.dat', encoding="utf-8")
I used data_files
data_files = [('', ['negar/data/untouchable.dat'])],
The first problem is that I didn't import my data file into the package with MANIFEST.in file. I imported it like this:
include negar/data/*.dat
After that my data file already imported with my package install. but because I had mistakes in open my data files, python couldn't find it. this question helped me to find the right way
Python Access Data in Package Subdirectory and now I use something like this:
import os
this_dir, this_filename = os.path.split(__file__)
DATA_PATH = os.path.join(this_dir, "data", "data.txt")
print open(DATA_PATH).read()
Maybe try:
package_data={'negar/data': ['data/*.dat']},
A solution that does not require any of:
MANIFEST.in
include_package_data=True
package_dir={}
and neither the __init__.py file in the folder!
Having the project like:
├── CHANGES.txt
├── LICENSE
...
├── negar
│ ├── Negar.py
│ ├── Virastar.py
│ ├── Virastar.pyc
│ ├── __init__.py
│ ├── data
│ │ ├── __init__.py
│ │ └── untouchable.dat
│ ├── subfolder
│ │ ├── __init__.py
│ │ ├── data_NOT_included
│ │ │ └── garbage.toml
│ │ └── data_with_no_init_file
│ │ ├── config.ini
│ │ └── options.yml
│ ├── gui.py
│ ├── gui.pyc
│ ├── i18n
│ │ ├── fa_IR.qm
│ │ └── fa_IR.ts
│ └── negar.pro
├── setup.py
...
The setup file:
setup(
...
packages = find_packages()
package_data= {
# all .dat files at any package depth
'': ['**/*.dat'],
# into the data folder (being into a module) but w/o the init file
'negar.subfolder': [ '**/*.ini', '**/*.yml', ]
},
entry_points={
'console_scripts': [
'negar = negar.Negar:main',
],
},
...
)
This worked great in my case and I avoided maintaining an additional MANIFEST.in file.
caveat: the "negar.subfolder" has to be a python module.
To the access the file:
from importlib.resources import files
config = files('negar.subfolder').joinpath('data_with_no_init_file').joinpath('config.ini').read_text()
from accessing-data-files-at-runtime