Few days ago I've created a little project called pyblime and right now I was trying to figure out how to create a proper setup.py that allowed me to upload the "right stuff" to PyPi so users will be able to enjoy the project by using pip without doing anything "too fancy" like calling fancy custom dev scripts, right now the project tree structure looks like this:
│ .gitignore
│ configure.py
│ MANIFEST.in
│ README.md
│ requirements.txt
│ setup.py
│
├───data
│ ├───commands
│ │ comment.py
│ │ fold.py
│ │
│ ├───screenshots
│ │ test_simple.png
│ │ test_themes.gif
│ │
│ ├───st_build_3149
│ │ ├───syntax
│ │ └───themes
│ └───testfiles
├───docs
│ build.md
│ contributing.md
│ guidelines.md
│ usage.md
│
├───examples
│ demo_00.py
│ tutorials.py
│ tutorial_00.py
│ tutorial_01.py
│ tutorial_02.py
│ tutorial_03.py
│ tutorial_04.py
│ tutorial_05.py
│ tutorial_06.py
│
├───pyblime
│ utils.py
│ view.py
│ __init__.py
│
├───sublime_text
│ sublime.py
│ sublime_plugin.py
│
└───tests
run_all.py
test_scopes.py
test_view.py
x.py
Rather than 1 question, I've got few simple doubts:
What'd be the "standard" way to instruct setup.py to copy sublime_text/sublime.py and sublime_text/sublime_plugin.py files into Lib/site-packages root?
How'd you tell setup.py to copy the whole folder pyblime adhoc in Lib/site-packages?
Finally, is it correct to upload tests/examples/tests/docs/data to PyPi? This is, content that won't be necessary to use the SDK/library itself... If it's not, where would you include this type of data... I'm aware there exists the concept of sdist&dist folders and I've already read a bit about it here but the question still remains :)
Right now my setup.py looks something like this:
from pathlib import Path
from setuptools import setup
root_path = Path(__file__).parent
requirements = (root_path / "requirements.txt").read_text()
requirements = [
v for v in requirements.split("\n")
if v.strip() and not v.strip().startswith("#")
]
readme = (root_path / "README.md").read_text()
setup(
author="mcve",
author_email="mcve",
classifiers=["mcve"],
description="mcve",
install_requires=requirements,
keywords=["mcve"],
long_description=(root_path / "README.md").read_text(),
name="mcve",
# package_data = {}, <---- How do i use this?
# packages = [], <---- Do I need to use this?
url="mcve",
version="0.0.1",
)
Ps. And yeah... I've already read the official docs out there about packaging... but if I had understood those docs I wouldn't be asking this on SO ;D . Thanks in advance!
Well, let's step by step.
For the first question. Mostly you shouldn't do that, instead, you should consider sublime_text as a package too. The structure should look like:
├───sublime_text
| __init__.py
│ sublime.py
│ sublime_plugin.py
And you should use it like from sublime_text import sublime in your other packages. That's could be better as you won't pollute the global namespace too much. Or if this is not a common package that you want to share between many other packages, you can directly include it as a submodule in your main package.
Or if you really wanna do this, you can place those two files in the root directory and use:
...
packages = find_packages(),
py_modules=["sublime", "sublime_plugin"],
...
For the second question. As that is a package, you can add that path to packages: packages=[""]. Or for convenience, you can use packages=find_packages(). It will help you to find all packages under the current directory which is "".
For your third question. Mostly it is not correct, you just need to submit what user needs to the PyPi. For docs, you should use readthedocs website. And for other examples and tests, just leave them on your Github. That's enough.
Related
I have searched multiple questions regarding this problem and although there are good answers, I haven't been able to find the exact solution to what I have.
I have a folder structure like this.
root
├── subprogram
│ └── module_1
│ │ └──__init__.py
│ │ └──dependency_0.py
│ └── module_2
│ └──__init__.py
│ └──dependency_1.py
│
└── main.py
From main.py, I do something like,
from subprogram.module_1.dependency_0 import ExampleClass
While inside the subprogram.moudle_1.dependency_0.py it does something like,
from module_2 import dependency_1
What is the least invasive (I make minimal to no change to ANYTHING inside the subprogram) method of using that ExampleClass from the root program?
Thank you.
While working on my first "bigger" Python project, I'm running into a multitude of issues while trying to debug and import various modules/sub-modules. Here's my tree:
netbox-setup/
├── README.md
├── TODO.md
├── netboxsetup
│ ├── __init__.py
│ ├── constants.py
│ ├── helpers
│ │ ├── __init__.py
│ │ ├── custom_napalm
│ │ │ ├── __init__.py
│ │ │ └── ios.py
│ │ ├── infoblox.py
│ │ ├── ise.py
│ │ ├── netbox.py
│ │ ├── solarwinds.py
│ │ └── utilities.py
│ └── main.py
├── requirements.txt
├── setup.py
└── tests
main.py imports:
from netboxsetup.helpers import utilities
from netboxsetup.helpers import solarwinds
utilities.py imports:
from napalm import get_network_driver
from netboxsetup.constants import USER
From what I've been reading, it's recommended to use absolute imports in a package rather than relative. If I try to run main.py from within the netboxsetup folder, it states that netboxsetup cannot be found. So I removed that and just called from helpers import utilities. Now running main.py works, but when it imports the utilities file, the imports in the utilities file fail. Information I'm finding regarding import use in a package/module seems to be inconsistent on what to do/use.
Finally, if I run a python3 shell from the the netbox-setup folder, and use from netboxsetup.helpers import utilities it imports. If I do the same from the netboxsetup folder, it states that "ModuleNotFoundError: No module named 'netboxsetup'". Going back to where it worked, I then followed https://napalm.readthedocs.io/en/latest/tutorials/extend_driver.html to create the same setup exactly, and it states that my new method isn't found when I run the grab_inventory function I defined in the utilities.py. And I did appropriately create the respective get_inventory function in the class in the ios.py file as the Napalm docs advise.
def grab_inventory(ip, password):
# make less assumption, account for nx-os as well
driver = get_network_driver('ios')
with driver(ip, USER, password) as client:
result = client.get_inventory()
return result
I'm guessing all of my issues are related to pathing - whether absolute or relative, but I'm just having a very difficult time determining what the exact pathing is that works for everything. Is anyone able to point me into a proper source for proper import of modules in custom packages? Thanks.
P.S. Is it possible to debug an individual function in VSCode, giving it arguments at runtime as well (rather than having to run through all of the main sequence code to get to the function)?
When I try to use the default docs in FastApi never works. If I use /docs it gives an error saying:
"Failed to load API definition. Error Hide Fetch Internal Server Error /openapi.json"
and if I use /redoc it loads forever and never appears anything. I searched a lot but the only thing I found out was someone using root_pah="/folder/" inside of the FastAPI() main instance, so I used the app folder which is the repository of the whole api project. I don't know if it will help you guys but the tree of the project is:
C:.
├───.pytest_cache
│ └───v
│ └───cache
├───alembic
├───app
│ ├───models
│ │ └───__pycache__
│ ├───routers
│ │ └───__pycache__
│ ├───tests
│ │ ├───.pytest_cache
│ │ │ └───v
│ │ │ └───cache
│ │ └───__pycache__
│ └───__pycache__
├───venv
│ ├───Include
│ │
│ └───Scripts
└───__pycache__
p.s. I wiped most of the part of the venv directory because I thought that it was useless for this and also when I tried to copy and paste the whole output in here, Stack Overflow would accept just part of it because it was too big.
I would like to load in a function/module with my working directory as project main directory but the function file is store below the a subdirectory level so the normal
from function_file import function_name
does not work.
This is what the project directory set up looks like:
└───project_main_directory
│ .gitattributes
│ .gitignore
│ README.txt
│
├───code
│ ├───exploration
│ └───scripts
│ │ script1.py
│ │ script2.py
│ │ script3.py
│ │
│ └───functions
│ │ function1.py
│ │ function2.py
│ └───__init__.py
│
├───data
│ └───example_data
│ data.csv
└───documents
So I tried to import functions via
import code.scripts.function.function1 from function1
and that doesn't work. I know it's because the other subdirectories aren't modules, but I want to ask if there is away around that?
-- EDIT
I'm working from .py file in code/scripts/script1.py but working directory is project_main_directory/
Add an empty file __init__.py to every subdirectories to make them as modules.
.
├── code
│ ├── __init__.py
│ └── scripts
│ ├── __init__.py
│ └── script1.py
└── main.py
Then if you have a function called hello in code/scripts/script1.py you can import that function by:
from code.scripts.script1 import hello
hello("yo")
If your current directory is project_main_directory, you can use:
from code.scripts.functions.function1 import function1
Directory of your script doesn't matter. Only your current directory matters (refer top of the IDE)
To import function/module from another python file, you have to do something like below -
from code.scripts.functions.function1 import function1
Above we are loading function1 from function1.py file which is stored in functions directory which is stored in scripts directory and finally in code directory.
EDIT - so you are saying, you want to load a function from function1.py in script1.py? In that case from .functions.function1 import function should work.
I need to perform an import of a function on my python project.
I know there're dozens of similar questions on SO, however, unfortunately, I couldn't find the right solution for me because the answers are either too question specific, or way too general, or they're just ugly hacks (like operating with absolute paths).
Here's how my folder structure looks like:
PythonClient:.
│ .gitignore
│ des.py
│ des_test.py
│ des_var2.py
│ gui.py
│ index.py
│ __init__.py
│
├───diffie_hellman
│ │ diffie_hellman.py
│ │ diffie_hellman_test.py
│ │ __init__.py
│ │
│ └───__pycache__
│ diffie_hellman.cpython-35.pyc
│
├───hashes
│ │ collision.py
│ │ hash_function.py
│ │ __init__.py
│ │
│ └───__pycache__
│ hash_function.cpython-35.pyc
│ __init__.cpython-35.pyc
│
└───__pycache__
des.cpython-35.pyc
des_var2.cpython-35.pyc
I need to import the ./hashes/hash_function.py from ./diffie_hellman/diffie_hellman.py.
The ./hashes/hash_function.py file contains the only function named hash_function.
I've tried quite a number of ways to perform the import but just couldn't do it.
I always get either
SystemError: Parent module '' not loaded, cannot perform relative
import
when I use . in my import statement (i.e. from .hashes.hash_function)
or I get this:
ImportError: No module named 'hashes'
Every __init__.py file is empty.
Here's the list of my attempts:
from hashes import hash_function
from hashes.hash_function import hash_function
from .hashes.hash_function import hash_function
from ..hashes.hash_function import hash_function
import hashes
import hash_function
from .. import hash_function
from . import hash_function
from PythonClient.hashes.hash_function import hash_function
Could you please help me to resolve my problem and to understand how to work with such imports?
PS: The solution couldn't be found here stackoverflow.com/questions/14132789/
I know you have already accepted an answer, but if you wanted a less "permanent" solution (that is to say, if you didn't want to install your code), another option would be to simply add the parent of your PythonClient directory to your path. This can be done permanently (which varies depending upon operating system) or temporarily in code:
import os
import sys
p = os.path.abspath('../..')
if p not in sys.path:
sys.path.append(p)
from PythonClient.hashes.hash_function import hash_function
Cheers!
The fact that you have an __init__.py tells me that PythonClient is itself a library. Do from PythonClient.hashes.hash_function import hash_function. I always like fully qualified paths.
You need to install your library too before you can import from it. That requires a setup.py file in your home directory. After that, you should pip install your library for testing, e.g., `pip install -e .