I have the following in my setup.py
entry_points={
'console_scripts': [
'my-app=main:run',
],
and my file structure is
▶ tree -L 2
.
├── Dockerfile
├── README.md
├── redis.conf
├── setup.cfg
├── setup.py
├── src
│ ├── main.py
│ ├── settings.py
│ ├── bfile.py
│ └── afile.py
and in my main.py
def run():
actual_run()
Locally, in a virtualenv when performing
pip install -e . && my-app
it runs smoothly.
However in an image created with the following Dockerfile
FROM python:3.9-slim-buster
WORKDIR /app
COPY . .
RUN pip3 install .
CMD [ "my-app"]
I get
Traceback (most recent call last):
File "/usr/local/bin/my-app", line 5, in <module>
from main import run
ModuleNotFoundError: No module named 'main'
What I noticed, is that in my image, in /usr/local/lib/python3.9/site-packages/ there only the a file named my-app-0.0.1.dist-info and not a dir with the source files.
Why is that?
You didn't give a minimal workable setup.py, but next could works, FYI:
Dockerfile:
FROM python:3.9-slim-buster
WORKDIR /app
COPY . .
RUN pip3 install .
CMD [ "my-app"]
setup.py:
from setuptools import setup, find_packages
setup(
name="my-app",
version="0.0.1",
packages=['main'],
package_dir={'main':'src'},
entry_points={
'console_scripts': [
'my-app=main.main:run',
],
}
)
src/main.py:
def run():
print("hello world")
Execution:
# docker build -t abc:1 .
# docker run --rm abc:1
hello world
Explain:
In above, src folder will map to main module, and packages=['main'] specify the python package name which will be installed into site-packages.
In final image, the package will be located as next:
root#7b5ce63d8226:/usr/local/lib/python3.9/site-packages/main# ls
__pycache__ main.py
So, you need to use main.main:run to call your function.
Related
Structure:
.
├── application
│ └── runner.py
└── dummyLibrary
├── helperFunctions.py
├── __init__.py
└── setup.py
runner.py:
import dummyLibrary
dummyLibrary.foo()
dummyLibrary.bar()
init.py:
(empty file)
helperFunctions.py:
def foo():
print("called foo()")
def bar():
print("called bar()")
setup.py:
#!/usr/bin/env python
from distutils.core import setup
setup(name='dummyLibrary', version='0.0.1')
After cd'ing into dummyLibrary/ ,
I tried installing dummyLibrary with
pip3 install -e .
This was the output:
Defaulting to user installation because normal site-packages is not writeable
Obtaining file:///home/ubuntu/Documents/pythonTest/dummyLibrary
Preparing metadata (setup.py) ... done
Installing collected packages: dummyLibrary
Running setup.py develop for dummyLibrary
Successfully installed dummyLibrary-0.0.1
I tried installing dummyLibrary with
python3 -m pip install -e .
This was the output:
Defaulting to user installation because normal site-packages is not writeable
Obtaining file:///home/ubuntu/Documents/pythonTest/dummyLibrary
Preparing metadata (setup.py) ... done
Installing collected packages: dummyLibrary
Attempting uninstall: dummyLibrary
Found existing installation: dummyLibrary 0.0.1
Uninstalling dummyLibrary-0.0.1:
Successfully uninstalled dummyLibrary-0.0.1
Running setup.py develop for dummyLibrary
Successfully installed dummyLibrary-0.0.1
After cd'ing into application/ and running
python3 runner.py
I get:
Traceback (most recent call last):
File "runner.py", line 1, in <module>
import dummyLibrary
ModuleNotFoundError: No module named 'dummyLibrary'
no matter how I try to install my library
Why is this?
Additional Information:
Using Ubuntu, Not using a virtual environment.
I tried rebooting after installing. Didn't help.
Welcome to Stack Overflow!
First thing: __init__.py and helperFunctions.py should be inside another nested folder with the same name as the package.
.
├── application
│ └── runner.py
└── dummyLibrary
├── dummyLibrary
│ ├── __init__.py
│ └── helperFunctions.py
└── setup.py
Secondly, with the code in runner.py as how you'd like it, the __init__.py should include the following line to import all of the functions in helperFunctions.py:
from .helperFunctions import *
Finally, your setup.py should also include the parameter packages as a list of folder names in the package, in this case, the parameter should be packages=['dummyLibrary'].
P.S.: it's a pythonic practice to name your files and packages in snake_case rather than camelCase.
I have a local python project (A) that depends on another local python project (B).
I have added the project B as a git submodule to a dependencies folder.
When I pull the repo with all the submodules, I would like to have a setup.py in A such that wenn I run pip install -e . in A, it also installs the setup.py in B.
The repo looks like this:
A
├── A
│ └── foo.py
│
├── dependencies
│ └── B
│ ├── B
│ │ └── foo.py
│ │
│ └── setup.py
│
└── setup.py
I tried from setuptools import setup, find_packages
setup(
name='A',
packages=find_packages(),
)
But that only finds package A, which makes sense.
I also tried adding a preinstall command
class PreInstallCommand(install):
"""Pre-installation for installation mode."""
def run(self):
check_call("pip install -e dependencies/B".split())
install.run(self)
setup(
...
cmdclass={
'install': PreInstallCommand,
},
...
)
But that one is not triggered when I install it in editable mode (pip install -e).
Thanks in advance for your advice! :)
Cheers!
Hi guys i have a problem with creating python package out of my project. I read some tutorials but every one leads to uploading files to pypi which i dont want to do. I just want to pip install it to my machine locally using tar.gz file.
Here's a structure of my project folder:
root
├── src
│ ├── __init__.py
│ ├── config.py
│ └── sth_a
│ │ ├── __init__.py
│ │ └── a.py
│ └── sth_b
│ ├── __init__.py
│ └── b.py
└── setup.py
Here is how my setup.py file looks like:
from setuptools import setup, find_packages
setup(name="mypkg",
version='0.0.1',
author="whatever",
packages=find_packages()
)
First i run command:
python setup.py sdist bdist_wheel
then it creatates dist dictionary with tar.gz file and wheel file, so i just run
pip3 install dist/mypkg-0.0.1.tar.gz
After this first problem emerges. To import these files somewhere else i need to write
from src.sth_a.a import *
but i want to do this like this
from mypgk.src.sth_a.a import *
or even if i just want to 'publish' for example functions from file a.py
from mypck.a import *
Also i was having another issues bit this answer helped me a bit but it is not still what i want pip install . creates only the dist-info not the package
If I have a tree that looks like:
├── project
│ ├── package
│ │ ├── __init__.py
│ │ ├── setup.py
├── env
└── setup.py
Is there a way to include the nested setup.py in the install for the top setup.py? I want to avoid this:
pip install -e . ; cd project/package ; pip install -e .
The solution is to have two separate projects: a main project (usually an application) and a sub-project (usually a library). The main application has a dependency to the library.
Tree structure and setup.py
The main project can have the following structure:
your_app/
|-- setup.py
ˋ-- src/
ˋ-- your_app/
|-- __init__.py
|-- module1.py
ˋ-- ...
The setup.py of your application can be:
from setuptools import find_packages
from setuptools import setup
setup(
name='Your-App',
version='0.1.0',
install_requires=['Your-Library'],
packages=find_packages('src'),
package_dir={'': 'src'},
url='https://github.com/your-name/your_app',
license='MIT',
author='Your NAME',
author_email='your#email.com',
description='Your main project'
)
You can notice that:
The name of your application can be slightly different to the name of your package;
This package has a dependency to "Your-Library", defined below;
You can put your source in the src directory, but it is optional. A lot of project have none.
The sub-project can have the following structure:
your_library/
|-- setup.py
ˋ-- src/
ˋ-- your_library/
|-- __init__.py
|-- lib1.py
ˋ-- ...
The setup of you library can be:
from setuptools import find_packages
from setuptools import setup
setup(
name='Your-Library',
version='0.1.0',
packages=find_packages('src'),
package_dir={'': 'src'},
url='https://github.com/your-name/your_library',
license='MIT',
author='Your NAME',
author_email='your#email.com',
description='Your sub-project'
)
Putting all things together
Create a virtualenv for your application and activate it
Go in the your_library/ directory and run:
pip install -e .
Then, go in your_app/ directory and run:
pip install -e .
You are now ready to code. Have fun!
See the Hitchhiker's Guide to Python: “Structuring Your Project”.
My folder structure looks like
.
├── my_package
│ ├── A.py
│ └── __init__.py
├── my_package2
│ ├── B.py
│ └── __init__.py
└── setup.py
And my setup.py looks like
from setuptools import setup
if __name__ == '__main__':
setup(name='my_packages',
packages=['my_package'])
When I run
python3 setup.py develop
The egg file is created locally and an egg-link file is placed in my site-packages directory. Also, the folder is added to easy_install.pth which means that both my_package and my_package2 are importable. This is different than running python3 setup.py install (only my_package would be available per the keyword argument passed to the setup function).
This same behavior occurs when installing with pip using the -e flag.
Is this intended behavior? Is there any way to replicate the functionality of install using develop?