I'm building my first project on GitHub and my python src code uses an open-source, 3rd-party library that I have installed on my computer. However, I heard it is best practice to create a dep (dependencies) folder to store any additional libraries I would need. How do I actually install the libraries in the dep folder and use them from there instead of my main computer?
You have to create a requirements.txt file with each package on a separate line. e.g.
pandas==0.24.2
You also might want to add a setup.py to your python package. In the setup you have to use "install_requires" argument. Although install_requires will not install packages when installing your package but will let the user know which packages are needed. The user can refer to the requirements.txt to see the requirements.
You can check it here: https://packaging.python.org/discussions/install-requires-vs-requirements/
The following is an example of setup.py file:
from distutils.core import setup
from setuptools import find_packages
setup(
name='foobar',
version='0.0',
packages=find_packages(),
url='',
license='',
author='foo bar',
author_email='foobar#gmail.com',
description='A package for ...'
install_requires=['A','B']
)
Never heard about installing additional libraries in a dependencies folder.
Create a setup python file in your root folder if you don't already have it, in there you can define what packages (libraries as you call them) your project needs. This is a simple setup file for example:
from setuptools import setup, find_packages
setup(
name = "yourpackage",
version = "1.2.0",
description = "Simple description",
packages = find_packages(),
install_requires = ['matplotlib'] # Example of external package
)
When installing a package that has this setup file it automatically also install every requirement in your VENV. And if you're using pycharm then it also warns you if there's a requirement that's not installed.
Related
I've read a discussion where a suggestion was to use the requirements.txt inside the setup.py file to ensure the correct installation is available on multiple deployments without having to maintain both a requirements.txt and the list in setup.py.
However, when I'm trying to do an installation via pip install -e ., I get an error:
Obtaining file:///Users/myuser/Documents/myproject
Processing /home/ktietz/src/ci/alabaster_1611921544520/work
ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory:
'/System/Volumes/Data/home/ktietz/src/ci/alabaster_1611921544520/work'
It looks like pip is trying to look for packages that are available on pip (alabaster) on my local machine. Why? What am I missing here? Why isn't pip looking for the required packages on the PyPi server?
I have done it before the other way around, maintaining the setup file and not the requirements file. For the requirements file, just save it as:
*
and for setup, do
from distutils.core import setup
from setuptools import find_packages
try:
from Module.version import __version__
except ModuleNotFoundError:
exec(open("Module/version.py").read())
setup(
name="Package Name",
version=__version__,
packages=find_packages(),
package_data={p: ["*"] for p in find_packages()},
url="",
license="",
install_requires=[
"numpy",
"pandas"
],
python_requires=">=3.8.0",
author="First.Last",
author_email="author#company.com",
description="Description",
)
For reference, my version.py script looks like:
__build_number__ = "_LOCAL_"
__version__ = f"1.0.{__build_number__}"
Which Jenkins is replacing the build_number with a tag
This question consists of two separate questions, for the rather philosopihc choice of how to arrange setup requirements is actually unrelated to the installation error that you are experiencing.
First about the error: It looks like the project you are trying to install depends on another library (alabaster) of which you apparently also did an editable install using pip3 install -e . that points to this directory:
/home/ktietz/src/ci/alabaster_1611921544520/work
What the error tells you is that the directory where the install is supposed to be located does not exist anymore. You should only install your project itself in editable mode, but the dependencies should be installed into a classical system directory, i. e. without the option -e.
To clean up, I would suggest that you do the following:
# clean up references to the broken editable install
pip3 uninstall alabaster
# now do a proper non-editable install
pip3 install alabaster
Concerning the question how to arrange setup requirements, you should primarily use the install_requires and extras_require options of setuptools:
# either in setup.py
setuptools.setup(
install_requires = [
'dep1>=1.2',
'dep2>=2.4.1',
]
)
# or in setup.cfg
[options]
install_requires =
dep1>=1.2
dep2>=2.4.1
[options.extras_require]
extra_deps_a =
dep3
dep4>=4.2.3
extra_deps_b =
dep5>=5.2.1
Optional requirements can be organised in groups. To include such an extra group with the install, you can do pip3 install .[extra_deps_name].
If you wish to define specific dependency environments with exact versions (e. g. for Continuous Integration), you may use requirements.txt files in addition, but the general dependency and version constraint definitions should be done in setup.cfg or setup.py.
Given a Python package with the following structure.
Installed it with pip
pip install --upgrade git+git://github.com/balandongiv/driving_tools.git
The installed directory looks as below
As shown in figure above, the subfolder sub_file and the nickname_override.py are missing in the installation folder.
May I know what modification is required to amend this issue.
Modification to be made as per Balaitous
from setuptools import setup,find_packages
setup(name='ppackage',
version='0.0.111',
description='make life easier',
author='testx',
packages=['ppackage','ppackage.sub_file'],
)
In packages argument of setup fonction, all modules have to be explicitly mentioned. Module can be a python file or a folder containing __init__.py.
It is not recursive. Here you have two modules ppackage and ppackage.sub_folder.
See: https://docs.python.org/3/distutils/setupscript.html#listing-whole-packages
So you should have:
setup(
name=...,
packages=["ppackage", "ppackage.sub_folder"],
...
)
If you want to embed all modules in you package, you can use find_packages
from setuptools import find_packages
setup(
packages=find_packages(),
...
)
So what i mean by the title is like this, I'm gonna create a python package and it needs pygame for my package to work, but I can't figure out how do you add that in setup.py
from setuptools import setup
setup(
...
install_reqs=[
"pygame",
],
...
)
I change install_reqs into???
install_requires.
See documentation of the setup function
I'm building a new PyPI package based on an existing open source project using setuptools and add some code modifications (they are not the same).
Example:
opensource-custom=2.13.1
Since this project requires dependencies that will look for opensource
what options can I pass to my setup.py when building my wheel files so when I do pip freeze/pip list I can see both?
opensource-custom=2.13.1
opensource=2.13.0
An example of this scenario is intel-numpy if you do a pip install of it, it will generate a copy of numpy.
>pip install intel-numpy
>pip freeze
icc-rt==2019.0
intel-numpy==1.15.1
intel-openmp==2019.0
mkl==2019.0
mkl-fft==1.0.6
mkl-random==1.0.1.1
numpy==1.15.1
tbb==2019.0
tbb4py==2019.0
It sounds like you want to make opensource a dependency of opensource-custom. To do this, you can specify the install_requires parameter in setup.py:
from setuptools import setup
setup(
name='opensource-custom',
install_requires=[
'opensource',
],
...
)
See https://packaging.python.org/guides/distributing-packages-using-setuptools/#install-requires
I have a Python project with the following structure (irrelevant source files omitted for simplicity):
myproject/
mysubmodule/
setup.py
setup.py
The file myproject/setup.py uses distutils.core.setup to install the module myproject and the relevant sources. However, myproject requires mysubmodule to be installed (this is a git submodule). So what I am doing right now is:
myproject/$ cd mysubmodule
myproject/mysubmodule/$ python setup.py install
myproject/mysubmodule/$ cd ..
myproject/$ python setup.py install
This is too tedious for customers, especially if the project will be extended by further submodules in the future.
Is there a way to automate the installation of mysubmodule when calling myproject/setup.py?
setuptools.find_packages() is able to discover submodules
Your setup.py should look like
from setuptools import setup, find_packages
setup(
packages=find_packages(),
# ...
)
Create a package for mysubmodule with its own setup.py and let the top-level package depend on that package in its setup.py. This means you only need to make the packages / dependencies available and run python setup.py install on the top-level package.
The question then becomes how to ship the dependencies / packages to your customers but this can be solved by putting them in a directory and configuring setup.py to include that directory when searching for dependencies.
The alternative is to "vendor" mysubmodule which simply means including it all in one package (no further questions asked) and having one python setup.py install to install the main package. For example, pip vendors (includes) requests so it can use it without having to depend on that requests package.