I'm not very experienced with dependency management in Python but it seems that all of the dependencies in requirements.txt could alternatively be placed in the setup.py file under the install_requires field as follows:
setup(
...
install_requires=['numpy=1.2.3', 'pandas=1.2.3']
...
)
Since pip install ./ using a setup.py file also provides lots of additional functionality compared to pip install -r requirements.txt, what is the use for the latter? Would a project ever have a valid reason to use both?
pip install . installs your custom package with all its dependencies. pip install -r requirements.txt installs only the dependencies.
You could also add -e . to your requirements.txt and install your package with pip install -r requirements.txt
Related
I have a requirement.txt file with the list of python package to install. One of the packages is psycopg2==2.6.2 I need to update this package to psycopg2==2.7. I tried to install by pip3 install psycopg2 But it doesn't affect requirement.txt file. Can you please point me in the right direction?
Notice that running pip3 install psycopg2 doesn't respect the requirements.txt file. To upgrade this package you need to use -U option:
pip3 install -U psycopg2
which is a shorthand for:
pip3 install --upgrade psycopg2
After that, you can update your requirements.txt with the following command:
pip freeze > requirements.txt
If you're looking for a solution to automatically update the requirements.txt file after you upgrade package/packages, you can use pip-upgrader.
Installation:
pip install pip-upgrader
Usage:
pip-upgrade
The above command auto-discovers the requirements file and prompts for selecting upgrades. You can also specify a path to the requirements file or/and specify a package to upgrade:
pip-upgrade /path/to/requirements.txt -p psycopg2
As you've discovered, pip doesn't update the requirements file. So the workflow you'd likely want to use is:
Update the version of psycopg2 in your requirements file from 2.6.2 to 2.7
Run pip install with the upgrade flag
pip3 install -U -r requirements.txt
If you're familiar with tools like npm that do update the version in the catalog file, you may be interested in using pipenv, which manages your dependencies and the virtual environment for you, much like npm does.
If you don't know the latest version of your package, then use pip to figure it out:
$ pip list --outdated | grep psycopg2
psycopg2 (2.7.3.2) - Latest: 2.7.4 [wheel]
you can try:
pip install --upgrade --force-reinstall -r requirements.txt
You can also ignore installed package and install the new one :
pip install --ignore-installed -r requirements.txt
For example, we have project Foo with dependency Bar (that in private Git repo) and we want install Bar into Foo directory via pip from requirements.txt.
We can manually install Bar with console command:
pip install --target=. git+ssh://git.repo/some_pkg.git#egg=SomePackage
But how to install Bar into current directory from requirements.txt?
The best way to do this would be to clone the repository, or just donwload the requirements.txt file, and then run pip install -r requirements.txt to install all the modules dependencies.
You can use the "editable" syntax in requirements.txt to install a package from a VCS (eg git)
From the docs:
pip install -e git+https://git.repo/some_pkg.git#egg=SomeProject
I have a shell script whose last line is:
pip install .
What does it do?
pip install <package-name> installs the specified package
pip install -r requirements.txt installs all packages specified in requirements.txt
But I am not sure what the above command does.
Explicitly, pip install . will execute the setup.py file in the current directory (which will usually load a requirements.txt file).
"Install the project found in the current directory".
This is just a specific case of pip install /path/to-source/tree.
To quote the the pip install documentation describing this usage:
pip install [options] [-e] <local project path> ...
I looked up how to install multiple packages from a requirements document using pip. The answers were mostly:
pip install -r requirements.txt
What does the -r do though? I can't find an answer for this and it isn't listed when I run pip help.
Instead of pip --help, look into pip install --help:
-r, --requirement Install from the given requirements
file. This option can be used multiple
times.
Also see these documentation paragraphs:
pip install
Requirements Files.
-r will search for requirement file.
pip install --help
will help you !!
May, 2022 Update:
If you run this command below without "-r":
pip install requirements.txt
You will get this error below:
ERROR: Could not find a version that satisfies the requirement requirements.txt (from versions: none)
HINT: You are attempting to install a package literally named "requirements.txt" (which cannot exist). Consider using the '-r' flag to install the packages listed in requirements.txt
ERROR: No matching distribution found for requirements.txt
Because "pip" tries to install the package "requirements.txt" instead of installing the packages listed in "requirements.txt". Of cource, the package "requirements.txt" doesn't exist in PyPI while for example, the packages "django" and "pillow" exist in PyPI:
pip install django
pip install pillow
So, to install the packages listed in "requirements.txt", you must need "-r";
pip install -r requirements.txt
You can check what "-r" means by running the command below:
pip install --help
-r, --requirement Install from the given requirements file. This option can be used multiple times.
In your case pip install -r requirements.txt will install the libraries listed in your requirements.txt file.
pip install requirements.txt
Above statement looks for a python package named requirements.txt. No such package exists. Your intention is that pip install opens the txt and reads the packages from there. The -r allows pip install to open requirements.txt and install the packages inside of it instead.
I have a script that creates a virtualenv, installs distribute and pip in it and then optionally clones a git repo.
Now I have the project I will be working on, installed. But its dependencies are not installed. How can I make pip install all the dependencies as if I have issued a pip install MyApp?
EDIT: Appareantly my question is a duplicate of this one.
Not exactly sure but pip install -e . seems to do what I want without too many extra stuff lying around. I'd prefer if my code wasn't linked from site-packages though.
If your dependencies are defined in the setup.py file, you can first dump them to an external file using:
python setup.py egg_info
This will list all your dependencies in YOUR_PROJECT.egg-info/requires.txt file. Then you can install them using pip:
pip install -r *.egg-info/requires.txt
to delete what you just created:
rm -rf *.egg-info/
to save some time copy pasting:
python setup.py egg_info
pip install -r *.egg-info/requires.txt
rm -rf *.egg-info/
In my package root issuing pip install -e . installs dependencies.
To install your project's dependencies (i.e. install_requires + extra_requires) you have to extract your dependencies using setuptools egg-info and then install the filtered list of the combined dependencies:
python setup.py egg_info
pip install `grep -v '^\[' *.egg-info/requires.txt`
You should use the pip requirements file.
Essentially, place all your requirements, one in each line in a file and pass that to pip using the command
pip install -r requirements.txt
What more, if you have a standard environment, pip can actually dump such a file from existing installs using the command:
pip freeze
You can put the file thus generated directly into the pip requirements, and call the previous command from your deployment script.
Pretty cool, isnt it? :)
You can use pip-tools to create a requirements.txt that only contains the dependencies of your package:
$ pip-compile -o requirements.txt setup.py
Note that the command above only works if you do not already have a requirements.txt file. If you happen to have one already, just delete it.
Using the generated requirements.txt you can then run pip to install the dependencies:
$ pip install -r requirements.txt
Bonus 1:
The requirements.txt will include comments that indicate where the regarding dependency originates from.
Bonus 2:
If you have have an extras_require section for optional dependencies in your setup.py that looks e.g. like this:
...
extras_require={
"development": [
"wheel",
"debugpy",
"pytest",
],
},
...
You can create the requirements.txt including the optional dependencies by using:
$ pip-compile -o requirements.txt --extra development setup.py