Is there a way to list which of my requirements in my requirements.txt don't have a corresponding wheel?
This sort of works but doesn't seem like a very nice option and takes a lot longer than I would like.
$ pip install -r requirements.txt --force-reinstall | grep '.tar.gz'
When I have, for example, a requirements-dev.txt and a requirements.txt, I know I can have -r requirements.txt inside requirements-dev.txt, for example, and running pip install -r requirements-dev.txt would install packages from both files.
That said, I was certain that any install option would work fine inside a requirements file. Turns out that when I place inside a requirements file something like:
mypackage==1.0.0 -t /path/to/local/dir
I get:
pip: error: no such option: -t
while running pip install mypackage==1.0.0 -t /path/to/local/dir works just fine. For complicated reasons, I need to place multiple packages in one requirements file, where some packages must target one directory, others must target another, and so goes on.
Any solutions to make this work?
As of today (in pip version 21.2.4), the -t, --target <dir> option is not supported in requirements.txt files. The section "Requirements File Format" of pip's User guide lists the currently supported options:
-i, --index-url
--extra-index-url
--no-index
-c, --constraint
-r, --requirement
-e, --editable
-f, --find-links
--no-binary
--only-binary
--prefer-binary
--require-hashes
--pre
--trusted-host
--use-feature
pip install -r requirements.txt -t /path/to/install
This should work. It worked for me.
If you want different modules to be installed to different locations, then I think you might have to put them into multiple requirements text files. This is at least as far as I know
I have my Python script and my requirements.txt ready.
What I want to do is to get all the packages listed in the "requirements.txt" into a folder. In the bundle, I'd for example have the full packages of "pymysql", "bs4" as well as all their dependencies.
I have absolutely no idea how to do this. Could you help me please? I am stuck and I am really struggling with this.
I am using Python 3.6
I am using "pip download -r requirements.txt" but it's not downloading the dependencies and outputs me only.whl files whereas I'm looking for "proper" folders..
To make pip prefer source files over wheels, use the --no-binary flag:
pip download -r requirements.txt --no-binary :all: -d /path/to/download/dir
I have a pip requirements file that changes during development.
Can pip be made to uninstall packages that do not appear in the requirements file as well as installing those that do appear? Is there a standard method?
This would allow the pip requirements file to be the canonical list of packages - an 'if and only if' approach.
Update: I suggested it as a new feature at https://github.com/pypa/pip/issues/716
This should uninstall anything not in requirements.txt:
pip freeze | grep -v -f requirements.txt - | grep -v '^#' | xargs pip uninstall -y
Although this won't work quite right with packages installed with -e, i.e. from a git repository or similar. To skip those, just filter out packages starting with the -e flag:
pip freeze | grep -v -f requirements.txt - | grep -v '^#' | grep -v '^-e ' | xargs pip uninstall -y
Then, obviously:
pip install -r requirements.txt
Update for 2016:
You probably don't really want to actually use the above approach, though. Check out pip-tools and pip-sync which accomplish what you are probably looking to do in a much more robust way.
https://github.com/nvie/pip-tools
Update for May, 2016:
You can now also use pip uninstall -r requirements.txt, however this accomplishes basically the opposite - it uninstalls everything in requirements.txt
Update for May, 2019:
Check out pipenv or Poetry. A lot has happened in the world of package management that makes this sort of question a bit obsolete. I'm actually still quite happily using pip-tools, though.
You can now pass the -r requirements.txt argument to pip uninstall.
pip uninstall -r requirements.txt -y
At least as of pip 8.1.2, pip help uninstall shows:
...
Uninstall Options:
-r, --requirement <file> Uninstall all the packages listed in the given requirements file. This option can be
used multiple times.
...
It's not a feature of pip, no. If you really want such a thing, you could write a script to compare the output of pip freeze with your requirements.txt, but it would likely be more hassle than it's worth.
Using virtualenv, it is easier and more reliable to just create a clean environment and (re)install from requirements.txt, like:
deactivate
rm -rf venv/
virtualenv venv/
source venv/bin/activate
pip install -r requirements.txt
The short answer is no, you can't do that with pip.
Here's a simple solution that works:
pip uninstall $(pip freeze) -y
This is an old question (but a good one), and things have changed substantially since it was asked.
There's an offhand reference to pip-sync in another answer, but deserves its own answer, because it solves precisely the OP's problem.
pip-sync takes a requirements.txt file as input, and "trues up" your current Python environment so that it matches exactly what's in that requirements.txt. This includes removing any packages that are present in your env but absent from requirements.txt.
Example: Suppose we want our env to contain (only) 3 libraries: libA, libB, and libC, like so:
> cat requirements.txt
libA==1.0
libB==1.1
libC==1.2
But our env currently contains libC and libD:
> pip freeze
libC==1.2
libD==1.3
Running pip-sync will result in this, which was our desired final state:
> pip-sync requirements.txt
> pip freeze
libA==1.0
libB==1.1
libC==1.2
Stephen's proposal is a nice idea, but unfortunately it doesn't work
if you include only direct requirements in your file, which sounds
cleaner to me.
All dependencies will be uninstalled,
including even distribute, breaking down pip itself.
Maintaining a clean requirements file while version tracking a virtual environment
Here is how I try to version-track my virtual environment.
I try to maintain a minimal requirements.txt, including only
the direct requirements, and not even mentioning version constraints where
I'm not sure.
But besides, I keep, and include in version tracking (say git),
the actual status of my virtualenv in a venv.pip file.
Here is a sample workflow:
setup virtualenv workspace, with version tracking:
mkdir /tmp/pip_uninstalling
cd /tmp/pip_uninstalling
virtualenv venv
. venv/bin/activate
initialize version tracking system:
git init
echo venv > .gitignore
pip freeze > venv.pip
git add .gitignore venv.pip
git commit -m "Python project with venv"
install a package with dependencies, include it in requirements file:
echo flask > requirements.txt
pip install -r requirements.txt
pip freeze > venv.pip
Now start building your app, then commit and start a new branch:
vim myapp.py
git commit -am "Simple flask application"
git checkout -b "experiments"
install an extra package:
echo flask-script >> requirements.txt
pip install -r requirements.txt
pip freeze > venv.pip
... play with it, and then come back to earlier version
vim manage.py
git commit -am "Playing with flask-script"
git checkout master
Now uninstall extraneous packages:
pip freeze | grep -v -f venv.pip | xargs pip uninstall -y
I suppose the process can be automated with git hooks, but let's not go off topic.
Of course, it makes sense then to use some package caching system
or local repository like pip2pi
you can create a new file with all installed packages
pip freeze > uninstall.txt
and then uninstall all of those
pip uninstall -r uninstall.txt -y
and then finally re-install the packages you had in your original requirements.txt file
pip install -r requirements.txt
Piggybacking off #stephen-j-fuhry here is a powershell equivalent I use:
pip freeze | ? { $_ -notmatch ((gc req.txt) -join "|") }
While this doesn't directly answer the question, a better alternative to requirements.txt now is using a Pipfile. This functions similarly to a Ruby Gemfile. Currently, you need to make use of the pipenv tool but hopefully this will eventually be incorporated into pip. This provides the pipenv clean command which does what you want.
(Note that you can import an existing requirements.txt with pipenv install -r requirements.txt. After this you should have a Pipfile and the requirements.txt can be removed.)
It is possible now using:
pip uninstall -r requirements.txt
I have a script that creates a virtualenv, installs distribute and pip in it and then optionally clones a git repo.
Now I have the project I will be working on, installed. But its dependencies are not installed. How can I make pip install all the dependencies as if I have issued a pip install MyApp?
EDIT: Appareantly my question is a duplicate of this one.
Not exactly sure but pip install -e . seems to do what I want without too many extra stuff lying around. I'd prefer if my code wasn't linked from site-packages though.
If your dependencies are defined in the setup.py file, you can first dump them to an external file using:
python setup.py egg_info
This will list all your dependencies in YOUR_PROJECT.egg-info/requires.txt file. Then you can install them using pip:
pip install -r *.egg-info/requires.txt
to delete what you just created:
rm -rf *.egg-info/
to save some time copy pasting:
python setup.py egg_info
pip install -r *.egg-info/requires.txt
rm -rf *.egg-info/
In my package root issuing pip install -e . installs dependencies.
To install your project's dependencies (i.e. install_requires + extra_requires) you have to extract your dependencies using setuptools egg-info and then install the filtered list of the combined dependencies:
python setup.py egg_info
pip install `grep -v '^\[' *.egg-info/requires.txt`
You should use the pip requirements file.
Essentially, place all your requirements, one in each line in a file and pass that to pip using the command
pip install -r requirements.txt
What more, if you have a standard environment, pip can actually dump such a file from existing installs using the command:
pip freeze
You can put the file thus generated directly into the pip requirements, and call the previous command from your deployment script.
Pretty cool, isnt it? :)
You can use pip-tools to create a requirements.txt that only contains the dependencies of your package:
$ pip-compile -o requirements.txt setup.py
Note that the command above only works if you do not already have a requirements.txt file. If you happen to have one already, just delete it.
Using the generated requirements.txt you can then run pip to install the dependencies:
$ pip install -r requirements.txt
Bonus 1:
The requirements.txt will include comments that indicate where the regarding dependency originates from.
Bonus 2:
If you have have an extras_require section for optional dependencies in your setup.py that looks e.g. like this:
...
extras_require={
"development": [
"wheel",
"debugpy",
"pytest",
],
},
...
You can create the requirements.txt including the optional dependencies by using:
$ pip-compile -o requirements.txt --extra development setup.py