How to pip freeze source package - python

I am learning how to use venv here: https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/#installing-from-source
And it says I can install a source package by:
python3 -m pip install .
Which works, but now if I do pip freeze then I see:
my-package # file:///Users/joesmith/my-package
The problem is if I export this to a requirements.txt and try to install this environment on another machine then it won't work cause the path to source changed obviously.
What is the proper way to use a source package locally like i did but also export it afterwards so that another person can recreate the environment/run the code on another machine?

Pip has support for VCS like git. You can upload your code to git (e.g. Github, Gitlab, ..) for example and then use the requirements.txt. like this:
git+http://git.example.com/MyProject#egg=MyProject
https://pip.pypa.io/en/stable/cli/pip_install/#vcs-support

You would install package from PyPI rather than from source.
i.e. pip install requests
In this way other developers will also easily run your project.

Related

What is the use of `pip install -e .` if I can simply run the python script using the environment?

Based on this answer, I can fully understand the use of:
pip install -e /path/to/locations/repo
However, I am yet to see the use of:
pip install -e .
I can understand it from the perspective of doing pip install -e /path/to/locations/repo, but from the working directory of the project dependency. But that's the only use case I can see.
In what use case would I want to install locally the same package I am now working on?
pip install -e
will just create a projekt_name.egg-info file in the venv\Lib\site-packages folder with a link to the repo location. Nothing is copied.
You can continue developing and you can access your project packages as if the repo was properly installed. No dirty sys.path.append-hacks needed.

pip3 installs modules to location python3 can't find

I have pip3, installed via the yum install of python3-pip.
I've done a pip3 global install of some modules I need, but python3 can't find them to import. After a little investigation I see that pip3 installed the modules to /usrlib/python3.6/site-packages/pip/_vendor/
The problem is that python3 doesn't seem to know to look at pip/_vendor, it only finds modules directly installed under site-package. If I just copy the modules from .../site-package/pip/_vendor to .../site-package everything works fine.
The issue doesn't appear to be related to file permissions or ability to read the modules.
I'm wondering how I configure either pip to install directly to site-package or python3 to understand how to look in the pip/_vendor location.
I'm configuring this all with ansible and would like as module an option as possible. For instance I could manually use an argument to tell pip3 to install to the folder I want, but I don't want to hardcode the exact site-package directory if I don't have to.
I recommend starting over with pip by downloading and running get-pip.py. This will not only install the latest version of pip, but it will also install packages to a Python-readable location (the version of Python you use to run get-pip.py).
As an aside, I would avoid installing packages system-wide unless there is a specific need for them. At the very least, you should be installing them as a regular user, and even better you should be using a virtualenv.

Pip install binary and preserve requirements.txt

I'm making a Python package dependent on spacy. Spacy works with binary language models. So I have the URLs listed at the end of my requirements.txt
https://github.com/explosion/spacy-models/releases/download/es_core_news_sm-2.0.0/es_core_news_sm-2.0.0.tar.gz#egg=spacy-english-model
But if I freeze the environment the package does not appear with the URL for download:
spacy-english-model==2.0.0
So if I add a package I can't pip install it and then pip freeze. How can specify the package in requirements.txt so that its URL shows up when freezeing?
You don't need to use pip freeze to distribute your package. When writing a package you'll need to add the models in your requirements.txt file, according to the documentation like so:
https://github.com/explosion/spacy-models/releases/download/es_core_news_sm-2.0.0/es_core_news_sm-2.0.0.tar.gz
you can see that happened here (check the dev-requirements.txt)
I don't know where did pip freeze get the spacy-english-model from. I'd start with a new python virtualenv and test everything again.

Python equivalent of npm or rubygems?

I've been looking around for a package manager that can be used with python. I want to list project dependencies in a file.
For example ruby uses Gemfile where you can use bundle install.
How can I achieve this in Python?
The pip tool is becoming the standard in equivalent of Ruby's gems.
Like distribute, pip uses the PyPI package repository (by default) for resolving and downloading dependencies.
pip can install dependencies from a file listing project dependencies (called requirements.txt by convention):
pip install -r requirements.txt
You can "freeze" the current packages on the Python path using pip as well:
pip freeze > requirements.txt
When used in combination with the virtualenv package, you can reliably create project Python environments with a project's required dependencies.
Pipenv
(I know it's an old question, and it already has an answer but for anyone coming here looking for a different answer like me.)
I've found a very good equivalent for npm, It's called pipenv. It handles both virtualenv and pip requirements at the same time so it's more like npm.
Simple Use Case
pip install pipenv
then you can make a new virtualenv with third version of python, as well as making a pipfile that will be filled with your projects requirement and other stuff:
pipenv install --three
using your created virtualenv:
pipenv shell
installing a new python package:
pipenv install requests
running your .py file is like:
pipenv run python somefile.py
you can find it's doc here.
Python uses pip for a package manager. The pip install command has a -r <file> option to install packages from the specified requirements file.
Install command:
pip install -r requirements.txt
Example requirements.txt contents:
Foo >= 1.2
PickyThing <1.6,>1.9,!=1.9.6,<2.0a0,==2.4c1
SomethingWhoseVersionIDontCareAbout
See the Requirements Parsing section of the docs for a full description of the format: https://pip.pypa.io/en/stable/user_guide/#requirements-files
This is how I restrict pip's scope to the current project. It feels like the opposite if you're coming from NodeJS's npm or PHP's composer where you explicitly specify global installations with -g or --global.
If you don't already have virtualenv installed, then install it globally with:
pip install virtualenv
Each Python project should have its own virtualenv installation. It's easy to set one up, just cd to your project's root and:
python3 -m virtualenv env # creates env folder with everything you need
Activate virtualenv:
source env/bin/activate
Now, any interaction with pip is contained within your project.
Run pip install package_name==version for each of your dependencies. They are installed in ./env/lib/python3.x/site-packages/
When you want to save your project's dependencies to a file, run:
pip freeze > requirements.txt
You actually don't need -l or --local if you're in an activated project-specific virtualenv (which you should be).
Now, when you want to install your dependencies from requirements.txt, set up your virtualenv, and run:
pip install -r requirements.txt
That's all.
This is an old question but things are constantly evolving.
Further to the other answer about pipenv. There is also a python package manger called poetry.
There is a detailed comparison between pipenv and poerty here: Feature comparison between npm, pip, pipenv and poetry package managers. It also links the features to common npm features.
Here is a comparison of pipenv vs poetry vs pdm: https://dev.to/frostming/a-review-pipenv-vs-poetry-vs-pdm-39b4
The conclusion is that pdm is the winner.
But in my experience, poetry is easier than pdm to integrate with IDEs.

Any productive way to install a bunch of packages

I had one machine with my commonly used python package installed.
and i would like to install the same package on another machine or same machine with different python version. I would like to know whether pip or easy-install or some other method can let me install those packages in a batch. When i use perl, it has something like a bundle package, how to do that in python?
Pip has some great features for this.
It lets you save all requirements from an environment in a file using pip freeze > reqs.txt
You can then later do : pip install -r reqs.txt and you'll get the same exact environnement.
You can also bundle several libraries into a .pybundle file with the command pip bundle MyApp.pybundle -r reqs.txt, and later install it with pip install MyApp.pybundle.
I guess that's what you're looking for :)
I keep a requirements.txt file in one of my repositories that has all my basic python requirements and use PIP to install them on any new machine.
Each of my projects also has it's own requirements.txt file that contains all of it's dependencies for use w/virtualenv.

Categories