I am working in a team and wrote some python code that uses libraries that need to be installed separately (because they are not part of standard python distribution). How should I specify those ? What is the right/correct/pythonic way to do this?
I personally use pip install -r requirements.txt
https://pip.pypa.io/en/latest/user_guide.html#requirements-files
Check out tool called pip. It's what most python projects use these days.
Typically, one would do the following (for example, we want to install the requests package for our new project):
pip install requests
and then
pip freeze > requirements.txt
Now, we have installed requests on our system and saved the dependency version to a file which we can distribute with our project.
At this point, requirements.txt contains:
requests==2.7.0
To install the same set of requirements (in our case only the requests package) on some other system, one would do the following:
pip install -r requirements.txt
You need to make a setup.py file for your package that specifies required packages. You may need to reorganize your file structure and include data and a manifest.
Then create a distribution of your package, EG: a wheel file.
Then when using pip install your_package_distro.whl, pip will determine your packages dependencies are met and install them from PyPI unless you specify another package source (EG: https://pypi.anaconda.org/)
Read through the following references to distribute your code:
Python 2.7.10 documentation - Distributing Python Modules, Section 2: Writing the Setup Script
Setuptools - Building and Distributing Packages with Setuptools
Hitchhiker's Guide to Packaging
Hitchhiker's Guide to Python - Packaging your Code
Related
Two options in setup.py develop and install are confusing me. According to this site, using develop creates a special link to site-packages directory.
People have suggested that I use python setup.py install for a fresh installation and python setup.py develop after any changes have been made to the setup file.
Can anyone shed some light on the usage of these commands?
python setup.py install is used to install (typically third party) packages that you're not going to develop/modify/debug yourself.
For your own stuff, you want to first install your package and then be able to frequently edit the code without having to re-install the package every time — and that is exactly what python setup.py develop does: it installs the package (typically just a source folder) in a way that allows you to conveniently edit your code after it’s installed to the (virtual) environment, and have the changes take effect immediately.
Note: It is highly recommended to use pip install . (regular install) and pip install -e . (developer install) to install packages, as invoking setup.py directly will do the wrong things for many dependencies, such as pull prereleases and incompatible package versions, or make the package hard to uninstall with pip.
Update:
The develop counterpart for the latest python -m build approach is as follows (as per):
From the documentation. The develop will not install the package but it will create a .egg-link in the deployment directory back to the project source code directory.
So it's like installing but instead of copying to the site-packages it adds a symbolic link (the .egg-link acts as a multiplatform symbolic link).
That way you can edit the source code and see the changes directly without having to reinstall every time that you make a little change. This is useful when you are the developer of that project hence the name develop. If you are just installing someone else's package you should use install
Another thing that people may find useful when using the develop method is the --user option to install without sudo. Ex:
python setup.py develop --user
instead of
sudo python setup.py develop
I have a Python script which uses open source third party libraries for geoprocessing (OGR and Shapely). My plan is to execute this script on a computer without having to install the required libraries.
I know that there are tools such as py2exe available for this purpose. However, compiling an executable file is not my first choice as I noticed that such files can get pretty large in size. Besides, I would like to use the code within another script. I would therefore like to create a portable python script which already includes the third party methods needed for executing.
Is it possible to include third party methods in a Python script in order to avoid the installation of third party libraries? And if not, what can I do instead, besides compiling an executable file? I work on Windows OS.
You can export your libraries using pip and embbed them into your application.
pip wheel requires the wheel package to be installed, which provides the "bdist_wheel" setuptools extension that it uses.
To build wheels for your requirements and all their dependencies to a local directory:
pip install wheel
pip freeze > requirements.txt
At this point check requirements.txt, clean it up, then you can download wheels in a local folder :
pip wheel --wheel-dir=/local/wheels -r requirements.txt
And then to install those requirements just using your local directory of wheels (and not from PyPI):
pip install --no-index --find-links=/local/wheels -r requirements.txt
Then though you'll need pip, though it's shipped with latest versions of python.
Check this : https://pip.readthedocs.io/en/latest/user_guide/#requirements-files
If your third party lib does not have any dependencies you can get the source files (.py) put into your project folder and use it as package by using import, else it has dependencies your project size grow more better create exe for your script.
I have a python project hosted on PyPI. I've discovered I have some dependency conflicts that need to be pinned. I know pip looks at the install_requires key in setup.py for dependencies, but I've read it's best to place pinned dependencies in a requirements.txt file. I've included this file (see below) using pip freeze, but I am unsure whether pip install project is sufficient to install dependencies as well.
# requirments.txt
numpy==1.9.2
pandas==0.16.2
I would like to make the simplest installation process for the user. For a package hosted on PyPI:
How do I setup requirements to simply pip install a project and include all of it's pinned dependencies automatically (similar to conda)?
Must install_requires=['numpy', 'pandas'] be included? If so, how do I best set it up to install the pinned versions only.
I have an external package I want to install into my python virtualenv from a tar file.
What is the best way to install the package?
I've discovered 2 ways that can do it:
Extract the tar file, then run python setup.py install inside of the extracted directory.
pip install packagename.tar.gz from example # 7 in https://pip.pypa.io/en/stable/reference/pip_install/#examples
Is if there is any difference doing them in these 2 ways.
On the surface, both do the same thing: doing either python setup.py install or pip install <PACKAGE-NAME> will install your python package for you, with a minimum amount of fuss.
However, using pip offers some additional advantages that make it much nicer to use.
pip will automatically download all dependencies for a package for you. In contrast, if you use setup.py, you often have to manually search out and download dependencies, which is tedious and can become frustrating.
pip keeps track of various metadata that lets you easily uninstall and update packages with a single command: pip uninstall <PACKAGE-NAME> and pip install --upgrade <PACKAGE-NAME>. In contrast, if you install a package using setup.py, you have to manually delete and maintain a package by hand if you want to get rid of it, which could be potentially error-prone.
You no longer have to manually download your files. If you use setup.py, you have to visit the library's website, figure out where to download it, extract the file, run setup.py... In contrast, pip will automatically search the Python Package Index (PyPi) to see if the package exists there, and will automatically download, extract, and install the package for you. With a few exceptions, almost every single genuinely useful Python library can be found on PyPi.
pip will let you easily install wheels, which is the new standard of Python distribution. More info about wheels.
pip offers additional benefits that integrate well with using virtualenv, which is a program that lets you run multiple projects that require conflicting libraries and Python versions on your computer. More info.
pip is bundled by default with Python as of Python 2.7.9 on the Python 2.x series, and as of Python 3.4.0 on the Python 3.x series, making it even easier to use.
So basically, use pip. It only offers improvements over using python setup.py install.
If you're using an older version of Python, can't upgrade, and don't have pip installed, you can find more information about installing pip at the following links:
Official instructions on installing pip for all operating systems
Instructions on installing pip on Windows (including solutions to common problems)
Instructions on installing pip for Mac OX
pip, by itself, doesn't really require a tutorial. 90% of the time, the only command you really need is pip install <PACKAGE-NAME>. That said, if you're interested in learning more about the details of what exactly you can do with pip, see:
Quickstart guide
Official documentation.
It is also commonly recommended that you use pip and virtualenv together. If you're a beginner to Python, I personally think it'd be fine to start of with just using pip and install packages globally, but eventually I do think you should transition to using virtualenv as you tackle more serious projects.
If you'd like to learn more about using pip and virtualenv together, see:
Why you should be using pip and virtualenv
A non-magical introduction to Pip and Virtualenv for Python beginners
Virtual Environments
python setup.py install is the analog of make install: it’s a limited way to compile and copy files to destination directories. This doesn’t mean that it’s the best way to really install software on your system.
pip is a package manager, which can install, upgrade, list and uninstall packages, like familiar package managers including: dpkg, apt, yum, urpmi, ports etc. Under the hood, it will run python setup.py install, but with specific options to control how and where things end up installed.
In summary: use pip.
The question is about the preferred method to install a local tarball containing a python package, NOT about the advantage of uploading package to an indexing service like PyPi.
As lest I know some software distributor does not upload their package to PyPi, instead asking developers to download package from their website and install.
python setup.py install
This can work but not recommended. It's not necessary to unwrap the tarball file and go into it to run setup.py file.
pip install ../path/to/packagename.tar.gz
This is the way designed and preferred. Concise and align with PyPi-style packages.
More information about pip install can be found here: https://pip.readthedocs.io/en/stable/reference/pip_install/
I had one machine with my commonly used python package installed.
and i would like to install the same package on another machine or same machine with different python version. I would like to know whether pip or easy-install or some other method can let me install those packages in a batch. When i use perl, it has something like a bundle package, how to do that in python?
Pip has some great features for this.
It lets you save all requirements from an environment in a file using pip freeze > reqs.txt
You can then later do : pip install -r reqs.txt and you'll get the same exact environnement.
You can also bundle several libraries into a .pybundle file with the command pip bundle MyApp.pybundle -r reqs.txt, and later install it with pip install MyApp.pybundle.
I guess that's what you're looking for :)
I keep a requirements.txt file in one of my repositories that has all my basic python requirements and use PIP to install them on any new machine.
Each of my projects also has it's own requirements.txt file that contains all of it's dependencies for use w/virtualenv.