i wanted to send a short code i was proud of to a fellow student and realized he won't be able to run it since there is no reason for him to have installed the library.
installing is of course super easy - but i realized this could happen often, mainly with beginners - wanted to build a simple function for it:
def smart_import(package_name):
try:
import package_name
except ImportError:
import pip
pip.main(['install', package_name])
problem is that i dont really know how to pass in the name of the package as a value that could be called by import
thought of converting a string back but that seems more complicated then i thought
This is a bad idea for many reasons, the main one being that people generally don't expect a Python function call to automatically attempt to install software on their machine.
Here are some other problems this approach has:
The import name is not always corresponding to the distribution name. For example dateutil module is provided by python-dateutil distribution.
If you try to use smart_import from another module, it will leave the name imported into the wrong namespace.
Some distributions export multiple top-level imports, e.g. setuptools provides setuptools, easy_install, and pkg_resources.
In some cases, pip itself may not be installed.
The script might not have permission to install packages.
The user might want to install distributions into their homedir explicitly with --user, and your script itself can't know that.
The invocation of pip might need to change depending on whether or not you are within a virtualenv.
The installation might attempt to pull in dependencies which cause conflicts with other distributions a user has installed.
IDEs likely won't see that a module dynamically imported is there, and may squiggly underline their subsequent use as a name which can not be resolved.
What to do instead:
Package your code up into its own distribution. Write a setup.py which mentions the dependencies using the install_requires argument to in the setup call. Ask your fellow student to pip install mypackage, and the dependencies will be collected at install time rather than at runtime. You can send your distribution directly to them (as a mypackage-0.1.tar.gz file or as a wheel). Or you can upload it to an index such as the test pypi.
The Python Packaging User Guide is a beginner-friendly resource describing how to create and upload your own distribution.
Related
I am creating a requirements.txt file for my Python project. When someone installs my project via pip; I want pip to also download the Tkinter and time Python modules, both of which are modules in the Python Standard Library.
I used the command pip freeze to list all of the currently installed Python packages, but when I scrolled down to the T section, I could not find Tkinter or time in the list. I then tried to install Tkinter via pip using the command pip install Tkinter but got the following error:
Could not find a version that satisfies the requirement Tkinter (from versions: )
No matching distribution found for Tkinter
I also did the same with time and got the same error. (I am running Python 3.6 and Windows 10.)
Is it possible to get pip to install those modules while pip is installing my package, or do I just have to trust that because the user has Python installed they will also have Tkinter and time installed?
All help is welcome and appreciated!
Python's Standard Library is called the Standard Library because it is a standard of Python. In other words, if there is no Standard Library installed, the python environment is not python at all.
The Standard Library is tested and released together with each Python release as part of this release (not as an addition or extension).
So, YES, you can expect these libraries to exist if the user has Python installed. Just by definition.
Regarding the upgrades of the built-in libraries: NO, you cannot do this. Because they are part of the python setup, not of the application environment. Python is very tightly bound to the specific code in those libraries. All python apps & libs expect the same behavior of those libraries, even if they are buggy.
In addition to that, you cannot install a module/package with the same name as one of the python's builtins, because it will create the ambiguity on import, and can confuse/break all other libraries which depend on it (or worse, the system applications if you install it into the system python).
However, in some cases you can find the backports of some of the libraries. Usually, they are backported from py3 to py2. Of course, their name is changed.
As an example, you can look into concurrent.features library, which is a handy builtin in py3.2+, but was absent in py2.7.
UPD: Though, as #JulienPalard hints in the comments, some OS distributions can split this standard library to simplify the binary dependencies: e.g., on Debian, Tkinter will be installable separately as python3-tk.
This makes sense, indeed, from the point of view of the binary OS packaging: it is not worth installing the UI parts of the python library if you have no UI at all and want to save the disk space.
However, you are still unable to install it via pip. Because this package is not packaged and available separately on PyPI. This standard library separation is made by the selected OS distributions and is resolved with the means of that OS distribution only.
pip installs packages from pypi, which does not expose the standard library, which is bundled into Python.
So in theory you should trust your users environment, if they have Python, they should have the whole stadard library.
But some distributions are splitting Python in a few packages for reasons (A minimal Debian already depends on Python, but they don't want Python to pull tk to pull libx11 to pull the life, the universe, and everything).
So in practice some package will be there, and you can trust the distribs for this, like time will always be here. And some package may not be here, like tkinter, in which case you may want to surround the import tkinter by a try to error a nice "Looks like your distribution does not provide tk by default, please install it.".
Be reassured, you won't have to surround every imports by try statements just in case some distribution splitted the stdlib, just tkinter.
I am currently developing a command line application in python, which I will be uploading to pypi for end users to pip install and use. I am taking advantage of the extras functionality in setup.py to support 2 versions of my application, one is a basic functionality version with minimal dependencies and the other is more feature rich but has a large amount of dependencies (numpy, pandas, networkx, matplotlib, etc)
So briefly:
pip install app # simple, no deps
pip install app[all] # all the deps
Now the problem is that one of my dependencies in the feature rich version has what has been described as "flakey" pypi support. Basically, it cannot be installed unless one of its dependencies is already pre-installed before the whole installation process occurs. Luckily (or not), my application (which pulls in this flakey module) also has the flakey modules needed module. Lets refer to the module that flakey module needs pre-installed fixer-module
So:
pip install app[all] # triggers the installation of flakey module
Installation will fail here if fixer-module is not installed, even though it will be installed before flakey module is. One must basically do this:
pip install fixer-module
pip install app[all]
Now what I would like to do is include some kind of checking code that accomplishes the following:
Runs only when the app[all] distribution is being installed
Does a try import fixer-module, except ImportError check and prints a message explaining the situation.
Stops and cleans up the installation process before it fails
I have been researching this for quite some time. I found some examples of checking the input args to setup.py and whatnot but none of them seem to cover how to handle stuff during the end user install process. Any pointers are much appreciated!
I have a virtualenv that serves a few separate projects. I'd like to write a utility library that each of these projects can use. It seems to make sense to stick that in the virtualenv. By all means shoot me down now but please give an alternative if you do.
Assuming I'm not completely crazy though, Where's the best place to stick my library?
My virtualenv sticks everything I install with pip in lib/pyton2.7/site-packages. I wonder if it would make more sense to follow suit or to hack in a separate home (further up the directory tree) so if things do ever clash, my work isn't overwritten by pip (et al).
If your project follows the standard packaging practices with setuptools, then all you have to do is run python setup.py develop inside the virtualenvs that you want the library to be used for. A .egg-link file will be created pointing to your package from which your other libraries will use much like any other packages, with the added benefit that your latest changes will be available to all packages at the same time (if that's your intention). If not, then either you could call python setup.py install or use multiple versions at different locations on the file system.
To get yourself started, take a look at Getting Started With setuptools and setup.py (you can skip the part on registering your package on pypi if this is your private work).
Another relevant stackoverflow thread: setup.py examples? The Hitchhiker's Guide to Packaging can also be quite useful.
Lets imagine I created new module. Why do I need to install it via setup file? I mean I can just add my module to PYTHONPATH variable and thats all. Thanks
For a simple one-file module, sure, that's enough.
But a setup.py file also lets you create a distribution, associate metadata with the distribution (author, homepage, description, etc.), register your package with the Python Package Index and most of all, lets you define what other packages might be needed to run your code. setup.py is not just for installing your module.
Installing a module with a setup.py based on setuptools also gives you additional functionality, such as support for namespaced packages (multiple distributions sharing a top-level name) and the ability to install multiple versions of a package side by side.
I would like to release a python module I wrote which depends on several packages. What's the easiest way to make it so these packages are programmatically downloaded just in case they are not available on the system that's being run? Most of these modules should be available by easy_install or pip or something like that. I simply want to avoid having the user install each module separately.
thanks.
pip uses requirements files, which have a very straightforward format.
For more Python packaging tooling recommendations, see the latest from the Python Packaging Authority (PyPA).
See the setuptools docs on how to declare your dependencies -- this will allow easy_install to find, download and install all of them (and transitive closure thereof) if everything's available in PyPi, or otherwise if you specify the dependencies' URLs.