A Python module is just a .py source file. A Python package is simply a collection of modules.
So why do we need programs such as pip to 'install' Python modules? Why not just download the files, put them in our project's folder and import them?
What exactly does it mean to 'install' a module or a package? And what exactly does pip do?
Are things different on Windows and on Linux?
So why do we need programs such as pip to 'install' Python modules? Why not just download the files, put them in our project's folder and import them?
It's just meant to facilitate the installation of softwares without having to bundle all the dependencies nor ask the user to download the files.
You can type pip install mysoftware and that will also install the required dependencies. You can also upgrade a software easily.
What exactly does it mean to 'install' a module or a package? And what exactly does pip do?
It will copy the files in a directory that is in your Python path. This way you will be able to import the package without having to copy the directory in your project.
With your proposal, for each and every project you have to download the required modules as dependencies. You have to download them again and again and add them with your project which is not very suitable though some platform like node.us do it.
What pip do is to keep the modules you installed in /use/lib/python*/site-packages/ so clearly it is included in your Python's path. So, when you try to import a module or package it checks in site-package if it exists. If exists,then this code will be used with your project. If not, you will get an error.
Related
I am quite new on python environment and on pyinstaller, but I really need your help.
Context
I have a python app which loads plugins within code. Indeed, we provide new features for our users in the form of plugins, and we want them to be able to install and use those that interest them. We don't want to build a new executable file every new plugin.
We bundled our app in a executable with pyinstaller, a one folder, because it seems easier to add plugins.
pyinstaller --clean --paths=lib/python3.8/site-packages src/main.py
My plugins have sometimes external dependencies (lib, etc.), so foreach plugin here is my setup.py (with requirements, values from requirements.txt file) :
setup(name='package_name',version='0.0.1',include_package_data=True,install_requires=requirements,extras_require={'all': requirements})
My issue
I know how to import installed modules dynamically to my code:
module=__import__(module_name,globals(),locals(),[name],0)
This part works.
But I have difficulties to install completely and dynamically those plugins within my code.
The solution must work on a linux ans Windows machine.
What I tried
Let's imagine I want to import my test.whl plugin, which has a dependency with 'test2' package
Install the plugin with pip:
subprocess.run(["pip", "install", "--upgrade", package, "-t", my_one_folder_location])
Pro: if python and pip are installed on the Host machine it works
Cons : You need to have python and "pip" (not "pip3" etc) on the user computer and the path. You have to be careful to have the same python version than on the package, etc. Not a recommanded solution...
I also tried to use the python installed in the pyinstaller folder but did not succeed.
Import every plugins in a specific directory:
PyInstaller understands the “egg” distribution format often used for Python packages. If your script imports a module from an “egg”, PyInstaller adds the egg and its dependencies to the set of needed files.
I tried to put my .wheel packages in a folder "egg". Then my program detect the plugin, but it doesn't import my lib dependencies
__main__:Unexpected error: no module named 'test2'
I'm going around in circles and can't find any solutions: What is the best way to import my external .whl packages and all it's dependencies in a pyinstaller solution?
Recently, in I download and install Python 3.10, then uninstall Python 3.7 in win10.However, when I want to import some modules in script, it shows there is no modules. After checking, I found all modules are still saved in C:\Users\...\AppData\Roaming\Python\Python37
So if I want to use them, just copy/paste them into Python310? Need I change other things, like classpath...
If not, what should I do? download from Internet again?
Don't copy site site-packages dir from one version of Python to another - at best you'll get undefined behaviour. The appropriate way of getting the modules you want from v3.7 to v3.10 is to run pip list from the v3.7 installation to see what you have installed, then pip freeze to write that to a file, and finally use the v3.10 pip to install from the file that you wrote the list to.
This way you will get the modules built with and for v3.10.
In short, my question is, how do I install the latest version of scikit-image into my usr/lib/python3/dist-packages so I can actually use it? I think there is a problem with my understanding of how third-party modules are installed. As a newb, I don’t know how to rectify that, hence this post.
I need help to understand how to install packages in python3 up until now I have used pip/pip3/apt-get/synaptic etc and it has worked fine for many packages. However, I have hit several barriers (Skimage, opencv, plantcv in python3). I must emphasise, the problem I am having is using these packages in python3, not 2.7.
For example, I want to use the latest version of scikit-image (0.14) with python3. (http://scikit-image.org/) I have tried using the installation instructions and have not yet successfully managed to install it. I have navigated to my usr/lib/python3/dist-packages and copied scikit-image into this directory (I have all the dependencies installed in here already).
Image of my folder for dist-packages as proof
As you can see, the folder containing skimage is in the directory I want to be installed in, how do I actually install it? Do I have to extract skimage out of the folder into the directory and then run the install command? If I navigate to usr/lib/python3/dist-packages/scikit-image and then run pip install -e . I get an error, stating that I need numpy. If I write a python script using python3 I can clearly see I have it installed (and I have been using it for a long time). So, there must be a problem in how I have this package in my file system. I think a janky workaround would be to copy all the modules into my working directory and Import them that way as if they were modules I have made myself, but this obviously negates the whole point of installing packages.
This has also happened with another package called plantcv. Where I went into the directory usr/lib/python3/dist-packages then cloned the source from git hub and installed as per instructions. When I import plantcv in my python3 script. It Imports fine. But, there is nothing in it, as python cannot see the modules which are inside this folder at usr/lib/python3/dist-packages/plantcv/plantcv.
There is clearly some comprehension here that I am missing, as I have a similar problem for two packages now. Please, Internet. Help me understand what I am missing!
You simply need to copy the folder in /usr/lib/python3/dist-packages/package-name
However, there are certain things that are specific to python packages. The folder named package name should be a valid package. A good indicator of that is it will contain a file "__init__.py". It is very likely that every sub-directory inside this package directory will contain a "__init__.py" file. It depends on whether there are modules inside these sub-directories.
In your code simply import the package like the following.
import package-name
where package-name can be skimage
Why we have to install the python packages before using them?
I am currently working on a small python mysql program. What i tried to download the python connector module from mysql webpage and simply unzip it and place it in the same folder of my code.
And I can import the module properly.
So what is the meaning of installing those packages? Can I use those packages like matplotlib, numpy without installing them ?
Is it possible to have all the required packages installed on a folder so that i can move it to another computer and run my program with only CPython installed (I don't want to install any package on this computer)?
it's not that simple :-)
some packages have dependencies, you also need to download and extract their dependencies (you need pacakge x,and package x uses y) pakcage manager handles that
some package have some c code (they need to be compiled before use (ujson or postgres module) package manager handles that
when your share your code instead of sharing dependencies you simply add a file containing the list of dependencies (requirements.txt) and other user can simply install all dependencies using package manager
Installing a python package enables us to use it anywhere on our system. If we just place the package in the same directory as our script then it may well work, but only for scripts in that directory.
Some packages also rely on others to function properly, and the installation of a package may well install those pre-requisite packages for you. You may be able to do this manually, but you'd have to put them all in the same directory as your script every time you wanted to run it.
So installing the packages is the easiest way to use them.
You don't have to install them, and in some cases you wouldn't install them on your system; if you had split your code across two files and imported one file at the top of the other for example.
In fact, you don't really need install package on your system.
But if you install it, you can use these packages every where on your system.
Also, you can create a requirement.txt file to enable install all packages that you need on other computer. You can check this manual https://pip.pypa.io/en/stable/user_guide/#requirements-files
Previously, I have been using Python 2.7. If I installed a module using pip, it would place the module in /usr/local/lib/python2.7/site-packages. Then, if I were to import that module in a python script, the site-packages directory would be searched to find that module.
However, I want to start using the Anaconda distribution. After downloading this, I notice that there are a number of packages located at /home/karnivaurus/Anaconda/pkgs, for example scikit-learn. Then, in the directory of each packages, there is the directory structure lib/python2.7/site-packages, which contains the modules for that package.
So, it seems that I have now gone from the situation where I only had one site-packages directory, to where I have a number of site-packages directories, one for each package.
My question is: When I create a python script, and want to import a module, how does python know where to look for these modules? Will it look in /usr/local/lib/python2.7/site-packages as well as the site-packages directories that come with Anaconda? What if I want to install another package that does not come with Anaconda -- where should this be installed to?
Thanks!
The paths in which Python will import packages can be seen with the following command:
python -c "import sys; print sys.path"
Please see the documentation, also (as stated in the documentation) if you need to modify the search path, look into PYTHONPATH.