When creating an executable file using PyInstaller, how can I bundle a specific Python interpreter and some library from specific versions?
I want to create an executable file from my .py script. My __main__.py downloads some data from a website and transform them, then saves them in a folder as excel files.
And here are requirements.
Use Python 3.6
For this, I found some related posts but never clear to me. My main.py needs Python3.6 to be executed, and the users of this .exe file won't necessarily have Python3.6, so I want to bundle this Python in the .exe file as well. But is just a single command python3 -m pyinstaller __main__.py really enough? I'm not confident and I want to know what exactly I need to do.
Use specific versions of libraries such as pandas==0.23, beautifulsoup4==4.9.1 etc.
I think I need to work with pure in .spec file as the documentation says "pure: pure python modules needed by the scripts". But I cannot find any documentation about what exactly I need to do make it include pandas==0.23, beautifulsoup4==4.9.1 etc.
Any help would be appreciated!
I do this using virtual environments: if you have an environment with the desired Python version and package versions, you can pip install PyInstaller there and build the .EXE from that environment.
The Conda version would be:
Create the environment:
conda create --name my_env_name python=3.6
Activate it:
conda activate my_env_name
Install your desired package versions and PyInstaller:
pip install pandas==0.23
pip install beautifulsoup4==4.9.1
pip install pyinstaller
# or to get the developer version, which fixes some issues I have run into
# pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip
And then build your EXE:
python pyinstaller myscript.py
Having this virtual environment can be a nice control as well for writing/testing your program, as it ensures no other dependencies are needed. Because of this, I tend to only install the necessary packages for the script into the environment.
However, I am not sure how one would do this with venv, if you do not have Anaconda. But maybe this will still point you in a direction (or someone else can elaborate? also see this post).
Related
I have a Windows 7 machine running Python 3.8.5 with a very large number of physics/electronics/data analysis/simulation packages. As it turned out, I must have - for some inexplicable reason - installed the 32-bit version of Python instead of the 64-bit one despite having a 64-bit system. And I didn't notice until very recently when I was trying to install some packages that require 64-bit Python. Hence I've now downloaded and installed the latest Python version that is supported by Windows 7, which seems to be 3.8.10.
Question: What is the easiest and also fail-safe way to reinstall all the user packages - that I currently have under 3.8.5 - to 3.8.10?
For some reason, I couldn't find any "canonical" solution for this online. As it seems, Python does not come with any built-in support for updating or system migration and I'm honestly wondering why...
Anyway, my first idea was to get a list of all user (= "local"?) packages currently installed under 3.8.5, but I don't know how. Reason: Doing help('modules') inside the interpreter will list all packages and I don't see a way to "selectively apply" pip to a specific Python version, e.g. something like python-3.8.5 -m pip list --local is not supported.
After getting a list of the user packages, I was thinking to pack it into a batch command pip install package_1 package_2 <...> package_N, thus reinstalling everything to Python 3.8.10. And afterwards uninstalling Python 3.8.5 and removing all environment variables from system PATH.
Is this the proper way to do this?
Anyway, my first idea was to get a list of all user (= "local"?) packages currently installed under 3.8.5, but I don't know how.
Create a list of installed packages with pip freeze > pkglist.txt or pip list --format=freeze. If you already have one, that's great.
Then uninstall 32-bit Python 3.8.5 and clean your path for all Python related variables. Now, install 64-bit Python 3.8.10.
After reinstalling, you can install back all the packages with pip install -r pkglist.txt and it will restore the exact versions of the packages.
If you insist on having both 32-bit and 64-bit versions installed and also have the Python Launcher installed, you could invoke 32 and 64 bit versions separately with py -3.8-64 -m pip and py -3.8-32 -m pip.
I don't see a way to "selectively apply" pip to a specific Python version.
This is possible with the Python Launcher on Windows. But only between major/minor versions and not the patch versions according to its help message.
I would also recommend creating a virtual environment this time before installing the packages and leaving the root environment alone. You can create one named venv with just python -m venv venv, activate it with ./venv/Scripts/activate and proceed with the installation of packages.
Nope, doesn't work. After installing the packages with the newer Python version in PATH, e.g. Jupyter won't start.
If the Jupyter error persists, you could try pinning packages to their most recent patch/minor versions to update them and yet not break your code.
As a last resort, you could try installing Python 3.10 alongside your current Python installation (without uninstall or editing the PATH) and then installing the absolute latest versions of the packages in a 3.10 virtual environment to see if it works for you. You would invoke the two versions with Py Launcher, e.g. py -3.10 and py -3.8.
If I understood correctly, you have multiple packages like NumPy, pandas etc. installed on your machine, and you want to reinstall them "automatically" on a fresh installation of python.
The method (I use) to perform such an operation is by creating a file named setup.py which includes a list of all the packages.
Bellow, I am attaching an example of such a file I use in one of my projects:
from setuptools import setup, find_packages
setup(
name='surface_quality_tools',
version='0.1',
install_requires=["matplotlib", "psutil", "numpy", "scipy", "pandas", "trimesh", "pyglet", "networkx", "protobuf",
"numpy-stl", "sklearn", "opencv-python", "seaborn", "scikit-image", "flask", "tqdm", "pytest"],
package_data={'': ['*.json']},
packages=find_packages(include=[])
)
to run the installation you should open a command prompt from inside the project directory and run:
pip install -e .
You can find a nice example in this blog page
One common way of handling packages in Python is via virtual environments. You can use Anaconda (conda), venv or any of several other solutions. For example, see this post:
https://towardsdatascience.com/virtual-environments-104c62d48c54#:~:text=A%20virtual%20environment%20is%20a,a%20system%2Dwide%20Python).
The way this works in by keeping the Python interpreter separate from the virtual environment that contains all the necessary packages.
Probably the main reason Python doesn't feature migration tools (at least as part of standard library) is because pip - the main package tool - doesn't handle conflict resolution all too well. When you update a version of Python it might so happen (especially with niche packages) that some of them won't work any more and pip often won't be able to solve the dependencies. This is why it's a good idea to keep a separate venv for different Python versions and different projects.
The other tool you could use for easy migration is Docker which is a semi-virtual machine working on top of your host OS and containing usually some linux distribution, Python along with the necessary packages necessary for running and development.
It takes a bit of time to set up a container image initially but afterwards setting everythin on a new machine or in the cloud becomes a breeze.
Listing currently installed packages is done via pip freeze command, the output of which you can then pipe into a file to keep a record of project requirements, for example pip freeze > requirements.txt.
I have some issues with a published package and wish to edit the code myself (may generate a pull request later to contribute). I am quite confused about how to do this since it seems there is a lack of step-by-step guidance. Could anybody give me a very detailed instruction about how this is done (or a link)? My understanding and also my questions about the workflow are:
Fork the package through git/github and have a local synced copy (done!).
Create a new Anaconda environment (done!)?
Install the package as normal: $conda install xxx or $python setup.py develop?
Do I make changes to the package directly in the package folder in Anaconda if I use python setup.py develop?
Or make changes to the local forked copy and install/update again and what are the commands for this?
Do I need to update the setup.py file as well before running it either way?
You can simply git-clone the package repo to your local computer and then install it in "development" or "editable" mode. This way you can easily make changes to the code while at the same time incorporating it into your own projects. Of course, this will also allow you to create pull requests later on.
Using Anaconda (or Miniconda) you have 2 equivalent options for this:
using conda (conda-develop):
conda develop <path_to_local_repo>
using pip (pip install options)
pip install --editable <path_to_local_repo>
What these commands basically do is creating a link to the local repo-folder inside the environments site-packages folder.
Note that for "editable" pip installs you need a a basic setup.py:
import setuptools
setuptools.setup(name=<anything>)
On the other hand the conda develop <path_to_local_repo> command unfortunately doesn't work in environment.yml files.
I'm using pyinstaller to pack one .py file into .exe. I have two Python3 environment on my WIN10 64-bit computer. One is from Anaconda and the other is Python3-32bit which is installed separately. This .py file is produced by Anaconda Python environment.
After I installed pyinstaller, I set the environment variable PATH like this:`
PATH=PATH;C:\Anaconda3\Scripts;C:\Python35-32\Scripts;
However, when I enter pyinstaller file.py it throws me lots of warnings:
It seems that pyinstaller searches dependencies in Python35-32's lib folder rather than Anaconda's. How can I make it searche dependencies under Anaconda's lib folder?
`
Did you install pyinstaller using pip or using (ana)conda? if so, check whether pip is the python3-32bit or the anaconda one with:
pip -V
This will tell you if pip, and hence pyinstaller, was installed on the 32-bit interpreter, hence why it's looking in that particular folder.
solution would be to install pyinstaller using conda or within the anaconda IDE. Probably would need to uninstall pyinstaller from python3-32bit
At work we have python installed, but no additional modules. I want to import some scipy modules but I have no access to the python directory for installation.
Similar questions have been asked on StackOverflow, but the answers always assumed easy install, pip or virtualenv were installed. At my workplace, none of these packages are installed. It's just the plain python installation and nothing else.
Is there still an option for me for installing modules in my local folder and calling them from python? If so, how do I go about it?
Not exactly installing modules on your local folder, but a solution nonetheless:
I used to work for a company that used windows and didn't have admin access, so I ended up using Portable python.
It seems portable python is no longer mantained, but you can see some other portable python solutions on their site, most of which you can run straight from your usb.
You can download pip from here http://pip.readthedocs.org/en/stable/installing/ and install it without root privileges by typing:
python get-pip.py --user
This will install to directory with prefix $HOME/.local so the pip executable will be in the directory $HOME/.local/bin/pip, for your convenience you can add this directory to $PATH by adding to end of .bashrc file this string
export PATH=$HOME/.local/bin/:$PATH
After this you can install any packages by typing
pip install package --user
Or you can alternatively compile the python distribution from source code and install to your home directory to directory $HOME/.local or $HOME/opt or any subfolder of $HOME you prefer, let's call this path $PREFIX. For doing this you have to download python source code from official site, unpack it and then run
./configure --prefix=$PREFIX --enable-shared
make install
And then add python binary to $PATH, python libraries to $LD_LIBRARY_PATH, by adding to the end of $HOME/.bashrc file whit strings
export PATH=$PREFIX/bin:$PATH
export LD_LIBRARY_PATH=$PREFIX/lib
and when after restarting bash you can also run
python get-pip.py
and pip and will be installed automatically to your $PREFIX directory. And all other packages those you will install with pip will be automatically installed also to $PREFIX directory. This way is more involved, but it allows you to have the last version of python.
I am new to python just a few weeks back i started using python(Classic Noob-Disclaimer)
Now whenever i install a module by copying the unzipped folder in site-packages under Lib and running the source install by using "c:\python27\lib\site-packages\tweepy-1.2\setup.py install" in command prompt it installs without any errors.
But now when i make a python script (*.py)
and store it on the desktop it wont work
and it gives out an error "No module found"
but when i store it in the same folder as the source it works perfectly.
also if i open the IDLE GUI it also returns the same error.
But this doesnt happen with the win32com module which i use for TTS.
I missing something..but i cudnt find the answer to it.
Plz help me!
i need to use many of these modules..they work great differently but not together as the modules are always missing!
Copying an unzipped folder to site-packages does not install a Python package.
To install manually, unzip the package to a temporary directory, then run:
python setup.py install
in this directory, after that you can remove the directory.
To download and install a pure Python package automatically, run:
pip install tweepy
if you have pip installed.
The simplest way to install Python packages that have C extensions is to use binary installers (*.exe, *.msi files).
To avoid all this use VirtualEnv
Virtualenv is a tool to create isolated Python environments.
The basic problem being addressed is one of dependencies and versions, and indirectly permissions. Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How can you use both these applications? If you install everything into /usr/lib/python2.7/site-packages (or whatever your platform's standard location is), it's easy to end up in a situation where you unintentionally upgrade an application that shouldn't be upgraded.
Or more generally, what if you want to install an application and leave it be? If an application works, any change in its libraries or the versions of those libraries can break the application.
Also, what if you can't install packages into the global site-packages directory? For instance, on a shared host.
The easiest way to install python packages is by using pip. First you need to install pip as explained here if you use windows. Then you can query some packages, from command line, for example
> pip search twitter
Then to install certain packages, just use pip something like this:
> pip install tweepy