We Keep Coding

ImportError: No module named 'distutils' in pandas pyinstaller

I have created an executable via pyinstaller. While running the exe found the error from pandas.
Traceback (most recent call last):
File "score_python.py", line 3, in <module>
import pandas as pd, numpy as np
File "d:\virtual\sc\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "site-packages\pandas\__init__.py", line 23, in <module>
File "d:\virtual\sc\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "site-packages\pandas\compat\__init__.py", line 32, in <module>
ImportError: No module named 'distutils'
Has anyone found the same?

This is an issue with virtualenv from version 16.4.0 onward, as indicated in the following issue on github:
https://github.com/pyinstaller/pyinstaller/issues/4064
These workarounds were suggested:
In the .spec file, at the line “hiddenimports=[]”, change to "hiddenimports=['distutils']", then run pyinstaller using the spec file.
Tried this, but it didn't work in my case, now distutils module could be found, but it threw an error while importing the module.
Downgrade virtualenv to an earlier version.
I downgraded virtualenv to version 16.1.0 and and recreated the execution bundle. The new execution file worked alright in my case.

Found the solution, it's because of the virtual environment.
The error occurred because of the creation of a new virtual environment while creating the project. I have deleted my existing virtual and created new virtual by setting up the python interpreter and opting the pre-existing interpreter option.
The IDE will create a virtual named venv and copies all the python files from Python/bin to this folder and then import modules from here, by activating the same solved my issue.

Trouble Generating .exe File Using Pyinstaller [duplicate]

I am trying to wrap a Python script into an exe using PyInstaller (development version) for windows.
The script uses Pandas and I have been running into an error when running the exe.
Traceback (most recent call last): File "site-packages\pandas\__init__.py", line 26, in <module> File "C:\Users\Eddie\Anaconda3\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 631, in exec_module
exec(bytecode, module.__dict__) File "site-packages\pandas\_libs\__init__.py", line 4, in <module> File "C:\Users\Eddie\Anaconda3\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 714, in load_module
module = loader.load_module(fullname) File "pandas/_libs/tslib.pyx", line 1, in init pandas._libs.tslib ModuleNotFoundError: No module named 'pandas._libs.tslibs.timedeltas'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "G5k Version file Extract (with tkinter).py", line 15, in <module> File "C:\Users\Eddie\Anaconda3\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 631, in exec_module
exec(bytecode, module.__dict__) File "site-packages\pandas\__init__.py", line 35, in <module> ImportError: C extension: No module named 'pandas._libs.tslibs.timedeltas' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.
I have tried doing this for programs without pandas and everything was fine.
This is very similar to another question already solved for Python 2, but I am using Python 3 and that solution does not apply the same way due to the changed .spec file format.
Python 3.6
PyInstaller - version 3.3
Pandas - version 0.20.3

PyInstaller 3.3, Pandas 0.21.0, Python 3.6.1.
I was able to solve this thanks to not-yet published/committed fix to PyInstaller, see this and this. AND keeping the ability to pack it into one executable file.
Basically:
Locate PyInstaller folder..\hooks, e.g. C:\Program Files\Python\Lib\site-packages\PyInstaller\hooks.
Create file hook-pandas.py with contents (or anything similar based on your error):
hiddenimports = ['pandas._libs.tslibs.timedeltas']
Save it + I deleted .spec file, build and dist folders just to be sure.
Run pyinstaller -F my_app.py.
This fix should work as long as you don't upgrade or reinstall PyInstaller. So you don't need to edit .spec file.
Maybe they will include the fix sooner for us! :)

I'm not sure it may help you but following the solution on the post you mention work for me with python 3.6 pyinstaller 3.3 and pandas 0.21.0 on windows 7.
So adding this to the spec file just after analysis :
def get_pandas_path():
import pandas
pandas_path = pandas.__path__[0]
return pandas_path
dict_tree = Tree(get_pandas_path(), prefix='pandas', excludes=["*.pyc"])
a.datas += dict_tree
a.binaries = filter(lambda x: 'pandas' not in x[0], a.binaries)
Also my spec file format is the same as the one in the post you mention.

I managed to solve this problem by using the "--hidden-import" flag. Hopefully this can be helpful to someone else that comes across this thread.
pyinstaller --onefile --hidden-import pandas._libs.tslibs.timedeltas myScript.py

If you are using Anaconda, it is highly likely that when you were trying to uninstall some package it has disrupted pandas dependency and unable to get the required script. If you just run conda install pandas you might end up with another error:
module 'pandas' has no attribute 'compat'.
So, try uninstalling and reinstalling pandas conda uninstall pandas, Install it again using conda install pandas this will solve the problem.
On the other hand, if you are not using Anaconda., try doing the same on Command prompt pointing to Python scripts folder pip uninstall pandas & pip install pandas.
Most of the times, this should solve the problem. Just to be cover all the possibilities, don't forget to Launch Spyder from Anaconda after installing pandas.

How can I convert a package to an exe using pyinstaller?

I've got an open source python command line program that runs on Python 2.7, Python3+, and is cross platform.
I'm trying to package it up into an executable for my windows users more easily. The source for this package is up on Github here: https://github.com/stormpath/stormpath-cli
I'm trying to package my Python program up using pyinstaller, but am having issues.
I'm running the following commands from a Windows 8 box:
$ pyinstaller --onefile setup.py
This successfully generates an EXE file for me, but when I go to run it, I get the following errors:
Traceback (most recent call last):
File "setup.py", line 4, in <module>
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
exec(bytecode, module.__dict__)
File "site-packages\setuptools\__init__.py", line 160, in <module>
File "site-packages\setuptools\monkey.py", line 93, in patch_all
File "site-packages\setuptools\monkey.py", line 145, in patch_for_msvc_specialized_compiler
File "importlib\__init__.py", line 37, in import_module
ImportError: No module named msvc
Failed to execute script setup
For testing purposes, to help narrow the issue down, I created a test.py script that contains the following code:
print('hello, world!')
And then packaged that into an exe as well:
$ pyinstaller --onefile test.py
When I run this resulting exe, everything works great! Hello world is output as expected.
I believe what's happening is that I'm not telling pyinstaller how to properly 'detect' that my project is a python package, and not a single file script.
I've read through the docs a lot, and have googled around, but haven't found a way to specify a package for pyinstaller to analyze.
What am I missing?

While I think it is a perfectly reasonable thing to do, it looks like PyInstaller simply doesn't support building an application from a package (with __main__.py).
See https://github.com/pyinstaller/pyinstaller/issues/2560.
As a workaround, you can write a small stub (outside of the package) that does the same the same thing as your __main__.py. Then point PyInstaller at that.

I think you forgot "pyinstaller -w --onefile test.py". You forgot '-w'.

Firstly install pyinstaller
pip install pyinstaller
To create exe executable folder, just run the following command:
pyinstaller exam_browser.py
If you want single exe file with a logo run this command:
pyinstaller exam_browser.py --onefile -F --icon logo.ico

How to include pytz with cx_Freeze when doing it in a virtualenv?

I had to ask this again, because a related question solves the problem when there is no virtualenv involved.
Now I have more than one virtualenv that I would like to package in executables using cx_Freeze. I run my cx_Freeze setup file normally using virtualenv python interpreter, but then when I try to execute my program, pytz doesn't find the files, probably because if you see the error below, it is searching them in the original directory of virtualenv where they were located. I checked the time zone files are located correctly into the zip file.
This is the error message I get when trying to run my bundled program:
cserpell#e431:~/src/build/exe.linux-x86_64-2.7$ ./validate
/home/cserpell/src/build/exe.linux-x86_64-2.7/library.zip/distutils/__init__.py:14: UserWarning: The virtualenv distutils package at %s appears to be in the same location as the system distutils?
Traceback (most recent call last):
File "/home/cserpell/src/virtualenvtest/local/lib/python2.7/site-packages/cx_Freeze/initscripts/Console.py", line 27, in <module>
exec(code, m.__dict__)
File "scripts/validate.py", line 9, in <module>
File "/home/cserpell/src/scripts/script.py", line 15, in <module>
from common import time_util
File "/home/cserpell/src/common/time_util.py", line 31, in <module>
SCL_TZ = pytz.timezone('America/Santiago')
File "/home/cserpell/src/virtualenvtest/local/lib/python2.7/site-packages/pytz/__init__.py", line 180, in timezone
raise UnknownTimeZoneError(zone)
pytz.exceptions.UnknownTimeZoneError: 'America/Santiago'

I found the answer myself. The only thing needed was to install setuptools in my virtualenv. For the record, it is very strange that I installed it using pip in the virtualenv, but then when I run pip list or pip freeze, it doesn't show setuptools as requirement, though it is for the cx freeze script.

Pushbullet on python throws import error due to python-magic unable to find libmagic?

I'm trying to use PushBullet.py which uses python-magic which in turn uses libmagic.
I have followed the dependencies instructions and installed Cygwin and copied the three files to Windows\system32 directory but still getting the following error:
Traceback (most recent call last):
File "C:\New Python ActiveX Scripting Engine.py", line 1, in <module>
from pushbullet import PushBullet
File "C:\Python27\lib\site-packages\pushbullet\__init__.py", line 2, in <module>
from .pushbullet import PushBullet
File "C:\Python27\lib\site-packages\pushbullet\pushbullet.py", line 4, in <module>
import magic
File "C:\Python27\lib\site-packages\magic.py", line 161, in <module>
raise ImportError('failed to find libmagic. Check your installation')
ImportError: failed to find libmagic. Check your installation
EDIT
If I put cygmagic-1.dll OR cygz.dll into C:\Python27\ and rename it to magic.dll I get the following error:
Traceback (most recent call last):
File "C:\New Python ActiveX Scripting Engine.py", line 1, in <module>
from pushbullet import PushBullet
File "C:\Python27\lib\site-packages\pushbullet\__init__.py", line 2, in <module>
from .pushbullet import PushBullet
File "C:\Python27\lib\site-packages\pushbullet\pushbullet.py", line 4, in <module>
import magic
File "C:\Python27\lib\site-packages\magic.py", line 143, in <module>
libmagic = ctypes.CDLL(dll)
File "C:\Python27\lib\ctypes\__init__.py", line 365, in __init__
self._handle = _dlopen(self._name, mode)
WindowsError: [Error 126] The specified module could not be found
If I put cygwin1.dll into C:\Python27\ and rename it to magic.dll I get the following error:
Traceback (most recent call last):
File "C:\New Python ActiveX Scripting Engine.py", line 1, in <module>
from pushbullet import PushBullet
File "C:\Python27\lib\site-packages\pushbullet\__init__.py", line 2, in <module>
from .pushbullet import PushBullet
File "C:\Python27\lib\site-packages\pushbullet\pushbullet.py", line 4, in <module>
import magic
File "C:\Python27\lib\site-packages\magic.py", line 185, in <module>
magic_open = libmagic.magic_open
File "C:\Python27\lib\ctypes\__init__.py", line 378, in __getattr__
func = self.__getitem__(name)
File "C:\Python27\lib\ctypes\__init__.py", line 383, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function 'magic_open' not found
I'm doing this on Windows 7 64bit running Python 32bit 2.7.8 (fresh install today to try and resolve this problem).
Does anyone know how to resolve the problem?
EDIT: Tried on a further 5 different windows computers and all 5 have the same problem(s).

I had the same problem with python-magic and solved it by fixing the following line in the magic.py file (C:\Python27\lib\site-packages\magic.py in my PC):
dll = ctypes.util.find_library('magic') or ctypes.util.find_library('magic1')
Since I've installed libmagic from Cygwin, the DLL was named cygmagic-1.dll. So I simply added another choice in the previous line:
dll = ctypes.util.find_library('magic') or ctypes.util.find_library('magic1') \
or ctypes.util.find_library('cygmagic-1')
Don't forget to add cygwin\bin to your PATH.
EDIT:
It seems that this issue has been addressed on the GitHub repository.
EDIT2:
These are the steps I followed to make PushBullet.py work on my machine:
Install PushBullet.py via pip (pip install PushBullet.py);
Install libmagic from Cygwin;
Add C:\cygwin\bin\ to the PATH environment variable;
Change the magic.py file as I've explained above. (In my case the error was on line 139)

From the Windows paths in your traceback listings I guess that you are trying to execute PushBullet script from Windows version of Python. Why did you install Cygwin if you are not using it? Python for Windows really won't use Cygwin's DLLs.
You have to execute PushBullet.py from Cygwin using a Python for Cygwin, not from Windows Console using a Python for Windows. That means execute for example bash (something like C:\cygwin64\bin\bash.exe --login -i) and from bash execute PushBullet script: python PushBullet.py.
It expects, that you have Python and python-magic for Cygwin (for Cygwin! not Windows version) already installed.
Copying DLLs to your system directories is nonsense, don't do it. Renaming them is uber-nonsense. How did you expect it could work? Each library is specific, you can't just rename it to change how it works internally.
I'm already using PushBullet for some time, so I was interested in this particular Python script. I'm also using the Cygwin for years. So I have installed PushBullet library:
pip install pushbullet.py
Then I have created very simple script test.py:
#!/usr/bin/python
from pushbullet import PushBullet
pb = PushBullet('my_access_token')
success, push = pb.push_note("Testing title", "Hello world!")
When I executed it using ./test.py I got an error message, that I'm missing magic library, so I installed python-magic library using Cygwin's setup utility.
I executed it again and voila - I have "Hello world!" message on my phone. Done.
Just a note again: I have executed it from the Cygwin's shell (zsh, but you'll have bash I guess), not from Windows Console. I also didn't use Python for Windows but Cygwin's version. Don't mix Windows and Cygwin executables!
So dumb-proof steps would be:
download and execute setup.exe from Cygwin.com
install python and python-magic
execute Cygwin's shell (you can execute "Cygwin Terminal" from your Start menu or just execute C:\cygwin64\bin\bash.exe --login -i for example from "Run" dialog or Windows Console)
install pip (see for example Pip install not functioning on windows 7 Cygwin install)
install PushBullet library: pip install pushbullet.py
prepare your testing script
execute it: python testing_script.py or just testing_script.py if it contains the shebang line and is executable
it should work, if not, post your errors, please
Thank you for a tip to useful library :)

You cannot mix Cygwin and MSVCRT binaries. Therefore, in order to use python-magic with the Windows Python, you must get a Windows-compiled libmagic and fix magic.py to find the libmagic DLL.
Otherwise, if you want to use Cygwin's python-magic as-is, you need to use Cygwin's python packages.