I have created an executable via pyinstaller. While running the exe found the error from pandas.
Traceback (most recent call last):
File "score_python.py", line 3, in <module>
import pandas as pd, numpy as np
File "d:\virtual\sc\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "site-packages\pandas\__init__.py", line 23, in <module>
File "d:\virtual\sc\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "site-packages\pandas\compat\__init__.py", line 32, in <module>
ImportError: No module named 'distutils'
Has anyone found the same?
This is an issue with virtualenv from version 16.4.0 onward, as indicated in the following issue on github:
https://github.com/pyinstaller/pyinstaller/issues/4064
These workarounds were suggested:
In the .spec file, at the line “hiddenimports=[]”, change to "hiddenimports=['distutils']", then run pyinstaller using the spec file.
Tried this, but it didn't work in my case, now distutils module could be found, but it threw an error while importing the module.
Downgrade virtualenv to an earlier version.
I downgraded virtualenv to version 16.1.0 and and recreated the execution bundle. The new execution file worked alright in my case.
Found the solution, it's because of the virtual environment.
The error occurred because of the creation of a new virtual environment while creating the project. I have deleted my existing virtual and created new virtual by setting up the python interpreter and opting the pre-existing interpreter option.
The IDE will create a virtual named venv and copies all the python files from Python/bin to this folder and then import modules from here, by activating the same solved my issue.
Related
I want to create a Windows executable using PyInstaller.
My project has the following dependencies:
deepspeech
PyAudio (which in turn depends on the PortAudio C library)
tkinter
I tried building the executable by running the command below:
pyinstaller entrypoint.py --name speech-to-text-gui --onedir --hidden-import numpy.random.common --hidden-import numpy.random.bounded_integers --hidden-import numpy.random.entropy
I initially tried building without any --hidden-import arguments, but then I would get errors like
ModuleNotFoundError: No module named 'numpy.random.common'
After the build, when I run the executable, it crashes with the following traceback:
Traceback (most recent call last):
File "entrypoint.py", line 1, in
from speech_to_text_gui.__main__ import main
File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
File "speech_to_text_gui\__main__.py", line 5, in
from speech_to_text_gui import speech_to_text_gui
File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
File "speech_to_text_gui\speech_to_text_gui.py", line 16, in
from deepspeech import Model
File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
File "deepspeech\__init__.py", line 23, in
File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
File "deepspeech\impl.py", line 13, in
ImportError: DLL load failed: The specified module could not be found.
[12236] Failed to execute script 'entrypoint' due to unhandled exception!
The program runs fine when I run it directly with python entrypoint.py.
I looked through the PyInstaller "If Things Go Wrong" page but nothing jumped out at me as wrong with how I am building.
I also tried building on Linux, and the executable built and ran just fine without any runtime errors on Linux.
From the traceback, my guess is loading a DLL for deepspeech failed, since that is the last call in the traceback. But take that with a grain of salt.
I am looking for any suggestions anyone might have to troubleshoot this build.
code:
import gym
env = gym.make('Breakout-v0')
I get an error:
Traceback (most recent call last):
File "C:/Users/danie/Downloads/Programming/Python/Programming/Pycharm/app.py", line 40, in
gym.make("Breakout-v0")
File "C:\Users\danie\AppData\Local\Programs\Python\Python37\lib\site-packages\gym\envs\registration.py", line 156, in make
return registry.make(id, **kwargs)
File "C:\Users\danie\AppData\Local\Programs\Python\Python37\lib\site-packages\gym\envs\registration.py", line 101, in make
env = spec.make(**kwargs)
File "C:\Users\danie\AppData\Local\Programs\Python\Python37\lib\site-packages\gym\envs\registration.py", line 72, in make
cls = load(self.entry_point)
File "C:\Users\danie\AppData\Local\Programs\Python\Python37\lib\site-packages\gym\envs\registration.py", line 17, in load
mod = importlib.import_module(mod_name)
File "C:\Users\danie\AppData\Local\Programs\Python\Python37\lib\importlib_init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 967, in find_and_load_unlocked
File "", line 677, in load_unlocked
File "", line 728, in exec_module
File "", line 219, in call_with_frames_removed
File "C:\Users\danie\AppData\Local\Programs\Python\Python37\lib\site-packages\gym\envs\atari_init.py", line 1, in
from gym.envs.atari.atari_env import AtariEnv
File "C:\Users\danie\AppData\Local\Programs\Python\Python37\lib\site-packages\gym\envs\atari\atari_env.py", line 9, in
import atari_py
File "C:\Users\danie\AppData\Local\Programs\Python\Python37\lib\site-packages\atari_py_init.py", line 1, in
from .ale_python_interface import *
File "C:\Users\danie\AppData\Local\Programs\Python\Python37\lib\site-packages\atari_py\ale_python_interface.py", line 18, in
'ale_interface/build/ale_c.dll'))
File "C:\Users\danie\AppData\Local\Programs\Python\Python37\lib\ctypes_init.py", line 434, in LoadLibrary
return self.dlltype(name)
File "C:\Users\danie\AppData\Local\Programs\Python\Python37\lib\ctypes_init.py", line 356, in init
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found
I was finally able to get around this problem with help from the following website: https://towardsdatascience.com/how-to-install-openai-gym-in-a-windows-environment-338969e24d30.
There were basically 3 things I had to do: 1) install Microsoft Visual C++ Build Tools, 2) install this specific verion of Atari https://github.com/Kojoley/atari-py.git, 3) Install Xming. Good luck to all Windows users :) This took me around 3 days to figure out.
I have tried to make it work with python 3.8 and 3.9 on Windows 10. Installing Python 3.7 and using it as the Python Interpreter on PyCharm resolved the issue. I have searched for the missing file while having 3.8 + version of Python and it did not exist at all. Atari-py is supported only for Python 3.7 (https://github.com/openai/atari-py) so having a higher version of that would not work. It is a bad idea to download files from unlicensed places so I advise you to install the supported version of Python (3.5, 3.6, and 3.7). Even though gym can work on 3.9 the atari version will not. I did not need anything else installed (although I may have Microsoft Visual C++ Build Tools, but I deff did not need Xming), just import gym 0.18.0, pyglet 1.5.0, atari -py 0.2.6
Welcome to SO. If you trace the exception trace you see that a shared object loading function is called in ctypes' init.py file aliased as dlopen. On Windows, this correponds to WINAPI's LoadLibrary method. LoadLibrary is trying to load ale_c.dll. ALE is 'arcade learning environment'.
Search your computer for ale_c.dll or libale_c.dll
If you have ale_c.dll (most likely you are on Windows), refer to this answer to see how DLL's loaded with ctypes are basically LoadLibrary calls as I mentioned. It mentions all the stuff about DLL loading order etc. but, if you need a quick way around this, copy your dll under system32 folder.
I had similar issue with Atari-Pacman. It is resolved by downloading atari_py-1.2.1-cp37-cp37m-win_amd64 which is for 64-bit Python 3.7, and installing it by using pip.
I am trying to wrap a Python script into an exe using PyInstaller (development version) for windows.
The script uses Pandas and I have been running into an error when running the exe.
Traceback (most recent call last): File "site-packages\pandas\__init__.py", line 26, in <module> File "C:\Users\Eddie\Anaconda3\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 631, in exec_module
exec(bytecode, module.__dict__) File "site-packages\pandas\_libs\__init__.py", line 4, in <module> File "C:\Users\Eddie\Anaconda3\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 714, in load_module
module = loader.load_module(fullname) File "pandas/_libs/tslib.pyx", line 1, in init pandas._libs.tslib ModuleNotFoundError: No module named 'pandas._libs.tslibs.timedeltas'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "G5k Version file Extract (with tkinter).py", line 15, in <module> File "C:\Users\Eddie\Anaconda3\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 631, in exec_module
exec(bytecode, module.__dict__) File "site-packages\pandas\__init__.py", line 35, in <module> ImportError: C extension: No module named 'pandas._libs.tslibs.timedeltas' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.
I have tried doing this for programs without pandas and everything was fine.
This is very similar to another question already solved for Python 2, but I am using Python 3 and that solution does not apply the same way due to the changed .spec file format.
Python 3.6
PyInstaller - version 3.3
Pandas - version 0.20.3
PyInstaller 3.3, Pandas 0.21.0, Python 3.6.1.
I was able to solve this thanks to not-yet published/committed fix to PyInstaller, see this and this. AND keeping the ability to pack it into one executable file.
Basically:
Locate PyInstaller folder..\hooks, e.g. C:\Program Files\Python\Lib\site-packages\PyInstaller\hooks.
Create file hook-pandas.py with contents (or anything similar based on your error):
hiddenimports = ['pandas._libs.tslibs.timedeltas']
Save it + I deleted .spec file, build and dist folders just to be sure.
Run pyinstaller -F my_app.py.
This fix should work as long as you don't upgrade or reinstall PyInstaller. So you don't need to edit .spec file.
Maybe they will include the fix sooner for us! :)
I'm not sure it may help you but following the solution on the post you mention work for me with python 3.6 pyinstaller 3.3 and pandas 0.21.0 on windows 7.
So adding this to the spec file just after analysis :
def get_pandas_path():
import pandas
pandas_path = pandas.__path__[0]
return pandas_path
dict_tree = Tree(get_pandas_path(), prefix='pandas', excludes=["*.pyc"])
a.datas += dict_tree
a.binaries = filter(lambda x: 'pandas' not in x[0], a.binaries)
Also my spec file format is the same as the one in the post you mention.
I managed to solve this problem by using the "--hidden-import" flag. Hopefully this can be helpful to someone else that comes across this thread.
pyinstaller --onefile --hidden-import pandas._libs.tslibs.timedeltas myScript.py
If you are using Anaconda, it is highly likely that when you were trying to uninstall some package it has disrupted pandas dependency and unable to get the required script. If you just run conda install pandas you might end up with another error:
module 'pandas' has no attribute 'compat'.
So, try uninstalling and reinstalling pandas conda uninstall pandas, Install it again using conda install pandas this will solve the problem.
On the other hand, if you are not using Anaconda., try doing the same on Command prompt pointing to Python scripts folder pip uninstall pandas & pip install pandas.
Most of the times, this should solve the problem. Just to be cover all the possibilities, don't forget to Launch Spyder from Anaconda after installing pandas.
I am trying to wrap a Python script into an exe using PyInstaller (development version) for windows.
The script uses Pandas and I have been running into an error when running the exe.
Traceback (most recent call last): File "site-packages\pandas\__init__.py", line 26, in <module> File "C:\Users\Eddie\Anaconda3\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 631, in exec_module
exec(bytecode, module.__dict__) File "site-packages\pandas\_libs\__init__.py", line 4, in <module> File "C:\Users\Eddie\Anaconda3\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 714, in load_module
module = loader.load_module(fullname) File "pandas/_libs/tslib.pyx", line 1, in init pandas._libs.tslib ModuleNotFoundError: No module named 'pandas._libs.tslibs.timedeltas'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "G5k Version file Extract (with tkinter).py", line 15, in <module> File "C:\Users\Eddie\Anaconda3\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 631, in exec_module
exec(bytecode, module.__dict__) File "site-packages\pandas\__init__.py", line 35, in <module> ImportError: C extension: No module named 'pandas._libs.tslibs.timedeltas' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.
I have tried doing this for programs without pandas and everything was fine.
This is very similar to another question already solved for Python 2, but I am using Python 3 and that solution does not apply the same way due to the changed .spec file format.
Python 3.6
PyInstaller - version 3.3
Pandas - version 0.20.3
PyInstaller 3.3, Pandas 0.21.0, Python 3.6.1.
I was able to solve this thanks to not-yet published/committed fix to PyInstaller, see this and this. AND keeping the ability to pack it into one executable file.
Basically:
Locate PyInstaller folder..\hooks, e.g. C:\Program Files\Python\Lib\site-packages\PyInstaller\hooks.
Create file hook-pandas.py with contents (or anything similar based on your error):
hiddenimports = ['pandas._libs.tslibs.timedeltas']
Save it + I deleted .spec file, build and dist folders just to be sure.
Run pyinstaller -F my_app.py.
This fix should work as long as you don't upgrade or reinstall PyInstaller. So you don't need to edit .spec file.
Maybe they will include the fix sooner for us! :)
I'm not sure it may help you but following the solution on the post you mention work for me with python 3.6 pyinstaller 3.3 and pandas 0.21.0 on windows 7.
So adding this to the spec file just after analysis :
def get_pandas_path():
import pandas
pandas_path = pandas.__path__[0]
return pandas_path
dict_tree = Tree(get_pandas_path(), prefix='pandas', excludes=["*.pyc"])
a.datas += dict_tree
a.binaries = filter(lambda x: 'pandas' not in x[0], a.binaries)
Also my spec file format is the same as the one in the post you mention.
I managed to solve this problem by using the "--hidden-import" flag. Hopefully this can be helpful to someone else that comes across this thread.
pyinstaller --onefile --hidden-import pandas._libs.tslibs.timedeltas myScript.py
If you are using Anaconda, it is highly likely that when you were trying to uninstall some package it has disrupted pandas dependency and unable to get the required script. If you just run conda install pandas you might end up with another error:
module 'pandas' has no attribute 'compat'.
So, try uninstalling and reinstalling pandas conda uninstall pandas, Install it again using conda install pandas this will solve the problem.
On the other hand, if you are not using Anaconda., try doing the same on Command prompt pointing to Python scripts folder pip uninstall pandas & pip install pandas.
Most of the times, this should solve the problem. Just to be cover all the possibilities, don't forget to Launch Spyder from Anaconda after installing pandas.
I am running python based software that requires a virtualenv with various additional python packages installed therein. Here are the install instructions:
Clone the repository to a shared filesysem on a cluster
>git clone https://github.com/jgurtowski/nanocorr
>cd nanocorr
Create a virtual environment to install python dependencies
>virtualenv nanocorr_ve
>source nanocorr_ve/bin/activate
install the following packages using pip:
pip install git+https://github.com/cython/cython
pip install numpy
pip install h5py
pip install git+https://github.com/jgurtowski/pbcore_python
pip install git+https://github.com/jgurtowski/pbdagcon_python
pip install git+https://github.com/jgurtowski/jbio
pip install git+https://github.com/jgurtowski/jptools
#Finally install the nanocorr package itself
> python setup.py install
I achieved this with no problems. However, While running the software, I found that it was failing with the following traceback.
(nanocorr_ve)[nanocorr_test]$ correctOxford
Traceback (most recent call last):
File "/local/workdir/malonge/nanocorr_test/nanocorr/nanocorr_ve/bin/correctOxford", line 9, in <module>
load_entry_point('jptools==0.1', 'console_scripts', 'correctOxford')()
File "/local/workdir/malonge/nanocorr_test/nanocorr/nanocorr_ve/lib/python2.7/site- packages/pkg_resources/__init__.py", line 558, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/local/workdir/malonge/nanocorr_test/nanocorr/nanocorr_ve/lib/python2.7/site- packages/pkg_resources/__init__.py", line 2682, in load_entry_point
return ep.load()
File "/local/workdir/malonge/nanocorr_test/nanocorr/nanocorr_ve/lib/python2.7/site- packages/pkg_resources/__init__.py", line 2355, in load
return self.resolve()
File "/local/workdir/malonge/nanocorr_test/nanocorr/nanocorr_ve/lib/python2.7/site- packages/pkg_resources/__init__.py", line 2361, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/local/workdir/malonge/nanocorr_test/nanocorr/nanocorr_ve/lib/python2.7/site- packages/jptools/correct.py", line 21, in <module>
from pbtools.pbdagcon.q_sense import output_dag_info
File "/local/workdir/malonge/nanocorr_test/nanocorr/nanocorr_ve/lib/python2.7/site- packages/pbtools/pbdagcon/q_sense.py", line 50, in <module>
from pbcore.io import FastaReader
File "/programs/python/lib/python2.7/site-packages/pbcore-1.0.0- py2.7.egg/pbcore/io/__init__.py", line 31, in <module>
from .BasH5IO import *
File "/programs/python/lib/python2.7/site-packages/pbcore-1.0.0- py2.7.egg/pbcore/io/BasH5IO.py", line 37, in <module>
import h5py, numpy as np, os.path as op
File "/programs/python/lib/python2.7/site-packages/h5py-2.5.0-py2.7-linux- x86_64.egg/h5py/__init__.py", line 13, in <module>
from . import _errors
ImportError: /programs/python/lib/python2.7/site-packages/h5py-2.5.0-py2.7-linux-x86_64.egg/h5py/_errors.so: undefined symbol: PyUnicodeUCS2_DecodeUTF8
I have found details as to the specific error itself, but I think the source of the problem is that the traceback clearly shows a transition from modules being imported from the /local virtualenv site-packages to another site-packages directory in /programs.
My question is the following: Given that I have the same python packages installed in two separate places, 1 in the system wide location and one in virtualenv, how do I ensure that other programs properly import all modules from within the same virtualenv sit-packages directory. I thought this would have been done automatically. I am not 100% sure that this is the sole reason for the error, but being able to fix this would help me to further debug.
It depends on how you set up your env. virtualenv has a flag option that forces it to not use the system site packages
virtualenv --no-site-packages my_env
It is the default behavior for recent versions (>=1.7).
If you've defined a PYTHONPATH environment variable, that will affect your virtualenv environment as well.