I am trying to package my program using pyinstaller. The code runs fine on windows, and uses SqlAlchemy, OpenCV and pyodbc packages.
I ran pyinstaller to create the executable and tried to run it. I'm getting an error:
ImportError: No module named ConfigParser
now, I reran the same thing and looked at logs from pyinstaller and got a warning:
WARNING: Hidden import "sqlalchemy.sql.functions.func" not found!
along with a few others.
then there was a warning about trying to import ConfigParser in lower and uppercase.
Attempted to add Python module twice with different upper/lowercases: ConfigParser
What might be the issue here?
So, I figured it out. To an extent.
Seems like pyInstaller doesn't deal with SWIG files that well.
In sqlalchemy.utils there's a file called compat.py. It is there to make the module compatible with all versions of python.
for example, in python2.x, there's ConfigParser whereas in py3, it is named configparser
So there is a part in compat.py to deal with it:
if py3:
import configparser
# Some other such import statements
elif py2:
import ConfigParser as configparser
Now, pyinstaller gets stumped here as it just focuses on the import, and hence it tries to import both and fails miserably.
My crude workaround to this involved modifying the compat.py file and retaining only the parts relevant to the python version I have (2.x).
Running pyinstaller again proved to be a success! :)
Although this is all very crude and there's probably something better out there, but I couldn't find anything, so I'm sharing what worked for me.
Related
I import vtt_to_srt converter.I read the instructions.I make the installation.From beginning to this "no problem" but when I import it as manuals describe python can't find the module there.
Installation
pip install vtt_to_srt3
Usage
from vtt_to_srt import vtt_to_srt
path = '/path/to/file.vtt'
vtt_to_srt(path)
Error
ImportError: cannot import name 'vtt_to_srt' from 'vtt_to_srt'
I am a rookie.Sorry for asking this question.
#SakuraFreak's answer was my immediate thought, too, but I ended up investigating this a bit further. It seems like vtt_to_srt stores all its API in the __main__.py file of the module. This file should normally be used when you want to a module using python -m.
This actually makes the module impossible to use the way that the documentation specifies. What I tried, then was:
from vtt_to_srt.__main__ import vtt_to_srt
print(vtt_to_srt)
This results in:
<function vtt_to_srt at 0x000002A0948216A8>
So it seems like this workaround is OK.
I do not know if storing all the module's code in __main__.py is some convention not supported by my version of Python (CPython 3.7.3 on Windows), or if it simply an error. Maybe you should approach the module's owners with this.
Because you're trying to import a function from the module which doesn't exist.
the correct way for this module is:
import vtt_to_srt
I'm self-taught in the Python world, so some of the structural conventions are still a little hazy to me. However, I've been getting very close to what I want to accomplish, but just ran into a larger problem.
Basically, I have a directory structure like this, which will sit outside of the normal python installation (this is to be distributed to people who should not have to know what a python installation is, but will have the one that comes standard with ArcGIS):
top_directory/
ArcToolbox.tbx
scripts/
ArcGIStool.py (script for the tool in the .tbx)
pythonmod/
__init__.py
general.py
xlrd/ (copied from my own python installation)
xlwt/ (copied from my own python installation)
xlutils/ (copied from my own python installation)
So, I like this directory structure, because all of the ArcGIStool.py scripts call functions within the pythonmod package (like those within general.py), and all of the general.py functions can call xlrd and xlwt functions with simple "import xlrd" statements. This means that if the user desired, he/she could just move the pythonmod folder to the python site-packages folder, and everything would run fine, even if xlrd/xlwt/xlutils are already installed.
THE PROBLEM:
Everything is great, until I try to use xlutils in general.py. Specifically, I need to "from xlutils.copy import copy". However, this sets off a cascade of import errors. One is that xlutils/copy.py uses "from xlutils.filter import process,XLRDReader,XLWTWriter". I solved this by modifying xlutils/copy.py like this:
try:
from xlutils.filter import process,XLRDReader,XLWTWriter
except ImportError:
from filter import process,XLRDReader,XLWTWriter
I thought this would work fine for other situations, but there are modules in the xlutils package that need to import xlrd. I tried following this advice, but when I use
try:
import xlrd
except ImportError:
import os, sys, imp
path = os.path.dirname(os.path.dirname(sys.argv[0]))
xlrd = imp.load_source("pythonmod.xlrd",os.path.join(path,"xlrd","__init__.py"))
I get a new import error: In xlrd/init.py, the info module is called (from xlrd/info.py), BUT when I use the above code, I get an error saying that the name "info" is not defined.
This leads me to believe that I don't really know what is going on, because I thought that when the init.py file was imported it would run just like normal and look within its containing folder for info.py. This does not seem to be the case, unfortunately.
Thanks for your interest, and any help would be greatly appreciated.
p.s. I don't want to have to modify the path variables, as I have no idea who will be using this toolset, and permissions are likely to be an issue, etc.
I realized I was using imp.load_source incorrectly. The correct syntax for what I wanted to do should have been:
imp.load_source("xlrd",os.path.join(path,"xlrd","__init__.py"))
In the end though, I ended up rewriting my code to not need xlutils at all, because I continued to have import errors that were causing many more problems than were worth dealing with.
In my IronPython script, I'm using standard libary modules like ConfigParser, logging and JSON.
Then I use pyc.py to create an executable. At first I ran into problems, namely '...ImportException: no module named ...'
since they weren't being included in the exe and accompanying dlls.
So I ran a solution from here: IronPython: EXE compiled using pyc.py cannot import module "os" and it mostly worked.
For example, importing 'ConfigParser' would work since in the IronPython 'Lib' folder as a module, it's there as 'ConfigParser.py'. However I'm still having trouble using JSON and logging since they're inside of folders with their name (packages?).
I'm feeling that I'm just missing something simple, and probably need to read up more on python modules and how they really work, but I'm not sure what I should be looking for.
Any help would be greatly appreciated.
Thanks!
EDIT:
I can't answer my own question yet, so I'll leave this here.
Somehow got it to work in a really 'hacky' way. There must be another much cleaner solution to this that I'm missing (some option in pyc.py?)
Here's what I did:
1) Made the StdLib.dll file generated from the link above (IronPython: EXE compiled using pyc.py cannot import module "os"). This would be missing the std lib packages.
2) Used SharpDevelop to compile the standard lib packages that weren't included in the above dll following the method here: http://community.sharpdevelop.net/blogs/mattward/archive/2010/03/16/CompilingPythonPackagesWithIronPython.aspx
3) Used SharpDevelop to build my program and tie together all the references.
- Reference to the dlls made in step 2
- Reference to the StdLib.dll made in step 1
Again, there must be a better solution to this.
I've found two ways to compile standard library python packages:
1st way: Individually compile each package into a dll
Using pyc.py, run something like (this example compiles logging package):
ipy pyc.py ".\Lib\logging\__init__.py" ".\Lib\logging\config.py" ".\Lib\logging\handlers.py" /target:dll /out:logging
This creates a logging.dll file, which you can then use like this:
import clr
clr.AddReference('StdLib') #from the compilation of non-package std libraries
clr.AddReference('logging')
import logging
**Note: This is assuming you've run the solution from IronPython: EXE compiled using pyc.py cannot import module "os" to create StdLib.dll
2nd way: Modify the compilation script that generated StdLib.dll
I changed this line:
#Build StdLib.DLL
gb = glob.glob(r".\Lib\*.py")
gb.append("/out:StdLib")
To this:
#Build StdLib.DLL
gb1 = glob.glob(r".\Lib\*.py")
gb2 = glob.glob(r".\Lib\*\*.py")
gb3 = glob.glob(r".\Lib\*\*\*.py")
gb = list(set(gb1 + gb2 + gb3))
gb.append("/out:StdLib")
This includes the subfolders in the Lib directory which get missed in the original regex (only modules get included). Now, packages like xml, json, logging, etc. get included into StdLib.dll
Okay, so in the past, I've made my own Python packages with Python 2.x (most recently, 2.7.5). It has worked fine. Let me explain how I did that, for reference:
Make a directory within the working directory. We'll call it myPackage.
Make a file called __init__.py in the directory myPackage.
Make sure all the modules that you want to be part of the package are imported within __init__.py. These modules are typically in the myPackage folder.
From a Python program in the working directory, type import myPackage (and it imports fine, and is usable).
However, in Python 3, I get errors with that. (ImportError: No module named 'Whatever the first imported module is')
I researched the problem and found the following:
Starred imports don't work in Python 3.3.
The __init__.py file is not required in Python 3.3.
So, I removed the stars from my imports, and leaving the __init__.py file in, I still got errors (ImportError: No module named 'Whatever the first imported module is'). So, I removed the __init__.py file, and I don't get any errors, but my package doesn't include any of my modules.
Okay, so I discovered by doing a web search for python3 __init__.py or some such that I can do the following, although I don't have any clue if this is the standard way of doing things:
In the modules in the package, make sure there are no plain imports (not just no starred ones). Only do from myModule import stuff. However, you need to put a . in front of myModule: e.g. from .myModule import stuff. Then, I can import myPackage.oneOfMyModules
I found that by following this rule in the __init__.py file, it also works.
Once again, I don't know if this is how it's supposed to work, but it seems to work.
I found this page that is supposed to have something to do with the changes in packages in Python 3.something, but I'm not sure how it relates to what I'm doing:
http://legacy.python.org/dev/peps/pep-0420/
So, what is the standard way to do this? Where is it documented (actually saying the syntax)? Is the way I'm doing it right? Can I do regular imports instead of from package import module?
After analyzing some Python 3 packages installed on my system (I should have tried that to start with!) I discovered that they often seem to do things a little differently. Instead of just doing from .myModule import stuff they would do from myPackage.myModule import stuff (inside the modules in the package). So, that works, too, I suppose, and seems to be more frequently used.
I recently wrote a Python 2.7 script (using PyDev on Eclipse) that took advantage of the built-in ConfigParser module, and the script works perfectly. But when I exported it and sent it to a colleague, he could not get it to work. He keeps getting an "unresolved import: ConfigParser" error even though we are using the exact same settings. This isn't supposed to happen as ConfigParser is built-in.
I've Googled everywhere but could not seem to find any working solution. Any help would be appreciated.
ConfigParser was renamed to configparser in python 3. Chances are he's using 3 and cannot find the old py2 name.
You can use:
try:
import configparser as ConfigParser
except ImportError:
import ConfigParser
To see what's happening it may be nice comparing on both computers which sys.path is being used (i.e.: put at the start of the module being run the code below and compare the output in each case):
import sys
print '\n'.join(sorted(sys.path))
Now, if the error is not when running the code (i.e.: it runs fine and you get no exceptions), and he gets the error only in PyDev, probably the interpreter configuration in his side is not correct and one of the paths printed through the command above is not being added to the PYTHONPATH (it could be that he's on a virtual env and didn't add the paths to the original /Lib or has added some path that shouldn't be there -- or even has some ConfigParser module somewhere else which is conflicting with the one from the Python standard library).