My script, built as an application, stops working when it tries to load pickle model data. The following is the problematic code:
with open('model_pickle','rb') as f:
mp = pickle.load(f)
This is the setup file I’m using:
setup(
name="Test",
version="1.0",
options={"build_exe":{"include_files":["model_pickle"]}},
executables=[target]
)
The program works as it should when executing the python script. However, after converting it to an executable via cx_freeze, the issue is with opening the pickle model data file. I have tested this with and without the data, so I’m pretty sure this is the issue.
You can import model as follows model = pickle.load(open('MODEL_PATH','rb')) .Hope this will work!
I’m 99.99% positive that you need to also add the model_pickle file to your setup(…) call as package data for this to work as an executable. Here’s a setup.py snippet showing the package data options:
from setuptools import setup, find_packages
PROJECT_NAME = 'my_project' # this should reflect your package structure
setup(
# …
packages=[package for package in find_packages() \
if package.startswith(PROJECT_NAME)],
package_dir={ 'my_project' : 'my_project' },
package_data={ '' : ['*.*'] },
include_package_data=True,
zip_safe=True,
# …
)
Also, if I were you, I’d rename the data file to something like model_pickle.pkl so that your package_data expression doesn’t have to be double-wildcarded (as above).
If that doesn’t immediately work, I would also recommend adding a MANIFEST.in file that explicitly names your binary data file.
Have you tested your program with the IDLE ? Maybe there is a pickle decoding error... If not, try using this code for creating your exe. It imports all the libraries that you have on your computer. It's longer, but easier: just one file to run.
Anyways, better user PyInstaller (also a module available on PyPI) - Related Stack Overflow answer
Related
I have a compiled a Python extension.
The resulting binary mylib.so file can be imported in the Python script and works fine.
Now I am wondering how to write the interface stub file mylib.pyi such, that pylint and the Python language server used in VS Code can use it?
For now the native library is just in the root of my scripts.
When putting the mylib.pyi next to it pylint ignores it.
I know this question is a year old but the answer might help the next person. I managed to do this by writing the following setup file:
from distutils.core import setup, Extension
extension = Extension(
name = 'dummy',
sources = ...,
...
)
setup(
name = 'dummy',
ext_modules = [extension],
packages = ['dummy'],
package_dir = {'dummy' : './stub'},
package_data = {
'dummy': ['__init__.pyi', 'py.typed']
}
)
From what I understand, the stub directory is treated as a separate package and installed alongside the extension. I've only tested it with vscode but I believe all IDEs should be able to parse it. Full code can be found here.
I'm creating a C++ extension module for Python3. Compilation of the setup.py file compiles just fine, but when I go to import my new module, I get
ImportError: libMyLib.so: cannot open shared object file: No such file or directory
this is the path to my .so:
/path/to/lib-cc7/libMyLib.so
I've tried to import my libraries in the setup.py in different ways, I have tried setting and re-setting my LD_LIBRARY_PATH variable in the terminal as well as in my .bash_profile. I have tried setting the paths in sys.path.
When I run this code before the import statement:
print(os.environ.get("LD_LIBRARY_PATH"))
print(os.environ.get("PYTHONPATH"))
I get the path to the correct library directory.
When I run strace the path to other .so's that I need show up, and I see it searching for libMyLib.so, but it just searches in what seems like all of the other directories and /path/to/lib-cc7/. In other library searches it checks /path/to/lib-cc7/.
I have sanity checked that the library is there about 5 times.
It seems like no matter what I do,
import MyModule.MySubModule as SubModule
always returns the same import error. Is there anything else that I haven't tried? Why does it seem like Python is looking in the wrong place for my .so?
EDIT 1:
This is what my setup.py (in essence) looks like:
Submodule1 = Extension('Module.Submodule1', sources=['Module/Submodule1/submod1.cpp'], language = C++, libraries=[..long list of libraries..],)
Submodule2 = Extension('Module.Submodule2', sources=['Module/Submodule2/submod2.cpp'], language = C++, libraries=[..long list of libraries..],)
setup(name = "Module", version = '1.0',
packages = ['Module', 'Module.Submodule1', 'Module.Submodule2'],
ext_modules = [Submodule1, Submodule2], )
So I'm trying to put together an auto-update functionality for my standalone OSX Python application, that's built on PyObjC. It works great simply packaging it via py2app, but I'm attemping to freeze it with Esky as part of an effort to implement the update feature.
As far as I can tell it's my setup.py formatting for Esky. I'm not sure exactly how to tell Esky to pass on the name of my .Xib file to py2app. Here's what my direct py2app setup.py looks like, successfully including the required .Xib file for the GUI:
setup.py for Py2app
from setuptools import setup
APP = ['MyApp.py']
DATA_FILES = ['MyApp.xib']
OPTIONS = {'argv_emulation': False, 'packages' : ['PIL']}
setup(
app=APP,
data_files=DATA_FILES,
options={'py2app': OPTIONS},
setup_requires=['py2app'],
iconfile="MyApp.icns"
)
Looking around at other people's posts, it looks like you can pass settings to py2app via the slightly differently structured Esky setup.py, but I can't for the life of me figure out the exact structure of the arguments to pass the .Xib file to py2app, from Esky.
setup.py for Esky
from esky import bdist_esky
from distutils.core import setup
setup(name="MyApp",
version="1.3.3",
iconfile="MyApp.icns",
data_files=['MyApp.xib'],
scripts=["MyApp.py","midheaven.py"],
options={"bdist_esky":{
"includes":["PIL"],
"excludes":['pydoc'],
"freezer_module": "py2app",
"freezer_options": {
"plist": {
'argv_emulation': False,
'packages': ['PIL'],
},
"data_files": ['MyApp.xib'],
},
},
},
)
Everything packages without an error, but of course if I try to run the Esky freeze of the app it crashes right away. I'm positive it's because it's not attaching the .Xib GUI properly. Anyone have experience with this, or ideas on how this should actually be formatted? Would absolutely love to figure this out and have it up on here for posterity.
You are correct esky does something different than what you might expect. Looking in the demo/tutorial folder is what got me on the right path.
setup(name="MyApp",
data_files=[('', ['MyApp.xib']),
('files', ['file1', 'file2']),
('img', glob(r'.\img\*.*'))
]
...
So you have a whole bunch of tuples, where the first entry is the path in your package to include the files and the second is an iterable of files to put there
You can remove the second instance of data_files that you have in the options dict.
Update
Try
from esky.bdist_esky import Executable
executables = [Executable('example_gui.py', icon='myico.ico', gui_only=True,)]
setup(
scripts = executables
...
I'm working with Python. My file GUI.py uses Yapsy for including new plugins into my software. I have finished all my work and now I want to use py2exe to generate a executable. I don't know how can I order py2exe to include Yapsy and all their modules. I have tried that and it doesn't work:
opts = {
'py2exe': { "includes" : ["yapsy.PluginManager","sip", "matplotlib.backends", "matplotlib.backends.backend_qt4agg",
"matplotlib.figure","pylab", "numpy","matplotlib.backends.backend_tkagg"]}
...
setup(
name='PImageQT',
version='1.0',
author='Jaime',
package={'images/*', 'Windows6S/*', 'modulos6S/*', 'OperacionesPlugins/*'},
scripts=['GUI.py'],
console=["GUI.py"],
options=opts,
data_files=data_files,
zipfile=None
)
I have found something in this link http://notinthestars.blogspot.com.es/2011/04/using-python-plugin-scripts-with-py2exe.html but it doesn't solve my problem.
In my Gui.py file for each plugin I add an action into a menu. This menu in the executable doesn't appear.
Does anyone know how I must do it?
Thank you very much.
Edit:
I have used breakpoints and yapsy works fine, but it doesn't found my plugins directory. The directory's name is the same.
Edit 2:
The path is ok and the directory is into the path. I don't know why it doesn't find the plugins directory after packing with py2exe. My code line for set the directory is the same like other web pages: self.manager.setPluginPlaces(["plugins"])
Edit 3:
I tried use a relative path to search the plugins and, it doesn't work.
self.manager.setPluginPlaces([os.getcwd() + os.sep + "plugins"])
Edit 4:
Today, I tried to list the directory were am I and my directory "plugins" is in. I don't understand nothing but the problem continue.
print(os.listdir('.'))
Edit 5:
This is the neverending story. I have used PYInstaller to make an executable file to check if the problem is py2exe. And it doesn't work. Now I think that the problem is Yapsy. Any ideas?
From your EDITS it seems that the problem is related to accessing some packaged plugin (and not the yapsy module per se).
For that you might want to check the following two points:
when you call self.manager.setPluginPlaces(["plugins"]) be aware that the path "plugin" may not be related to the directory you think. You should use a more specific path, using tips from the "where Am I" faq of py2exe: http://www.py2exe.org/index.cgi/WhereAmI
you might also try to package the plugins as data_files (like in the web page you've linked to), I'm not sure anymore how exactly the "package" argument (in your code sample) is handled by py2exe.
EDIT: for people that may come by later, the answer is in the comments below and basically what happened is that one of the plugin had an import error (a dependency not packaged by py2exe) and this import error could only be seen if python's standard logging is properly configured/activated with:
import logging
logging.basicConfig(level=logging.DEBUG)
you probably need to include all yapsy
opts = {
'py2exe': { "includes" : ["yapsy","sip", "matplotlib.backends", "matplotlib.backends.backend_qt4agg",
"matplotlib.figure","pylab", "numpy","matplotlib.backends.backend_tkagg"]}
...
setup(
name='PImageQT',
version='1.0',
author='Jaime',
package={'images/*', 'Windows6S/*', 'modulos6S/*', 'OperacionesPlugins/*'},
scripts=['GUI.py'],
console=["GUI.py"],
options=opts,
data_files=data_files,
zipfile=None
)
just a wild guess however
I have a problem with making exe using py2exe. In my project i'm using sqlalchemy with mssql module.
My setup.py script looks like:
from distutils.core import setup
import py2exe
setup(
windows=[{"script" : "pyrmsutil.py"}],
options={"pyrmsutil" : {
"includes": ["sqlalchemy.dialects.mssql", "sqlalchemy"],
"packages": ["sqlalchemy.databases.mssql", "sqlalchemy.cresultproxy"]
}})
But when i'm starting procedure like:
python.exe setup.py py2exe
I'm receiving build log with following errors:
The following modules appear to be missing
['_scproxy', 'pkg_resources', 'sqlalchemy.cprocessors', 'sqlalchemy.cresultproxy']
And in "dist" folder i see my pyrmsutil.exe file, but when i'm running it nothing happens. I mean that executable file starts, but do nothing and ends immediately without any pyrmsutil.exe.log. It's very strange.
Can anybody help me with this error?
I know it's no an answer per se but have you tries pyInstaller? I used to use py2exe and found it tricky to get something truly distributable. pyInstaller requires a little more setup but the docs are good and the result seems better.
For solving this issue you could try searching for the mentioned dlls and placing them in the folder with the exe, or where you build it.
Looks like py2exe can't find sqlalchemy c extensions.
Why not just include the egg in the distribution, put sqlachemy in py2exe's excludes and load the egg on start?
I use this in the start script:
import sys
import path
import pkg_resources
APP_HOME = path.path(sys.executable).parent
SUPPORT = APP_HOME / 'support'
eggs = [egg for egg in SUPPORT.files('*.egg')]
reqs, errs = pkg_resources.working_set.find_plugins(
pkg_resources.Environment(eggs)
)
map(pkg_resources.working_set.add, reqs)
sys.path.extend(SUPPORT.files('*.egg'))
i use Jason Orendorff's path module (http://pypi.python.org/pypi/path.py) but you can easily wipe it out if you want.