I'm trying to use pyInstaller to package a wxpython application. I looking for a variation of the "one-folder" mode whereby the dlls and pyds are not stored in the top-level directory but in a subdirectory instead (like "dlls" or "libs").
This is the spec file currently:
# -*- mode: python -*-
import os
a = Analysis\
(
["..\\job_scraper\\load_gui.py"],
pathex = ["C:\\Users\\Administrator\\Documents\\Projects\\python\\PyInstaller\\load_gui"],
hiddenimports = [],
hookspath = None,
runtime_hooks = None
)
a_binaries = []
for (name, path, data_type) in a.binaries:
(non_ext, ext) = os.path.splitext(name)
if(ext in [".pyd", ".dll"]):
a_binaries.append((os.path.join("libs", name), path, data_type))
else:
a_binaries.append((name, path, data_type))
a.binaries = TOC(a_binaries)
pyz = PYZ(a.pure)
exe = EXE\
(
pyz,
a.scripts,
exclude_binaries = True,
name = "load_gui.exe",
debug = False,
strip = None,
upx = True,
console = False
)
coll = COLLECT\
(
exe,
a.binaries,
a.zipfiles,
a.datas,
[("control.csv", "..\\job_scraper\\control.csv", "DATA")],
strip = None,
upx = True,
name = "load_gui"
)
This does to put the dlls (not the pyds) into a lib folder, however it seems to do this after linking and so the program fails to launch because it can't find the expected dlls.
The problem is that the sys.path doesn't include your subdirectories. So when the program runs, it doesn't know where to look for your .dll or .pyd files.
Of course putting the code sys.path.append("relative/path/to/your/subdirectories") on top of your main script would come to mind. But one again, this code is only executed after everything is loaded and in place.
According to this blog, the solution is using the runtime hook of pyinstaller.
Runtime hook tells the bootstrapping code to run any of your arbitrary code before your main script runs - before importing any stuff.
1. Preparation
Create a hooker.py which will add all your custom paths to sys.path. Put it somewhere and do it 1 time only.
import sys
import os
sys.path.append(os.path.join(os.path.dirname(sys.argv[0]), "lib")) # for pyd
sys.path.append(os.path.join(os.path.dirname(sys.argv[0]), "windll")) # for dll
2. Tell pyinstaller to include the hooker.py
with spec file:
a = Analysis\
(
["..\\job_scraper\\load_gui.py"],
pathex = ["C:\\Users\\Administrator\\Documents\\Projects\\python\\PyInstaller\\load_gui"],
hiddenimports = [],
hookspath = None,
runtime_hooks = "absolute/path/to/hooker.py" # <----- add it here
)
or with commandline:
pyinstaller --runtime-hook="absolute/path/to/hooker.py" the_rest_parameters
3. Run pyinstaller
As usually, it will create dist/your_main_script_name folder which contains the exe file, manifest, library.zip, and a bunch of .dll and .pyd
4. Create custom folders
Now you can create a windll folder and lib or anything that you add to sys.path at step 1. Then move all .pyd files to lib and all .dll files to windll.
Run your exe and it will crash! So move back these below files to parent folder.
pythonXX.dll , where XX is your python version
VCRUNTIME140.dll
pywintypesXX.dll, where XX is your python version and if it is included
These files are needed for the bootstrap so we cannot move them without modifying the bootstrap code.
Run the exe again and it should work normally.
Soon you will get bored of repeating all of the above over and over again.
So here is what I have been doing.
Create compiler.bat with the content similar to:
pyinstaller --runtime-hook="absolute/path/to/hooker.py" --onedir --icon path/to/icon ^
--exclude-module=UnNeeded_module_A ^
--exclude-module=UnNeeded_module_B ^
%1
#echo off
for %%F in (%1) do set pyi_output=%%~nxF
set pyi_output=%pyi_output:~0,-3%
mkdir dist\%pyi_output%\windll
mkdir dist\%pyi_output%\lib
move dist\%pyi_output%\*.dll dist\%pyi_output%\windll
move dist\%pyi_output%\*.pyd dist\%pyi_output%\lib
move dist\%pyi_output%\windll\python36.dll dist\%pyi_output%
move dist\%pyi_output%\windll\VCRUNTIME140.dll dist\%pyi_output%
if exist dist\%pyi_output%\windll\pywintypes36.dll (
move dist\%pyi_output%\windll\pywintypes36.dll dist\%pyi_output%
)
pause
To not mess up your project, create a copy of your code folder and place this compile.bat inside. Then, just drag and drop your main_script.py to compile.bat.
The pause command keeps the console windows open so that you know if the compilation is successful or not.
another way, new loadmyapp.c:
#include<stdlib.h>
main(int argc,char *argv[]) {
execv("yourapp/app.exe", argv);
}
gcc -o loadmyapp loadmyapp.c
./loadmyapp
Related
I have the following directory structure to my python project:
eplusplus/
|
|
----__main__.py
----model/
----exception/
----controller/
----view/
The directories: model, exception, controller and view each one has its
__init__.py. When I run the program at my machine I always use this following command: py -m eplusplus. But when I tried to use py2exe or pytinstaller the the points to: permission denied. For what I found, this is because its a directory I trying to compile, but when I compiled the __main__.py it compiled normally, but when I try to execute it says: Error! No eplusplus module founded!
I have no setup.py file and I don't know how they worked.
After some very intensive research and error and try I succeeded by doing this:
I added an empty __init__.py at the eplusplus folder
Out of the eplusplus folder, I had to write a compilation.py file (the file doesn't necessary must have this) to include all libraries I was using (I will post the file at the end of this answer)
Finally, at the PowerShell, all I have to type was py compilation.py py2exe
Thanks for all that tried to help me!
compilation.py file:
#To compile we need to run: python compilation.py py2exe
from distutils.core import setup
from glob import glob
import os
import py2exe
import pyDOE
VERSION=1.0
includes = [
"sip",
"PyQt5",
"PyQt5.QtCore",
"PyQt5.QtGui",
"PyQt5.QtWidgets",
"scipy.linalg.cython_blas",
"scipy.linalg.cython_lapack",
"pyDOE"
]
platforms = ["C:\\Python34\\Lib\\site-packages\\PyQt5\\plugins" +
"\\platforms\\qwindows.dll"]
dll = ["C:\\windows\\syswow64\\MSVCP100.dll",
"C:\\windows\\syswow64\\MSVCR100.dll"]
media = ["C:\\Users\\GUSTAVO\\EPlusPlus\\media\\title.png",
"C:\\Users\\GUSTAVO\\EPlusPlus\\media\\icon.png"]
documents = ["C:\\Users\\GUSTAVO\\EPlusPlus\\docs\\"+
"documentacaoEPlusPlus.pdf"]
examples = ["C:\\Users\\GUSTAVO\\EPlusPlus\\files\\"+
"\\examples\\baseline2A.idf",
"C:\\Users\\GUSTAVO\\EPlusPlus\\files\\"+
"\\examples\\vectors.csv",
"C:\\Users\\GUSTAVO\\EPlusPlus\\files\\"+
"\\examples\\BRA_SC_Florianopolis.838970_INMET.epw"]
datafiles = [("platforms", platforms),
("", dll),
("media", media),
("docs", documents),
("Examples", examples)]
imageformats = glob("C:\\Python34\\Lib\\site-packages\\PyQt5\\"+
"plugins\\imageformats\\*")
datafiles.append(("imageformats", imageformats))
setup(
name="eplusplus",
version=VERSION,
packages=["eplusplus"],
url="",
license="",
windows=[{"script": "eplusplus/__main__.py"}],
scripts=[],
data_files = datafiles,
options={
"py2exe": {
"includes": includes,
}
}
)
I am working with a scons build system for building shared libraries. Everything builds fine up to this part but now I am having some difficulties.
I cannot get scons to move the output file to a directory outside of the scons folder structure.
The only thing that I've sen on SO about it is this question here
If I try to write a python function to do it, the function runs first before the build finishes so I get some file not found errors.
How can I get scons to move a file to a directory outside of the directory where SConstruct file is defined?
import os
import shutil
Import('env')
android_files = [
'os_android.cpp',
'pic_android.cpp',
'file_access_android.cpp',
'dir_access_android.cpp',
'audio_driver_opensl.cpp',
'file_access_jandroid.cpp',
'dir_access_jandroid.cpp',
'thread_jandroid.cpp',
'audio_driver_jandroid.cpp',
'ifaddrs_android.cpp',
'android_native_app_glue.c',
'java_glue.cpp',
'cpu-features.c',
'java_class_wrapper.cpp'
]
env = env.Clone()
if env['target'] == "profile":
env.Append(CPPFLAGS=['-DPROFILER_ENABLED'])
android_objects=[]
for x in android_files:
android_objects.append( env.SharedObject( x ) )
prog = None
abspath=env.Dir(".").abspath
pp_basein = open(abspath+"/project.properties.template","rb")
pp_baseout = open(abspath+"/java/project.properties","wb")
pp_baseout.write( pp_basein.read() )
refcount=1
name="libpic"+env["SHLIBSUFFIX"]
dir="#bin/"+name
output=dir[1:]
ANDROID_HOME=os.environ.get('ANDROID_HOME')
ant_build=Dir('.').abspath+"/java/"
ANT_TARGET=ant_build+'local.properties'
ANT_SOURCES=ant_build+'build.xml'
ANDROID_HOME=os.environ.get('ANDROID_HOME')
ANT_COMMAND='ant release -Dsdk.dir='+ANDROID_HOME+' -f $SOURCE'
for x in env.android_source_modules:
pp_baseout.write("android.library.reference."+str(refcount)+"="+x+"\n")
refcount+=1
pp_baseout.close()
pp_basein = open(abspath+"/AndroidManifest.xml.template","rb")
pp_baseout = open(abspath+"/java/AndroidManifest.xml","wb")
manifest = pp_basein.read()
manifest = manifest.replace("$$ADD_APPLICATION_CHUNKS$$",env.android_manifest_chunk)
pp_baseout.write( manifest )
for x in env.android_source_files:
shutil.copy(x,abspath+"/java/src/com/android/pic")
for x in env.android_module_libraries:
shutil.copy(x,abspath+"/java/libs")
env.SharedLibrary("#bin/libpic",[android_objects],SHLIBSUFFIX=env["SHLIBSUFFIX"])
env.Command('#platform/android/java/libs/armeabi/libpic_android.so', dir, Copy('platform/android/java/libs/armeabi/libpic_android.so', output))
apk = env.Command(ANT_TARGET, source=ANT_SOURCES, action=ANT_COMMAND)
#env.Install('[install_dir]', apk)
#Cannot get this part to work, tried env.Install() but donno
#if env['target'] == 'release_debug'
#copy 'platform/android/java/bin/Pic-release-unsigned.apk' to templates as 'android_debug.apk'
#else:
#copy 'platform/android/java/bin/Pic-release-unsigned.apk' to templates as 'android_release.apk'
You should consider using the SCons Install Builder. This is the only way to install targets outside of the SCons project hierarchy.
In our build we're creating an executable file with unit tests like this:
tests = env.Program(os.path.join(env['testDir'], name + '_test'),
src + createManifest(env),
LIBS = libs,
LIBPATH = buildLibPath(env),
LINKFLAGS = env['LINKFLAGS'],
CPPPATH = cppPath)
This correctly creates an executable, which later is being run by the following builder:
action = tests[0].abspath + '&& echo %DATE% %TIME% > ${TARGET}'
runTests = env.Command(source = tests,
target = 'test_'+name+'.tmp',
action = action)
Up to this point everything works fine: the tests are being run during the build.
I've recently found Visual Leak Detector tool and wanted to include this in the build. So, I've changed the environment for the builders like this:
vldInclude = os.path.join(os.path.normpath(env['vldIncDir']), 'vld.h')
env.Append(CPPFLAGS='/FI' + vldInclude)
env.Append(LIBPATH = env['vldLibDir'])
vldLib = os.path.join(env['vldLibDir'], 'vld.lib')
libs.append(vldLib) # used in the Program call for the LIBS parameter, see above
scons: *** [build\debug\libname\test_libname.dummy] Error 309
This error message isn't very helpful. What does it mean and how to fix it?
It turns out that the magic number 309 is more googleable when written as: 0xC0000135 (no idea why C, but 135HEX == 309DEC), and it is an identifier of the STATUS_DLL_NOT_FOUND error.
So, it's not a SCons error, but Windows error, that leaks through SCons.
This means that some DLLs are missing, needed by VLD. Lurking into the VLD installation directory (usually: C:\Program Files (x86)\Visual Leak Detector) two DLL files and one manifest file can be found in the bin\Win32 subdirectory.
Not to have the build being dependent on the machine's environment, you can either add the directory to env['ENV']['PATH'] or copy the files to the directory where the tests are being run.
To do the latter:
You need another VLD configuration option, beside the library directory, namely the binaries directory. Let's call it vldBinDir. At the build's startup you can copy these files to the build directory:
def setupVld(env):
sourcePath = env['vldBinDir']
targetPath = env['testDir']
toCopy = ['dbghelp.dll',
'vld_x86.dll',
'Microsoft.DTfW.DHL.manifest']
nodes = []
for c in toCopy:
n = env.Command(os.path.join(targetPath, c),
os.path.join(sourcePath, c),
SCons.Defaults.Copy("${TARGET}", "${SOURCE}"))
nodes.append(n)
env['vldDeps'] = nodes
And then, when creating particular tests, make sure to add the dependency:
for n in env['vldDeps']:
env.Depends(tests, n)
Platform is Windows 7 64bit using python 2.7 and GTK3+ installed from http://sourceforge.net/projects/pygobjectwin32/files/?source=navbar
The exe is compiled but fails to run, due to this
The following modules appear to be missing
['gi.repository.Gdk', 'gi.repository.Gtk', 'overrides.registry']
How can i properly include these files?
imports in my .py file
from gi.repository import Gtk, Gdk
my setup file
#!/usr/bin/env python
from distutils.core import setup
import py2exe, sys
sys.path.append("C:\Python27\Lib\site-packages\gnome")
sys.path.append("C:\Python27\Lib\site-packages\repository")#tried including these extra dirs
sys.path.append("C:\Python27\Lib\site-packages\override")#tried including these extra dirs
sys.path.append("C:\Python27\Lib\site-packages\gi") #tried including these extra dirs
setup(
options = {
'py2exe': {
'bundle_files': 1,
#this does not work 'includes': ['Gtk']
}
},
console=["gui.py"],
zipfile=None
)
The executable error when ran:
ImportError: MemoryLoadLibrary failed loading gi\_gi.pyd
Thanks
You need to add "gi" to "packages".
'options': {
'py2exe': {
'packages': 'gi',
}
}
I haven't tested it on 64bit but this is the setup.py I've used to build with cx_freeze, py2exe looks like is not maintained for a long time.
from cx_Freeze import setup, Executable
import os, site, sys
## Get the site-package folder, not everybody will install
## Python into C:\PythonXX
site_dir = site.getsitepackages()[1]
include_dll_path = os.path.join(site_dir, "gtk")
## Collect the list of missing dll when cx_freeze builds the app
missing_dll = ['libgtk-3-0.dll',
'libgdk-3-0.dll',
'libatk-1.0-0.dll',
'libcairo-gobject-2.dll',
'libgdk_pixbuf-2.0-0.dll',
'libjpeg-8.dll',
'libpango-1.0-0.dll',
'libpangocairo-1.0-0.dll',
'libpangoft2-1.0-0.dll',
'libpangowin32-1.0-0.dll',
'libgnutls-26.dll',
'libgcrypt-11.dll',
'libp11-kit-0.dll'
]
## We also need to add the glade folder, cx_freeze will walk
## into it and copy all the necessary files
glade_folder = 'glade'
## We need to add all the libraries too (for themes, etc..)
gtk_libs = ['etc', 'lib', 'share']
## Create the list of includes as cx_freeze likes
include_files = []
for dll in missing_dll:
include_files.append((os.path.join(include_dll_path, dll), dll))
## Let's add glade folder and files
include_files.append((glade_folder, glade_folder))
## Let's add gtk libraries folders and files
for lib in gtk_libs:
include_files.append((os.path.join(include_dll_path, lib), lib))
base = None
## Lets not open the console while running the app
if sys.platform == "win32":
base = "Win32GUI"
executables = [
Executable("main.py",
base=base
)
]
buildOptions = dict(
compressed = False,
includes = ["gi"],
packages = ["gi"],
include_files = include_files
)
setup(
name = "test_gtk3_app",
author = "Gian Mario Tagliaretti",
version = "1.0",
description = "GTK 3 test",
options = dict(build_exe = buildOptions),
executables = executables
)
Depending on the libraries you have used you might have to add some missing dll, look at the output of cx_freeze.
I've posted the same some time ago on gnome's wiki:
https://wiki.gnome.org/Projects/PyGObject#Building_on_Win32_with_cx_freeze
My application uses QGraphicsPixmapItem, and to make it able to load jpeg files I've placed qjpeg4.dll under 'imageformats' subdirectory in the 'dist' directory.
It works, but only as long as 'bundle_files' option is set to 3.
If I set it to 1 or 2, qt4 (pyqt4) is no longer able to find needed dlls, and so QGraphicsPixmapItems is not visible.
setup.py:
from distutils.core import setup
import py2exe
setup(
options = {'py2exe': {'bundle_files': 1}},
description = "",
name = "name",
windows = ["mainwindow.py"],
zipfile=None,
)
You should be able to convince py2exe to include the dll by using:
setup(
# other options,
data_files=[('imageformats', 'qjpeg4.dll'),
#other options
)
For future reference, data_files should look like this (afaik):
data_files = [ (dir1, [file1, file2, ...]), (dir2, [file3, file4, ...]), ...]
EDIT 1: You could try using a directory structure like this (source):
yourapp.exe
[qt.conf] (optional? see lower down)
plugins/
imageformats/
qjpeg4.dll
And if that doesn't work, here suggests using a qt.conf file that looks like this:
[Paths]
Plugins = <directory containing the imageformats directory>
Which apparently should work fine so long as the core dll QtCore4.dll has been included correctly (as it needs this .dll to interpret your qt.conf file).