I am building an application which would make use of htmltidy, a library that cleans up bad HTML. There is a python binding which makes use of the soname libtidy.so. However, I intend to distribute the application using cx_Freeze. How should I do this? Also, I would like this to be a multiplatform solution, rather than just for Linux.
I tried to simply copy the soname to the created /lib folder, but that does not seem to work, most likely because of the soname search paths.
I also tried to use the bin_includes option in cx_Freeze setup.py, but that does not work either.
Script: testtidy.py
from tidylib import tidy_document
document, errors = tidy_document('''<p>fõo <img src="bar.jpg">''',
options={'numeric-entities':1})
print(document)
print(errors)
setup.py
import sys
from cx_Freeze import setup, Executable
# Dependencies are automatically detected, but it might need fine tuning.
# "packages": ["os"] is used as example only
build_exe_options = {"packages": ["os"],
"excludes": ["tkinter"],
"bin_includes": ["libtidy.so"]
}
base = None
setup(
name="testtidy",
version="0.1",
description="My application",
options={"build_exe": build_exe_options},
executables=[Executable("testtidy.py", base=base)],
)
The bin_includes option presumably does not work because the given library name is not an existing path.
In your setup.py, try to build an existing path to the tidy library:
import ctypes
from distutils import sysconfig
import os
lib_dir = sysconfig.get_config_var('LIBDIR')
libtidy_name = ctypes.util.find_library('tidy')
libtidy_path = os.path.join(lib_dir, libtidy_name)
libtidy_path should now contain the (symlink) path /usr/lib64/libtidy.so.58, use it in the bin_includes option:
build_exe_options = {...
"bin_includes": [libtidy_path]
}
After freezing, you should now see the symlink libtidy.so.58 and the the library itself libtidy.so.5.8.0 in the lib subdirectory of the build directory.
This still might not be enough to solve your problem. Looking at the source code of the python binding tidylib, starting from line 86, you will notice that it does not only rely on the soname libtidy.so, it rather relies on ctypes.util.find_library('tidy') and a pre-configured LIB_NAMES list as fallback. I'm not sure whether the modification proposed above will suffice to make ctypes.util.find_library('tidy') work in the frozen executable, if not we will still need to teach tidylib to find the library by tweaking the LIB_NAMES in the frozen executable.
EDIT:
Regarding building a multiplatform solution, see the cx_Freeze documentation:
cx_Freeze works on Windows, Mac and Linux, but on each platform it only makes an executable that runs on that platform. So if you want to freeze your programs for Windows, freeze it on Windows; if you want to run it on Macs, freeze it on a Mac.
You will thus need to make your source code run on the target platform, including htmltidy and its python binding pytidylib, and build the running code into an executable using cx_Freeze on that platform. Tweaks will probably be necessary there as well to let cx_Freeze include all dependencies of tidylib correctly, and these tweaks may look different for each platform.
Related
I have an application in which I added a module that plots data using vispy and scipy (for Delaunay).
It works fine when I run within the Python (3.4 x64 on Windows) interpreter, but not when frozen using cx_freeze. It does not give me any error message, simply it does not run (quietly).
Here my cx_freeze script:
buildOptions = dict(packages = ['osgeo._gdal', 'scipy.sparse.csgraph._validation'])
import sys
base = 'Win32GUI' if sys.platform=='win32' else None
executables = [
Executable('main.py', base=base, targetName = 'myApp.exe', icon='ico/myApp.ico')
]
setup(name='MyApp',
version = '0.0.1',
description = 'My fancy app',
author = 'xxxx#xxxx.xxx',
options = dict(build_exe = buildOptions),
executables = executables)
I have to add 'scipy.sparse.csgraph._validation' to fix a previous missing inclusion as suggested here: scipy with py2exe and here
Looking for DLL issues, I have already attempted with Dependency Walker but without luck.
If I comment out the module with the vispy plot, everything works fine. Any hint?
I have attempted to freeze just the module with the problem by adding a main with a QApplication that display the QWigdet with the vispy.app.canvas. This helped because I got a very useful backtrace error related to vispy.app.backends._pyside.
After explicitly adding this to my posted cx_freeze script, the frozen application works:
packages = ['osgeo._gdal', 'vispy.app.backends._pyside', 'scipy.sparse.csgraph._validation']
The difference that I found in the build directory is the presence of QtOpenGL4.dll and PySide.QtOpenGL.pyd. They were not there without the explicit package inclusion (my application was already using PySide).
We've not thought about freezing apps with Vispy much yet. The pitfall that I'd expect matches with gmas80's answer; Vispy can use multiple backends, which means that these are dynamically loaded and cx_Freeze is unable to select the backend modules as a dependency. Depending on the backend that you need, you need to add some modules in vispy.backends to the list of includes.
I'm a python programmer and I'm trying to build an executable binary to distribute my software to my clients, even if it's not fully executable I want to be able to distribute my software in a way so that it is convenient for the end user.
I have already tried PyInstaller as well as Py2Exe and I'm facing the same problem with a particular software.
I used the splinter module for my program (which of course is a new high level framework to interact with other frameworks like Selenium) and every time I try to compile it there seems to be a file called "webdriver.xpi" that is always left out from the final package and therefore when the program attempts to execute the web-driver it fails with an IO Error saying that the file "webdriver.xpi" was not found....but other than that the GUI and everything works perfectly fine.
So is there a way to include it even manually? I tried including it manually by browsing to the specific folder # library.zip file but it didn't work.
I'm not really expert in this matter and I rely on GUI2Exe for building everything...and I would really appreciate some advice if possible on how to fix this.
Thanks.
I was at this all day and found a workaround, it's sneaky but it works. In the error message I was receiving I noticed that there was a space between in library .zip. I could not trace it down in the source code for py2exe or selenium. I too had tried putting the xpi file in the library zip and it did not work. The workaround is:
In your setup file use these options:
setup(
console=['yourFile.py'],
options={
"py2exe":{
"skip_archive": True,
"unbuffered": True,
"optimize": 2
}
}
)
Run the py2exe install
Copy the xpi file into the dist directory
That should do it.
You need an instruction in your setup.py to include any resource files in your distribution. There is a couple of ways of doing this (see distutils, setuptools, distribute - depending on what you are using to build your distribution), but the py2exe wiki has an example.
You may need to use this py2exe tip to find your resources if you're installing them into the same directory as your exe.
See this answer for some additional info on including resource files in your distribution.
Here is a solution of your question:
I have modify a code little and it should be work since I had a same issue and I solved it:
from distutils.core import setup
import py2exe
wd_base = 'C:\\Python27\\Lib\site-packages\\selenium-2.44.0-py2.7.egg \\selenium\\webdriver'
RequiredDataFailes = [
('selenium/webdriver/firefox', ['%s\\firefox\\webdriver.xpi'%(wd_base), '%s\\firefox\\webdriver_prefs.json'%(wd_base)])
]
setup(
windows=[{"script":"gui_final.py"}],options={"py2exe":{"skip_archive": True,"includes":["sip"]}},
data_files=RequiredDataFailes,
)
I know this is old, but I wanted to give an updated answer to avoid suggesting that programmers do something manually.
There is a py2exe option to specify a list of data files as tuples. (pathtocopyto, [list of files and where to get them])
Example:
from disutils.core import setup
import py2exe
wd_base = 'C:\\Python27\\Lib\\site-packages\\selenium\\webdriver'
RequiredDataFailes = [
('selenium/webdriver/firefox', ['%s\\firefox\\webdriver.xpi'%(wd_base), '%s\\firefox\\webdriver_prefs.json'%(wd_base)])
]
setup(
console=['MyScript.py'],
data_files=RequiredDataFiles,
options={
**mypy2exeopts
}
)
The only downside I am aware of currently is that you still need skip_archive = True. There are workarounds to get the data files in the library.zip, but I haven't had much luck with the webdriver's info.
For some reason on my system (Windows XP 32-bit, Python 2.6) PyQt is able to display gifs perfectly when run in the python interpreter, but when I run it through py2exe, they no longer display.
I've tried everything I've googled: copying the gif DLLs from PyQt into an imageformats/ folder, setting up a qt.conf (as another stackoverflow thread suggested), done a setLibraryPaths to where the imageformat DLLs were, copied the setup file from http://wiki.wxpython.org/py2exe-python26 .
Nothing seems to work -- what on earth could I be doing wrong?
I'm not sure how you even managed to compile PyQt with py2exe; I had no success, and switched to pyinstaller.
Py2exe wasn't cooperating well PyQt and refused to compile depending on what widgets were featured.
I'd recommend switching to pyinstaller for compiling PyQt; see if it allows what you want in this case
from distutils.core import setup
import py2exe
DATA=[('imageformats',['C:\\Python26/Lib/site-packages/PyQt4/plugins/imageformats/qjpeg4.dll',
'C:\\Python26/Lib/site-packages/PyQt4/plugins/imageformats/qgif4.dll',
'C:\\Python26/Lib/site-packages/PyQt4/plugins/imageformats/qico4.dll',
'C:\\Python26/Lib/site-packages/PyQt4/plugins/imageformats/qmng4.dll',
'C:\\Python26/Lib/site-packages/PyQt4/plugins/imageformats/qsvg4.dll',
'C:\\Python26/Lib/site-packages/PyQt4/plugins/imageformats/qtiff4.dll'
])]
setup(windows=[{"script":"your_python_script.py"}],
data_files = DATA,
options={"py2exe":{
"includes":["sip", "PyQt4.QtNetwork", "PyQt4.QtWebKit", "PyQt4.QtSvg" ],
"bundle_files":3,
"compressed":True,
"xref":True}},
zipfile=None)
Just in case anyone is in this situation, I found the solution. These is what you need to do when compiling with py2exe so your image files are shown:
All the image files (gif, png, jpg) need to be copied to the dist folder
Qt dll files need to be copied from Qt installation folder to
dist\imageformats
For the dll files you need to set this on your setup.py file:
windows = [{
"script":"yourPythonScript.py",
"icon_resources": [(1, "nameOfIcoFile.ico")],
"dest_base":"nameOfExeFile"
}],
data_files = [
('imageformats',
[r'C:\Python27\Lib\site-packages\PyQt4\plugins\imageformats\qico4.dll',
r'C:\Python27\Lib\site-packages\PyQt4\plugins\imageformats\qgif4.dll'
])],
)
With my Java projects at present, I have full version control by declaring it as a Maven project. However I now have a Python project that I'm about to tag 0.2.0 which has no version control. Therefore should I come accross this code at a later date, I won't no what version it is.
How do I add version control to a Python project, in the same way Maven does it for Java?
First, maven is a build tool and has nothing to do with version control. You don't need a build tool with Python -- there's nothing to "build".
Some folks like to create .egg files for distribution. It's as close to a "build" as you get with Python. This is a simple setup.py file.
You can use SVN keyword replacement in your source like this. Remember to enable keyword replacement for the modules that will have this.
__version__ = "$Revision$"
That will assure that the version or revision strings are forced into your source by SVN.
You should also include version keywords in your setup.py file.
Create a distutils setup.py file. This is the Python equivalent to maven pom.xml, it looks something like this:
from distutils.core import setup
setup(name='foo',
version='1.0',
py_modules=['foo'],
)
If you want dependency management like maven, take a look at setuptools.
Ants's answer is correct, but I would like to add that your modules can define a __version__ variable, according to PEP 8, which can be populated manually or via Subversion or CVS, e.g. if you have a module thingy, with a file thingy/__init__.py:
___version___ = '0.2.0'
You can then import this version in setup.py:
from distutils.core import setup
import thingy
setup(name='thingy',
version=thingy.__version__,
py_modules=['thingy'],
)
how to create a good plugin engine for standalone executables created with pyInstaller, py2exe or similar tools?
I do not have experience with py2exe, but pyInstaller uses an import hook to import packages from it's compressed repository. Of course I am able to import dynamically another compressed repository created with pyInstaller and execute the code - this may be a simple plugin engine.
Problems appears when the plugin (this what is imported dynamically) uses a library that is not present in original repository (never imported). This is because import hook is for the original application and searches for packages in original repository - not the one imported later (plugin package repository).
Is there an easy way to solve this problem? Maybe there exist such engine?
When compiling to exe, your going to have this issue.
The only option I can think of to allow users access with thier plugins to use any python library is to include all libraries in the exe package.
It's probably a good idea to limit supported libraries to a subset, and list it in your documentation. Up to you.
I've only used py2exe.
In py2exe you can specify libraries that were not found in the search in the setup.py file.
Here's a sample:
from distutils.core import setup
import py2exe
setup (name = "script2compile",
console=['script2compile.pyw'],
version = "1.4",
author = "me",
author_email="somemail#me.com",
url="myurl.com",
windows = [{
"script":"script2compile.pyw",
"icon_resources":[(1,"./ICONS/app.ico")] # Icon file to use for display
}],
# put packages/libraries to include in the "packages" list
options = {"py2exe":{"packages": [ "pickle",
"csv",
"Tkconstants",
"Tkinter",
"tkFileDialog",
"pyexpat",
"xml.dom.minidom",
"win32pdh",
"win32pdhutil",
"win32api",
"win32con",
"subprocess",
]}}
)
import win32pdh
import win32pdhutil
import win32api
PyInstaller does have a plugin system for handling hidden imports, and ships with several of those already in. See the webpage (http://www.pyinstaller.org) which says:
The main goal of PyInstaller is to be compatible with 3rd-party packages out-of-the-box. This means that, with PyInstaller, all the required tricks to make external packages work are already integrated within PyInstaller itself so that there is no user intervention required. You'll never be required to look for tricks in wikis and apply custom modification to your files or your setup scripts. Check our compatibility list of SupportedPackages.