We Keep Coding

How to add missing .dll files while preparing .exe file with pyinstaller

I have just prepared simple script importing some module and printing something:
from clicknium import clicknium as cc
print(cc.edge.browsers)
So I have created the venv, installed clicknium (pip3 install clicknium==0.1.9).
After that I have prepared spec file:
pyi-makespec spec_file_name script_name.py
After running the command with created .spec file:
pyinstaller spec_file_name.spec
The pyinstaller is creating the .exe file.
After running the .exe I got an error:
System.IO.FileNotFoundException: Unable to find assembly 'C:\Users\user_1\AppData\Local\Temp\_MEI197042\clicknium\.lib\automation\ClickniumJavaBridge-32.dll'
Of course I understand the error but I'm not sure how to fix it.
When I has some problems with missing files I have added it by using --add-data while making the spec file. But It's not working for me with .dll files.
clicknium==0.1.9
pyinstaller==5.4.1
Update
Right now I'm using velow command to create .spec file:
pyi-makespec --onefile --add-data="C:\Users\...\project_name\venv\Lib\site-packages\clicknium\.lib\automation\*;clicknium\.lib\automation" --name app app.py
The error above is fixed but there is something new.
The code below causes the error:
clr.AddReference(Apath)
System.BadImageFormatException: Could not load file or assembly 'file:///C:\Users\user_1\AppData\Local\Temp\_MEIxxxxxx\clicknium\.lib\automation\ClickniumJavaBridge-32.dll' or one of its dependencies. The module was expected to contain an assembly manifest.
So as I understand the .dll file is still not visible there and clicknium is still looking for the dll files in Temp files.

clicknium supplied package project/folder function, can generate the exe.
you can refer to this: https://www.clicknium.com/documents/tutorial/vscode/project_management
first, in vscode, run command "Clicknium: Create Project", you can select the current folder;
then, run command "Clicknium: Package Project", it will generate the exe file

Sometimes you have to add *.dll files to your build process in the *.spec file
Look here, there is a lot of discussion about it:
Bundling data files with PyInstaller (--onefile)

If it not work for you try using other options like one file:
pyi-makespec -F script_name.py spec_file_name
also the order need to be the script first
pyi-makespec script_name.py -n spec_file_name

Creating a python package (deb/rpm) from cmake

I am trying to create a python package (deb & rpm) from cmake, ideally using cpack. I did read
https://cmake.org/cmake/help/latest/cpack_gen/rpm.html and,
https://cmake.org/cmake/help/latest/cpack_gen/deb.html
The installation works just fine (using component install) for my shared library. However I cannot make sense of the documentation to install the python binding (glue) code. Using the standard cmake install mechanism, I tried:
install(
FILES __init__.py library.py
DESTINATION ${ACME_PYTHON_PACKAGE_DIR}/project_name
COMPONENT python)
And then using brute-force approach ended-up with:
# debian based package (relative path)
set(ACME_PYTHON_PACKAGE_DIR lib/python3/dist-packages)
and
# rpm based package (full path required)
set(ACME_PYTHON_PACKAGE_DIR /var/lang/lib/python3.8/site-packages)
The above is derived from:
debian % python -c 'import site; print(site.getsitepackages())'
['/usr/local/lib/python3.9/dist-packages', '/usr/lib/python3/dist-packages', '/usr/lib/python3.9/dist-packages']
while:
rpm % python -c 'import site; print(site.getsitepackages())'
['/var/lang/lib/python3.8/site-packages']
It is pretty clear that the brute-force approach will not be portable, and is doomed to fail on the next release of python. The only possible solution that I can think of is generating a temporary setup.py python script (using setuptools), that will do the install. Typically cmake would call the following process:
% python setup.py install --root ${ACME_PYTHON_INSTALL_ROOT}
My questions are:
Did I understand the cmake/cpack documentation correctly for python package ? If so this means I need to generate an intermediate setup.py script.
I have been searching through the cmake/cpack codebase (git grep setuptools) but did not find helper functions to handle generation of setup.py and passing the result files back to cpack. Is there an existing cmake module which I could re-use ?
I did read, some alternative solution, such as:
How to build debian package with CPack to execute setup.py?
Which seems overly complex, and geared toward Debian-only based system. I need to handle RPM in my case.

As mentionned in my other solution, the ugly part is dealing with absolute path in cmake install() commands. I was able to refactor the code to avoid usage of absolute path in install(). I simply changed the installation into:
install(
# trailing slash is important:
DIRECTORY ${SETUP_OUTPUT}/
# "." syntax is a reliable mechanism, see:
# https://gitlab.kitware.com/cmake/cmake/-/issues/22616
DESTINATION "."
COMPONENT python)
And then one simply needs to:
set(CMAKE_INSTALL_PREFIX "/")
set(CPACK_PACKAGING_INSTALL_PREFIX "/")
include(CPack)
At this point all install path now need to include explicitely /usr since we've cleared the value for CMAKE_INSTALL_PREFIX.
The above has been tested for deb and rpm packages. CPACK_BINARY_TGZ does properly run with the above solution:
https://gitlab.kitware.com/cmake/cmake/-/issues/22925

I am going to post the temporary solution I am using at the moment, until someone provide something more robust.
So I eventually manage to stumble upon:
https://alioth-lists.debian.net/pipermail/libkdtree-devel/2012-October/000366.html and,
Using CMake with setup.py
Re-using the above to do an install step instead of a build step can be done as follow:
find_package(Python COMPONENTS Interpreter)
set(SETUP_PY_IN "${CMAKE_CURRENT_SOURCE_DIR}/setup.py.in")
set(SETUP_PY "${CMAKE_CURRENT_BINARY_DIR}/setup.py")
set(SETUP_DEPS "${CMAKE_CURRENT_SOURCE_DIR}/project_name/__init__.py")
set(SETUP_OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/build-python")
configure_file(${SETUP_PY_IN} ${SETUP_PY})
add_custom_command(
OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/setup_timestamp
COMMAND ${Python_EXECUTABLE} ARGS ${SETUP_PY} install --root ${SETUP_OUTPUT}
COMMAND ${CMAKE_COMMAND} -E touch ${CMAKE_CURRENT_BINARY_DIR}/setup_timestamp
DEPENDS ${SETUP_DEPS})
add_custom_target(target ALL DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/setup_timestamp)
And then the ugly part is:
install(
# trailing slash is important:
DIRECTORY ${SETUP_OUTPUT}/
DESTINATION "/" # FIXME may cause issues with other cpack generators
COMPONENT python)
Turns out that the documentation for install() is pretty clear about absolute paths:
https://cmake.org/cmake/help/latest/command/install.html#introduction
DESTINATION
[...]
As absolute paths are not supported by cpack installer generators,
it is preferable to use relative paths throughout.
For reference, here is my setup.py.in:
from setuptools import setup
if __name__ == '__main__':
setup(name='project_name_python',
version='${PROJECT_VERSION}',
package_dir={'': '${CMAKE_CURRENT_SOURCE_DIR}'},
packages=['project_name'])
You can be fancy and remove the __pycache__ folder using the -B flag:
COMMAND ${Python_EXECUTABLE} ARGS -B ${SETUP_PY} install --root ${SETUP_OUTPUT}
You can be extra fancy and add debian option such as:
if(CPACK_BINARY_DEB)
set(EXTRA_ARG "--install-layout" "deb")
endif()
use as:
COMMAND ${Python_EXECUTABLE} ARGS -B ${SETUP_PY} install --root ${SETUP_OUTPUT} ${EXTRA_ARG}

Using waf, how can I refer to a file in build directory as input to another build command?

I'm using waf as the build system for my project and I need to execute two consecutive shell commands during build process, in which output file from first command should be given as an input file at command-line to second command. According to waf book, general template for executing OS commands looks like this:
bld(rule='cp ${SRC} ${TGT}', source='input.txt', target='output.txt')
Using this template, the target directory will be automatically prepended to target file. But it's not clear how to refer to that file as an input file in later commands.
Linux OS, Python version 2.7, waf version 1.8.9
How can this be done?

Usually you just have to use the target file. Most WAF tools try to first find a file in the build directory and in the source directory. If not found it's something to build. So you can do:
rule = 'cp ${SRC} ${TGT}'
bld(rule=rule, source='input.txt', target='output.txt')
bld(rule=rule, source='output.txt', target='output2.txt')
And you will get something like:
[1/2] output.txt: input.txt -> build/output.txt
[2/2] output2.txt: build/output.txt -> build/output2.txt
WAF looks for relative paths from build and source directory.

Debianzing a Python program to get a .deb [duplicate]

This question already has answers here:
Is there a standard way to create Debian packages for distributing Python programs?
(5 answers)
Closed 5 years ago.
Aim
To create an installable .deb file (or package). Which when clicked would install the software on a Linux machine and an icon would be put on the GNOME panel. So as to launch this application from there.
What I have referred to
I referred to two debianizing guides.
Guide 1
Guide 2
The first one had a video which was impossible to understand, partly because of the accent and partly because it was hopelessly outdated.(it was uploaded in 2007)
And the second one was completely text. I got till the 4th Step, Builds the package. But when I did it I got output that did not match what was given in the guide.
What I require
I have a simple python program. It takes your age and then prints back out if the age is below, equal to, or above 18 years. There is just one file and no other dependency for this program. And I want to build this into a .deb.
Specs
-Python 2.7
-Linux Mint
Edit
I followed the exact directory structure as you instructed as you. And replaced all myscript with cowsandbulls. The build completed and I got the Debian. When I installed it and then ran the command cowsandbulls from the terminal I got the following error:
Traceback (most recent call last):
File "/usr/bin/cowsandbulls", line 9, in <module>
load_entry_point('cowsandbulls==1.0', 'gui_scripts', 'cowsandbulls')()
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 337, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2311, in load_entry_point
return ep.load()
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2017, in load
entry = __import__(self.module_name, globals(),globals(), ['__name__'])
ImportError: No module named cowsandbulls

I just tested stdeb (see https://pypi.python.org/pypi/stdeb), a Python package for turning any other Python package into a Debian package.
First I installed stdeb:
apt-get install python-stdeb
Then I made a simple script called myscript.py with the following contents:
def main():
print "Hello world, says myscript!"
# wait for input from the user
raw_input()
if __name__ == '__main__':
main()
Importantly, your directory structure should be:
somewhere/myscript/
setup.py
myscript/
__init__.py
myscript.py
In the setup.py file, you do something like:
import os
from setuptools import setup
from nvpy import nvpy
setup(
name = "myscript",
version = "1.0",
author = "Charl P. Botha",
author_email = "cpbotha#vxlabs.com",
description = "Demo of packaging a Python script as DEB",
license = "BSD",
url = "https://github.com/cpbotha/nvpy",
packages=['myscript'],
entry_points = {
'console_scripts' : ['myscript = myscript.myscript:main']
},
data_files = [
('share/applications/', ['vxlabs-myscript.desktop'])
],
classifiers=[
"License :: OSI Approved :: BSD License",
],
)
The console_scripts directive is important, and it'll create an executable script called my_script, which will be available system-wide after you install the resultant DEB. If your script uses something like tkinter or wxpython and has a graphical user interface, you should use gui_scripts instead of console_scripts.
The data_files directive will install a suitable desktop file into /usr/share/applications, so that you can also start myscript from your desktop environment. vxlabs-myscript.desktop looks like this:
[Desktop Entry]
Version=1.0
Type=Application
Name=myscript
Comment=Minimal stdeb example
# myscript should wait for user input at the end, else the terminal
# window will disappear immediately.
Exec=myscript
Icon=/usr/share/icons/gnome/48x48/apps/file-manager.png
Categories=Utility;
# desktop should run this in a terminal application
Terminal=true
StartupNotify=true
StartupWMClass=myscript
To build the DEB, you do the following in the top-level myscript:
python setup.py --command-packages=stdeb.command bdist_deb
Which will create a .deb in the deb_dist directory.
After having installed the DEB I created like this, I could run myscript from the command-line, and I could also invoke it from my desktop environment.
Here's a GitHub repository with the example code above: https://github.com/cpbotha/stdeb-minimal-example

The right way of building a deb package is using dpkg-buildpackage, but sometimes it is a little bit complicated. Instead you can use dpkg -b <folder> and it will create your Debian package.
These are the basics for creating a Debian package with dpkg -b <folder> with any binary or with any kind of script that runs automatically without needing manual compilation (Python, Bash, Pearl, and Ruby):
Create the files and folders in order to recreate the following structure:
ProgramName-Version/
ProgramName-Version/DEBIAN
ProgramName-Version/DEBIAN/control
ProgramName-Version/usr/
ProgramName-Version/usr/bin/
ProgramName-Version/usr/bin/your_script
The scripts placed at /usr/bin/ are directly called from the terminal. Note that I didn't add an extension to the script.
Also, you can notice that the structure of the deb package will be the structure of the program once it's installed. So if you follow this logic if your program has a single file, you can directly place it under ProgramName-Version/usr/bin/your_script, but if you have multiple files, you should place them under ProgramName-Version/usr/share/ProgramName/all your files and place only one file under /usr/bin/ that will call your scripts from /usr/share/ProgramName/.
Change all the folder permission to root:
chown root:root -R /path/to/ProgramName-Version
Change the script's permissions:
chmod 0755 /path/to/the/script
Finally, you can run: dpkg -b /path/to/the/ProgramName-Version and your deb package will be created! (You can also add the post/pre inst scripts and everything you want; it works like a normal Debian package.)
Here is an example of the control file. You only need to copy-paste it in to an empty file called "control" and put it in the DEBIAN folder.
Package: ProgramName
Version: VERSION
Architecture: all
Maintainer: YOUR NAME <EMAIL>
Depends: python2.7, etc , etc,
Installed-Size: in_kb
Homepage: http://foo.com
Description: Here you can put a one line description. This is the short Description.
Here you put the long description, indented by 1 space.
If you want to build using dpkg -b <folder> you can use this program that will do everything with one command. If you regularly build packages, it is a pain to do all the stuff that I mentioned!
*The guide was taken from Basics of Debian Packages.

python + distutils: add .py file to package at build/install time?

For various not-very-good-but-unfortunately-necessary reasons I'm using a setup.py file to manage some binary assets.
During py setup.py build or install I would like to create a .py file in the "normal" Python package being installed by setup.py which contains some details about these binary assets (their absolute path, version information, etc).
What's the best way to create that file?
For example, I'd like it to work something like this:
$ cd my-python-package
$ py setup.py install
...
Installing version 1.23 of my_binary_assets to /some/path...
...
$ python -c "from my_python_package import binary_asset_version_info as info; print info"
{"path": "/some/path", "version": "1.23"}
(note: I'm using the cmdclass argument to setup(…) to manage the building + installation of the binary assets… I'd just like to know how to create the binary_asset_version_info.py file used in the example)

At first sight, there is a catch-22 in your requirements: The most obvious place to create this .py file would be in the build or build_py command (to get usual distutils operations like byte-compilation), but you want that file to contain the paths to the installed assets, so you’d have to create it during the install step. I see two ways to solve that:
a) Create your info.py file during build_py, and use distutils machinery to get the installation paths of the assets files
b) Create info.py during install and call distutils.util.byte_compile to byte-compile it
I find both ideas distasteful, but well :) Now, do you know how to fill in the file (i.e. get the install paths from distutils)?