We Keep Coding

Nuitka doesn't get along with PySimpleGUI: `CALL_FUNCTION_WITH_ARGS14' error

I try to make an EXE on windows with Nuitka but always get error like this: CALL_FUNCTION_WITH_ARGS14
I googled 'CALL_FUNCTION_WITH_ARGS14' but nothing shows up. Does anyone have similar issue?
I am using Python 3.9 and 3.10 and this is my command
nuitka --standalone --onefile --mingw64 --show-memory --show-progress --follow-imports --output-dir=out Demo_Hello_World.py
The Demo_Hello_World.py is from PySimpleGUI/DemoPrograms/Demo_Hello_World.py
ValueTraceLoopComplete 38798 30843 7955
ValueTraceLoopIncomplete 16811 14635 2176
ValueTraceMerge 470835 399699 71136
ValueTraceUninitialized 82073 66112 15961
ValueTraceUnknown 238500 209701 28799
Nuitka:INFO: Running data composer tool for optimal constant value handling.
Nuitka:INFO: Running C compilation via Scons.
Nuitka-Scons:INFO: Backend C compiler: gcc (gcc).
Nuitka-Scons:INFO: Backend linking program with 14 modules (no progress information available).
module.PySimpleGUI.PySimpleGUI.c: In function 'impl_PySimpleGUI$PySimpleGUI$$$function__578__github_issue_post_make_markdown':
module.PySimpleGUI.PySimpleGUI.c:549680:35: warning: implicit declaration of function 'CALL_FUNCTION_WITH_ARGS14'; did you mean 'CALL_FUNCTION_WITH_ARGS10'? [-Wimplicit-function-declaration]
549680 | tmp_assign_source_2 = CALL_FUNCTION_WITH_ARGS14(unicode_builtin_format, args);
| ^~~~~~~~~~~~~~~~~~~~~~~~~
| CALL_FUNCTION_WITH_ARGS10
module.PySimpleGUI.PySimpleGUI.c:549680:33: warning: assignment to 'PyObject *' {aka 'struct _object *'} from 'int' makes pointer from integer without a cast [-Wint-conversion]
549680 | tmp_assign_source_2 = CALL_FUNCTION_WITH_ARGS14(unicode_builtin_format, args);
| ^
Nuitka-Scons:INFO: Running 'C:\\Users\\nytrg\\AppData\\Local\\Nuitka\\Nuitka\\gcc\\x86_64\\11.3.0-14.0.3-10.0.0-msvcrt-r3\\mingw64\\bin\\gcc.exe -o "F:\\PySimpleGUI\\DemoPrograms\\out\\Demo_Hello_World.dist\\Demo_Hello_World.exe" -fuse-linker-plugin -flto=6 -fpartial-inlining -freorder-functions -Wl,--exclude-all-symbols -Wl,--out-implib,.\\import.lib -municode -O3 -s -Wl,--enable-auto-import -Wl,--stack,9863168 -static-libgcc #".\\#link_input.txt" -LC:\\Users\\nytrg\\AppData\\Local\\Programs\\Python\\Python39\\libs -lm -lpython39' took 164.76 seconds
c:/users/nytrg/appdata/local/nuitka/nuitka/gcc/x86_64/11.3.0-14.0.3-10.0.0-msvcrt-r3/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\nytrg\AppData\Local\Temp\cccIcn6f.ltrans80.ltrans.o:<artificial>:(.text+0x32b8): undefined reference to `CALL_FUNCTION_WITH_ARGS14'
collect2.exe: error: ld returned 1 exit status
scons: *** [F:\PySimpleGUI\DemoPrograms\out\Demo_Hello_World.dist\Demo_Hello_World.exe] Error 1

I had the same problem and the only thing that helped me was compiling in a clean Python venv, the issue may be due to some package conflict but requires further investigation.
So to sum that up:
Create a clean venv
Activate the venv using the appropriate script (Refer to the table below)
Install just the necessary prerequisites through pip (including Nuitka of course)
Try compiling with nuitka again

My application, WSA Sideloader is a PySimpleGUI program compiled using Nuitka. This is the command I used:
nuitka --standalone sideloader.py --enable-plugin=tk-inter --windows-disable-console
Of course the name of your main Python file should be used after --standalone.
Note that if it requires other files to function you will need to add them to the dist folder once it's compiled.

numpy.f2py - extension doesn't exist when source distribution is installed with pip

I'm trying to wrap my head around f2py because my organization has a lot of legacy fortran code that I would like to incorporate into some newer python-based tools I'm writing. Ideally, I would package these tools either in source packages or wheels to make it easier to distribute to the rest of the organization.
I've written a small test package based on some other examples I've seen that just sums an array of floats. The package contents are included below. If I build a source distribution tarball using py setup.py sdist, everything looks like it works. It even looks like pip successfully installs it. However, if I open a python shell and try to import the newly installed module, I get an error on the from fastadd import fadd line in the initialization script saying
AttributeError: module 'fastadd' has no attribute 'fastadd'
So it seems like it didn't actually successfully build the f2py module. Doing some troubleshooting, if I open a powershell window in the package folder and just run
py -m numpy.f2py -c fadd.pyf fadd.f90
and then open a python shell in the same folder and try to import fastadd, I get an error, ImportError: DLL load failed: The specified module could not be found. (This is after I installed the Visual Studio build tools, a fix suggested on several threads). Following the advice on this thread, changing the command to
py -m numpy.f2py -c --fcompiler=gnu95 --compiler=mingw32 fadd.pyf fadd.f90
will build a module file that I can successfully import and use. Okay, great.
However, when I change config.add_extension in the setup file to include the keyword argument f2py_options=["--fcompiler=gnu95","--compiler=mingw32"] and try to build a package distribution file with setup.py sdist command and then install using py -m pip install fastadd-1.0a1.tar.gz, I get yet a different error that says
ERROR: No .egg-info directory found in C:\Users\username\AppData\Local\Temp\pip-pip-egg-info-c7406k03
And now I'm completely flummoxed. Other configurations of the f2py_options either result in setup.py throwing an error or fail to create the extension altogether, similar to above. Using a simple string for the options gives an error, so apparently f2py_options does in fact expect a list input. I can't seem to find any good documentation on whether I'm using f2py_options correctly, and I have no idea why just adding that option would cause pip to not know where its info directory is. That makes no sense to me. I'd really appreciate some help on this one.
I'm running Python 3.7.0 32-bit, numpy 1.20.1, and pip 21.0.1 on a Windows 10 machine.
--EDIT--
Looking in the installation directory of the test module, I found a new wrinkle to this problem: the installation directory does not actually include any files listed in MANIFEST, not even the __init__.py file. If I copy __init__.py into the directory, trying to import the module gives the same ImportError: DLL load failed error I've been getting.
Also, inspecting the output of py -m pip install, it looks like numpy.distutils doesn't recognize --fcompiler or --compiler as valid options and just ignores them, even though numpy.f2py does recognize them.
--END EDIT--
PACKAGE CONTENTS:
+-fastadd
---__init__.py
---fadd.f90
---fadd.pyf
-MANIFEST.in
-README
-setup.py
fadd.f90 has the following contents:
subroutine fadd(vals,n,mysum)
integer, intent(in) :: n
real*8, intent(out):: mysum
real*8, dimension(n), intent(in) :: vals
mysum = sum(vals)
end subroutine fadd
fadd.pyf has the following contents:
python module fastadd ! in
interface ! in :fastadd
subroutine fadd(vals,n,mysum) ! in :fastadd:fadd.f90
real*8 dimension(n),intent(in) :: vals
integer, optional,intent(in),check(len(vals)>=n),depend(vals) :: n=len(vals)
real*8 intent(out) :: mysum
end subroutine fadd
end interface
end python module fastadd
__init__.py:
"""This is the documentation!"""
from .fastadd import fadd
MANIFEST.in:
include README
recursive-include fastadd *.f90
recursive-include fastadd *.pyf
recursive-include fastadd *.py
and, finally, setup.py:
def configuration(pth=None):
from numpy.distutils.misc_util import Configuration
config = Configuration(
'fastadd',
top_path=pth,
version='1.0a1',
author='John Doe',
author_email='john.doe#fake-org.biz',
url='fake-org.biz/fastadd',
description="Testing f2py build process. Sums an arbitrary-length list of numbers.")
config.add_extension(
'fastadd',
sources=['fastadd\\fadd.pyf','fastadd\\fadd.f90']
)
return config
if __name__ == '__main__':
from numpy.distutils.core import setup
setup(**configuration('fastadd').todict())
If it helps at all, the final MANIFEST file looks like this after the setup script is run:
# file GENERATED by distutils, do NOT edit
README
setup.py
C:\Users\username\Documents\Development\python_modules\fastadd\fastadd\fadd.f90
C:\Users\username\Documents\Development\python_modules\fastadd\fastadd\fadd.pyf
fastadd\__init__.py
fastadd\fadd.f90
fastadd\fadd.pyf

Installing tfprof command line

I want to profile my tensorflow application using tfprof. I have a running tensorflow 1.3 installation where the the tfprof command line tool is missing. I also tried the provided pip packages locally, but there I also can't find tfprof.
Is there a way to compile and link the tfprof command line tool agains my running tensorflow application?
I already git-cloned the tensorflow repository and tried to build it with bazel 0.5.2
$ bazel build --config opt tensorflow/core/profiler/...
WARNING: Output base '/home/USERNAME/.cache/bazel/_bazel_USERNAME/e5cce820cc082410b4fcc604db349066' is on NFS. This may lead to surprising failures and undetermined behavior.
WARNING: Config values are not defined in any .rc file: opt
ERROR: /tmp/tensorflow/tensorflow/core/BUILD:1416:1: no such target '//tensorflow/tools/git:gen/spec.json': target 'gen/spec.json' not declared in package 'tensorflow/tools/git' defined by /tmp/tensorflow/tensorflow/tools/git/BUILD and referenced by '//tensorflow/core:version_info_gen'.
ERROR: /tmp/tensorflow/tensorflow/core/BUILD:1416:1: no such target '//tensorflow/tools/git:gen/head': target 'gen/head' not declared in package 'tensorflow/tools/git' defined by /tmp/tensorflow/tensorflow/tools/git/BUILD and referenced by '//tensorflow/core:version_info_gen'.
ERROR: /tmp/tensorflow/tensorflow/core/BUILD:1416:1: no such target '//tensorflow/tools/git:gen/branch_ref': target 'gen/branch_ref' not declared in package 'tensorflow/tools/git' defined by /tmp/tensorflow/tensorflow/tools/git/BUILD and referenced by '//tensorflow/core:version_info_gen'.
ERROR: Analysis of target '//tensorflow/core/profiler:profiler' failed; build aborted.
INFO: Elapsed time: 167.083s
or just copy the command mentioned here
bazel build --config opt third_party/tensorflow/core/profiler/...
WARNING: Output base '/home/USERNAME/.cache/bazel/_bazel_USERNAME/e5cce820cc082410b4fcc604db349066' is on NFS. This may lead to surprising failures and undetermined behavior.
WARNING: Config values are not defined in any .rc file: opt
ERROR: no targets found beneath 'third_party/tensorflow/core/profiler'.

I think the path is not right. You should use pathtensorflow/core/profiler/, if your current directory is the cloned tensorflow repository.

Run ./configure script from your tensorflow directory to set the environment variables.

How to package a linked DLL and a pyd file into one self contained pyd file?

I am building a python module with Cython that links against a DLL file. In order to succesfully import my module I need to have the DLL in the Windows search path. Otherwise, the typical error message is:
ImportError: DLL load failed: The specified module could not be found.
Is there a way to packaged the DLL directly into the produced pyd file to make the distribution easier?
One example of this is with the OpenCV distribution, where a (huge) pyd file is distributed and is the only file needed for the Python bindings to work.

Python's packaging & deployment is still a pain point for many of us. There is just not a silver bullet. Here are several methods:
1. OpenCV build method
The method is decribed here : https://opencv-python-tutroals.readthedocs.org/en/latest/py_tutorials/py_bindings/py_bindings_basics/py_bindings_basics.html#bindings-basics
OpenCV generates these wrapper functions automatically from the C++
headers using some Python scripts which are located in
modules/python/src2.
Basically it parse the header files and generate the static PyObject keywords whenever it's needed. Once the header are created appropriately, it just call python setup. Honestly, it might work, but I would not advise this method.
2. Makefiles
If you already use Makefile, just create a rule to place your lib accordinlgy. Example, from my own code :
setup.py
from distutils.core import setup, Extension
setup(name='sha1_hmac', version='1.0', \
ext_modules=[Extension('sha1_hmac',
library_dirs=['C:\MinGW\lib'],
sources= ['../tools/sha1.c','sha1_hmac.c'])])
Makefile
# The hmac generation used by the webserver is done
# using the sha1.c implementation. There is a binding needed to
# glue the C code with the python script
libsha1_hmac:
ifeq ($(OS), Windows_NT)
$(PYTHON) setup.py build --compiler=mingw32
else
$(PYTHON) setup.py install --home=$(CURDIR)
endif
.PHONY: webserver
webserver: libsha1_hmac
ifeq ($(OS), Windows_NT)
mv $(shell find build -type f -name "sha1*.pyd") $(LIB)
else
mv -f $(shell find $(LIB)/python -type f -name "sha1*.so") $(LIB)
endif
$(PYTHON) hmac_server.py
3. Modern deployement tools
There are several new tools to deploy python applications, namely wheels which seem to gain traction. I don't use it, but it look like it can ease up your bundling problem :
How can I make a Python Wheel from an existing native library?
Once it wheeled, you can install it like this : pip install some-package.whl

compiling libtorrent Rasterbar for mavericks

I just compiled and installed boost from source using
$pwd
/Downloads/boost_1_58_0
./b2 threading=multi link=static runtime-link=static cxxflags="-stdlib=libstdc++" linkflags="-stdlib=libstdc++"
and got message after build completed,
The Boost C++ Libraries were successfully built!
The following directory should be added to compiler include paths:
/Downloads/boost_1_58_0
The following directory should be added to linker library paths:
/Downloads/boost_1_58_0/stage/lib
now that I was trying to install lib torrent's python pending using sudo pip install .
I got error message b2: command not found since I knew where the b2 command was in my location from where I build boost, i updated setup.py to the specific path,
but still when I try to sudo pip install . python binding of lib torrent I get below message.
Complete output from command python setup.py egg_info:
Unable to load Boost.Build: could not find "boost-build.jam"
---------------------------------------------------------------
BOOST_ROOT must be set, either in the environment, or
on the command-line with -sBOOST_ROOT=..., to the root
of the boost installation.
Attempted search from /private/tmp/pip-OWjwyj-build up to the root
at /Downloads/share/boost-build
and in these directories from BOOST_BUILD_PATH and BOOST_ROOT: /usr/share/boost-build.
Please consult the documentation at 'http://www.boost.org'.
/Downloads/boost_1_58_0/b2 boost=source link=static geoip=static boost-link=static release optimization=space stage_module --abbreviate-paths -j4
build failed
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-OWjwyj-build
after following the steps mentioned in first answer I tried to run make but it gave following error:
In file included from ../include/libtorrent/torrent_handle.hpp:55:
../include/libtorrent/storage.hpp:346:3: error: no template named 'scoped_ptr'
in namespace 'boost'; did you mean 'boost::asio::detail::scoped_ptr'?
boost::scoped_ptr<storage_interface> m_storage;
^~~~~~~~~~~~~~~~~
boost::asio::detail::scoped_ptr
/opt/local/include/boost/asio/detail/scoped_ptr.hpp:27:7: note:
'boost::asio::detail::scoped_ptr' declared here
class scoped_ptr
^
In file included from piece_picker.cpp:41:
In file included from ../include/libtorrent/aux_/session_impl.hpp:66:
In file included from ../include/libtorrent/torrent_handle.hpp:55:
../include/libtorrent/storage.hpp:279:53: error: no viable overloaded
'operator->'
error_code const& error() const { return m_storage->error(); }
~~~~~~~~~^
/opt/local/include/boost/asio/detail/scoped_ptr.hpp:49:6: note: candidate
function not viable: 'this' argument has type 'const
boost::scoped_ptr<storage_interface>', but method is not marked const
T* operator->()
^
In file included from piece_picker.cpp:41:
In file included from ../include/libtorrent/aux_/session_impl.hpp:66:
In file included from ../include/libtorrent/torrent_handle.hpp:55:
../include/libtorrent/storage.hpp:280:59: error: no viable overloaded
'operator->'
...std::string const& error_file() const { return m_storage->error_file(); }
~~~~~~~~~^
/opt/local/include/boost/asio/detail/scoped_ptr.hpp:49:6: note: candidate
function not viable: 'this' argument has type 'const
boost::scoped_ptr<storage_interface>', but method is not marked const
T* operator->()
^
In file included from piece_picker.cpp:41:
../include/libtorrent/aux_/session_impl.hpp:624:4: error: no template named
'scoped_ptr' in namespace 'boost'; did you mean
'boost::asio::detail::scoped_ptr'?
boost::scoped_ptr<boost::thread> m_thread;
^~~~~~~~~~~~~~~~~
boost::asio::detail::scoped_ptr
/opt/local/include/boost/asio/detail/scoped_ptr.hpp:27:7: note:
'boost::asio::detail::scoped_ptr' declared here
class scoped_ptr
^

Since you don't seem to mind using boost-build to build, the simplest way to do it is to:
export BOOST_ROOT=/Downloads/boost_1_58_0
export BOOST_BUILD_PATH=$BOOST_ROOT/tools/build
export PATH=$PATH:$BOOST_BUILD_PATH/src/engine/bin.macosxx86_64
echo "using darwin ;" >~/user-config.jam
cd libtorrent/bindings/python
b2 boost=source
The first 4 lines is basically installing boost-build (b2). It's assuming the output directory when you built b2 was bin.macosxx86_64.
That will build the libtorrent python module, it won't install it though.