I just compiled and installed boost from source using
$pwd
/Downloads/boost_1_58_0
./b2 threading=multi link=static runtime-link=static cxxflags="-stdlib=libstdc++" linkflags="-stdlib=libstdc++"
and got message after build completed,
The Boost C++ Libraries were successfully built!
The following directory should be added to compiler include paths:
/Downloads/boost_1_58_0
The following directory should be added to linker library paths:
/Downloads/boost_1_58_0/stage/lib
now that I was trying to install lib torrent's python pending using sudo pip install .
I got error message b2: command not found since I knew where the b2 command was in my location from where I build boost, i updated setup.py to the specific path,
but still when I try to sudo pip install . python binding of lib torrent I get below message.
Complete output from command python setup.py egg_info:
Unable to load Boost.Build: could not find "boost-build.jam"
---------------------------------------------------------------
BOOST_ROOT must be set, either in the environment, or
on the command-line with -sBOOST_ROOT=..., to the root
of the boost installation.
Attempted search from /private/tmp/pip-OWjwyj-build up to the root
at /Downloads/share/boost-build
and in these directories from BOOST_BUILD_PATH and BOOST_ROOT: /usr/share/boost-build.
Please consult the documentation at 'http://www.boost.org'.
/Downloads/boost_1_58_0/b2 boost=source link=static geoip=static boost-link=static release optimization=space stage_module --abbreviate-paths -j4
build failed
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-OWjwyj-build
after following the steps mentioned in first answer I tried to run make but it gave following error:
In file included from ../include/libtorrent/torrent_handle.hpp:55:
../include/libtorrent/storage.hpp:346:3: error: no template named 'scoped_ptr'
in namespace 'boost'; did you mean 'boost::asio::detail::scoped_ptr'?
boost::scoped_ptr<storage_interface> m_storage;
^~~~~~~~~~~~~~~~~
boost::asio::detail::scoped_ptr
/opt/local/include/boost/asio/detail/scoped_ptr.hpp:27:7: note:
'boost::asio::detail::scoped_ptr' declared here
class scoped_ptr
^
In file included from piece_picker.cpp:41:
In file included from ../include/libtorrent/aux_/session_impl.hpp:66:
In file included from ../include/libtorrent/torrent_handle.hpp:55:
../include/libtorrent/storage.hpp:279:53: error: no viable overloaded
'operator->'
error_code const& error() const { return m_storage->error(); }
~~~~~~~~~^
/opt/local/include/boost/asio/detail/scoped_ptr.hpp:49:6: note: candidate
function not viable: 'this' argument has type 'const
boost::scoped_ptr<storage_interface>', but method is not marked const
T* operator->()
^
In file included from piece_picker.cpp:41:
In file included from ../include/libtorrent/aux_/session_impl.hpp:66:
In file included from ../include/libtorrent/torrent_handle.hpp:55:
../include/libtorrent/storage.hpp:280:59: error: no viable overloaded
'operator->'
...std::string const& error_file() const { return m_storage->error_file(); }
~~~~~~~~~^
/opt/local/include/boost/asio/detail/scoped_ptr.hpp:49:6: note: candidate
function not viable: 'this' argument has type 'const
boost::scoped_ptr<storage_interface>', but method is not marked const
T* operator->()
^
In file included from piece_picker.cpp:41:
../include/libtorrent/aux_/session_impl.hpp:624:4: error: no template named
'scoped_ptr' in namespace 'boost'; did you mean
'boost::asio::detail::scoped_ptr'?
boost::scoped_ptr<boost::thread> m_thread;
^~~~~~~~~~~~~~~~~
boost::asio::detail::scoped_ptr
/opt/local/include/boost/asio/detail/scoped_ptr.hpp:27:7: note:
'boost::asio::detail::scoped_ptr' declared here
class scoped_ptr
^
Since you don't seem to mind using boost-build to build, the simplest way to do it is to:
export BOOST_ROOT=/Downloads/boost_1_58_0
export BOOST_BUILD_PATH=$BOOST_ROOT/tools/build
export PATH=$PATH:$BOOST_BUILD_PATH/src/engine/bin.macosxx86_64
echo "using darwin ;" >~/user-config.jam
cd libtorrent/bindings/python
b2 boost=source
The first 4 lines is basically installing boost-build (b2). It's assuming the output directory when you built b2 was bin.macosxx86_64.
That will build the libtorrent python module, it won't install it though.
Related
I try to make an EXE on windows with Nuitka but always get error like this: CALL_FUNCTION_WITH_ARGS14
I googled 'CALL_FUNCTION_WITH_ARGS14' but nothing shows up. Does anyone have similar issue?
I am using Python 3.9 and 3.10 and this is my command
nuitka --standalone --onefile --mingw64 --show-memory --show-progress --follow-imports --output-dir=out Demo_Hello_World.py
The Demo_Hello_World.py is from PySimpleGUI/DemoPrograms/Demo_Hello_World.py
ValueTraceLoopComplete 38798 30843 7955
ValueTraceLoopIncomplete 16811 14635 2176
ValueTraceMerge 470835 399699 71136
ValueTraceUninitialized 82073 66112 15961
ValueTraceUnknown 238500 209701 28799
Nuitka:INFO: Running data composer tool for optimal constant value handling.
Nuitka:INFO: Running C compilation via Scons.
Nuitka-Scons:INFO: Backend C compiler: gcc (gcc).
Nuitka-Scons:INFO: Backend linking program with 14 modules (no progress information available).
module.PySimpleGUI.PySimpleGUI.c: In function 'impl_PySimpleGUI$PySimpleGUI$$$function__578__github_issue_post_make_markdown':
module.PySimpleGUI.PySimpleGUI.c:549680:35: warning: implicit declaration of function 'CALL_FUNCTION_WITH_ARGS14'; did you mean 'CALL_FUNCTION_WITH_ARGS10'? [-Wimplicit-function-declaration]
549680 | tmp_assign_source_2 = CALL_FUNCTION_WITH_ARGS14(unicode_builtin_format, args);
| ^~~~~~~~~~~~~~~~~~~~~~~~~
| CALL_FUNCTION_WITH_ARGS10
module.PySimpleGUI.PySimpleGUI.c:549680:33: warning: assignment to 'PyObject *' {aka 'struct _object *'} from 'int' makes pointer from integer without a cast [-Wint-conversion]
549680 | tmp_assign_source_2 = CALL_FUNCTION_WITH_ARGS14(unicode_builtin_format, args);
| ^
Nuitka-Scons:INFO: Running 'C:\\Users\\nytrg\\AppData\\Local\\Nuitka\\Nuitka\\gcc\\x86_64\\11.3.0-14.0.3-10.0.0-msvcrt-r3\\mingw64\\bin\\gcc.exe -o "F:\\PySimpleGUI\\DemoPrograms\\out\\Demo_Hello_World.dist\\Demo_Hello_World.exe" -fuse-linker-plugin -flto=6 -fpartial-inlining -freorder-functions -Wl,--exclude-all-symbols -Wl,--out-implib,.\\import.lib -municode -O3 -s -Wl,--enable-auto-import -Wl,--stack,9863168 -static-libgcc #".\\#link_input.txt" -LC:\\Users\\nytrg\\AppData\\Local\\Programs\\Python\\Python39\\libs -lm -lpython39' took 164.76 seconds
c:/users/nytrg/appdata/local/nuitka/nuitka/gcc/x86_64/11.3.0-14.0.3-10.0.0-msvcrt-r3/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\nytrg\AppData\Local\Temp\cccIcn6f.ltrans80.ltrans.o:<artificial>:(.text+0x32b8): undefined reference to `CALL_FUNCTION_WITH_ARGS14'
collect2.exe: error: ld returned 1 exit status
scons: *** [F:\PySimpleGUI\DemoPrograms\out\Demo_Hello_World.dist\Demo_Hello_World.exe] Error 1
I had the same problem and the only thing that helped me was compiling in a clean Python venv, the issue may be due to some package conflict but requires further investigation.
So to sum that up:
Create a clean venv
Activate the venv using the appropriate script (Refer to the table below)
Install just the necessary prerequisites through pip (including Nuitka of course)
Try compiling with nuitka again
My application, WSA Sideloader is a PySimpleGUI program compiled using Nuitka. This is the command I used:
nuitka --standalone sideloader.py --enable-plugin=tk-inter --windows-disable-console
Of course the name of your main Python file should be used after --standalone.
Note that if it requires other files to function you will need to add them to the dist folder once it's compiled.
I have built a cython package on windows using mingw.
Everything was fine, the only warnings i got during the compilation were:
Mylib\__init__.c: In function '__Pyx_ImportType':
\__init__.c:126091:13: warning: unknown conversion type character 'z' in format [-Wformat=]
126091 | "%s.%s size changed, may indicate binary incompatibility. "
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
\__init__.c:126092:24: note: format string is defined here
126092 | "Expected %zd from C header, got %zd from PyObject",
| ^
\__init__.c:126091:13: warning: unknown conversion type character 'z' in format [-Wformat=]
126091 | "%s.%s size changed, may indicate binary incompatibility. "
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
\__init__.c:126092:47: note: format string is defined here
126092 | "Expected %zd from C header, got %zd from PyObject",
| ^
\__init__.c:126091:13: warning: too many arguments for format [-Wformat-extra-args]
126091 | "%s.%s size changed, may indicate binary incompatibility. "
but for the moment, since the compilation was ok (Only these warnings and no errors), i decided to test the library built in cython installing via pip the .whl package that was generated.
The rest of the output of the compilation is the following:
writing build\temp.win-amd64-3.8\Release\mylib\mylib.cp38-win_amd64.def
creating build\lib.win-amd64-3.8
D:\mingw64\mingw64\bin\gcc.exe -shared -s build\temp.win-amd64-3.8\Release\mylib\__init__.o build\temp.win-amd64-3.8\Release\mylib\mylib.cp38-win_amd64.def -L./ -LC:\Users\Utente\AppData\Local\Programs\Python\Python38\libs -LC:\Users\Utente\AppData\Local\Programs\Python\Python38\PCbuild\amd64 -llmylib -lpython38 -lvcruntime140 -o build\lib.win-amd64-3.8\mylib.cp38-win_amd64.pyd -static-libgcc -static-libstdc++ -Wl,-Bstatic,--whole-archive -lwinpthread -Wl,--no-whole-archive "-L ."
installing to build\bdist.win-amd64\wheel
running install
running install_lib
creating build\bdist.win-amd64
creating build\bdist.win-amd64\wheel
copying build\lib.win-amd64-3.8\mylib.cp38-win_amd64.pyd -> build\bdist.win-amd64\wheel\.
running install_egg_info
Copying Mylib.egg-info to build\bdist.win-amd64\wheel\.\Mylib-1.0.0-py3.8.egg-info
running install_scripts
adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
creating build\bdist.win-amd64\wheel\Mylib-1.0.0.dist-info\WHEEL
creating 'dist\Mylib-1.0.0-cp38-cp38-win_amd64.whl' and adding 'build\bdist.win-amd64\wheel' to it
adding 'mylib.cp38-win_amd64.pyd'
adding 'mylib-1.0.0.dist-info/LICENSE'
adding 'mylib-1.0.0.dist-info/METADATA'
adding 'mylib-1.0.0.dist-info/WHEEL'
adding 'mylib-1.0.0.dist-info/top_level.txt'
adding 'mylib-1.0.0.dist-info/RECORD'
removing build\bdist.win-amd64\wheel
c:\users\utente\desktop\mylib-main\.eggs\wheel-0.37.1-py3.8.egg\wheel\bdist_wheel.py:80: RuntimeWarning: Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect
if get_flag('Py_DEBUG',
Now, i moved to dist/ and i run:
py -3.8 -m pip install Mylib-1.0.0-cp38-cp38-win_amd64.whl
Everything was fine.
I tapped:
py -3.8 -m pip show Mylib
And the output was:
Name: Mylib
Version: 1.0.0
Summary: My library
Home-page: https://github.com/Mylib
Author: Me
Author-email: Me#gmail.com
License: UNKNOWN
Location: c:\users\utente\appdata\local\programs\python\python38\lib\site-packages
Requires:
Required-by:
Now, i moved to c:\users\utente\appdata\local\programs\python\python38\lib\site-packages
and i saw:
a .PYD file: mylib.cp38-win_amd64
and a directory: mylib.cp38-win_amd64
Inside mylib.cp38-win_amd64 there are only these files:
INSTALLER.txt
METADATA.txt
LICENSE.txt
RECORD.txt
WHEEL.txt
When i run on python3.8:
import mylib
this error occurs:
"ImportError: DLL load failed while importing mylib: The specified module could not be found."
I tried to run from the same directory of the pyd file. Same error.
I also tried to import the path with:
import mysys
sys.path.append('directory of the pyd')
Again same error.
For what i have read there could be a problem with other dll dependencies.
So, i used dumpbin on the pyd and this was the output:
File Type: DLL
Image has the following dependencies:
python38.dll
libmylib.dll
KERNEL32.dll
msvcrt.dll
Summary
1000 .CRT
1000 .bss
6000 .data
1000 .edata
4000 .idata
2000 .pdata
5000 .rdata
1000 .reloc
1000 .rsrc
51000 .text
1000 .tls
2000 .xdata
If the problem is a problem of other dll dependencies i just tought that i needed to import the dlls paths. So ,as suggested by fix pyd with other dependencies i used these commands before importing mylib:
import os
os.add_dll_directory('directory of all the dlls previously showed')
But again, same error, so the problem is not a problem of other dll dependencies.
The only thing i can imagine this problem come from is the warnings previously showed. And in particular the last line of the compilation:
c:\users\utente\desktop\mylib-main\.eggs\wheel-0.37.1-py3.8.egg\wheel\bdist_wheel.py:80: RuntimeWarning: Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect
if get_flag('Py_DEBUG',
That could cause the fact that, maybe (?), the library should be imported in debug mode (?), and so the pyd file (for what i have read) should have the signature mylib_d.pyd(?). But i tried to rename it in this way and it seems it does not work either.
NOTE:
On unix i was able to cythonize everything and import mylib in python.
Edit 1:
It seems that if i call even only a single function of the dll library mylib that is compiled with the code in cython inside a def function defined in the pyx module then that error appears
I found the solution:
Instead of compiling a .dll shared library i compiled a .lib library.
Then, when compiling in cython i linked the .lib library.
Usign dumpbin on the generated .pyd file it seems that mylib.pyd didn't depends to any libmylib.dll library and i can import it without any problem and calling functions that call in turns functions from mylib.lib
I want to build pyobjc-7.3, because it has fix for send2trash.
Classic building on BigSur 20.5.0 is strait forward.
cd pyobjc-7.3/pyobjc-framework-Cocoa
python3 setup.py build
though once I run same build inside nix-shell magic happens.
nix-shell -p pkgs.python39Packages.setuptools
python3 setup.py build
clang-7: error: argument unused during compilation:
'-fno-strict-overflow' [-Werror,-Wunused-command-lin\ e-argument]
ok. no big deal. let's disable warning.
CFLAGS="-Wno-unused-argument" python3 setup.py build
what? now clang is like a blind kitten.
Modules/pyobjc-api.h:19:10: fatal error: 'objc/objc.h' file not found
#include <objc/objc.h>
-isysroot option and -I has no effect.
-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk
I noticed lots additions to -I flag in clang such as:
-iwithprefix /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include
it helps clang to find objc header file, though this is not the end of the story.
Modules/pyobjc-api.h:21:9: fatal error: 'Foundation/Foundation.h' file
not found
how come?!
oh there is another header files of special kind - frameworks. Wheel reinvention...
clang, take another argument
-iframeworkwithsysroot /System/Library/Frameworks
Here I get tons of type errors and I run out of ideas what could I try next:
/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSString.h:138:1:
error:
function cannot return function type 'NSComparisonResult' (aka 'int (int)')
(NSComparisonResult)compare:(NSString *)string options:(NSStringCompareOptions)mask range:(NSR...
After days of trying I've found solution.
There are a few bugs are causing the problems:
First is nix provides older (10.12) sdk while setup.py thinks is 10.15.
This enables CPP sections for unsupported SDK API therefore type errors.
Following hack makes pyobjc to think that SDK is older that it is.
with pkgs;
with pkgs.lib;
with pkgs.python39Packages;
let
pyobjc-core = buildPythonPackage rec {
pname = "pyobjc-core";
version = "7.3";
name = "${pname}-${version}";
src = pkgs.python39Packages.fetchPypi {
pname = "pyobjc-core";
inherit version;
sha256 = "0x3msrzvcszlmladdpl64s48l52fwk4xlnnri8daq2mliggsx0ah";
};
preBuild=''
export SDKROOT="/Library/Developer/CommandLineTools/SDKs/MacOSX10.12.sdk"
Second problem is with header discovery and overstrict lint from python nix
CFLAGS = "-iwithsysroot /usr/include -Wno-unused-argument";
Third problem big sur linkder is dynamic and ffi libray is not found.
Providing through nix derivation
buildInputs = [ pkgs.libffi ];
Forth problem is tests are broken
doCheck = false;
If I build CPython from source on Windows I encounter problems when I want to pip install a package that contains a C-Extension. It seems like the error happens while linking the libraries.
For example when installing cython (but it also crashes with the same error on other C extension packages):
LINK : fatal error LNK1104: cannot open file 'python38.lib'
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.23.28105\bin\HostX86\x86\link.exe' failed with exit status 1104
The reason why it cannot open the "python38.lib" is because the ".lib" file in debug mode is called "python38_d.lib".
A minimal reproducible example would be (on the command-line) based on the Quick Reference of the CPython developer guide:
git clone --branch v3.8.0 https://github.com/python/cpython.git
cd cpython
git checkout v3.8.0
.\PCbuild\build.bat -e -d
.\PCbuild\win32\python_d.exe -m ensurepip
.\PCbuild\win32\python_d.exe -m pip install pip --upgrade -vv
.\PCbuild\win32\python_d.exe -m pip install setuptools --upgrade -vv
.\PCbuild\win32\python_d.exe -m pip install cython -vv
The resulting distutils.sysconfig.get_config_vars() is:
{'BINDIR': '...\\cpython\\PCbuild\\win32',
'BINLIBDEST': ...\\cpython\\Lib',
'EXE': '.exe',
'EXT_SUFFIX': '_d.cp38-win32.pyd',
'INCLUDEPY': '...\\cpython\\include;...\\cpython\\PC',
'LIBDEST': '...\\cpython\\Lib',
'SO': '_d.cp38-win32.pyd',
'VERSION': '38',
'exec_prefix': '...\\cpython',
'prefix': '...\\cpython',
'srcdir': '...\\cpython'}
Is there something I'm missing? Is building C-Extensions on Python-debug builds on Windows simply not supported? If it is supported: how would I do it?
This code is a little bit hacky, but works for me on MSVC19, allowing to debug application without building debug python libraries.
#ifdef _DEBUG
#define _DEBUG_WAS_DEFINED
#undef _DEBUG
#endif
#include "Python.h"
#ifdef _DEBUG_WAS_DEFINED
#define _DEBUG
#undef _DEBUG_WAS_DEFINED
#endif
Linking against pythonXY.lib is a little bit sneaky on Windows. When you look at the command line for linking, you will see that no python-library is passed to the linker, i.e. 'link.exe`. Note: This is also the case for Linux, but on Linux one doesn't have to because the needed symbols will be provided by the python-executable.
However, it is easy to check via dumpbin /dependents resulting.pyd, that there is a dependency on pythonXY.dll, also adding extra_link_args = ["/VERBOSE:LIB"] to extension-definition and triggering verbose-mode of the linker will show that the linker uses pythonXY.lib.
The sneaky part: Microsoft Compler has a convinience-pragma #pragma comment(lib, ...) to automatically trigger linking of a library, which is also used in Python-headers:
# if defined(_MSC_VER)
/* So MSVC users need not specify the .lib
file in their Makefile (other compilers are
generally taken care of by distutils.) */
# if defined(_DEBUG)
# pragma comment(lib,"python39_d.lib")
# elif defined(Py_LIMITED_API)
# pragma comment(lib,"python3.lib")
# else
# pragma comment(lib,"python39.lib")
# endif /* _DEBUG */
# endif /* _MSC_VER */
As you can see, to link against the debug version, one needs to define _DEBUG.
_DEBUG is automatically defined by distutils on Windows, if build_ext is called with options --debug, e.g.
python setup.py build_ext -i --debug
That can be translated to pip as
pip install --global-option build --global-option --debug XXXXX
which can be interpreted roughly as: trigger build command (which also includes build_ext-command) with option --debug prior to installing.
Another subtility when building debug C-extensions, there is more to it on Windows:
#ifdef _DEBUG
# define Py_DEBUG
#endif
Having defined Py_DEBUG macro meant incompartible ABIs until Python3.8, because it also assumed Py_TRACE_REFS which leads to different memory layout of PyObject and some additional functionality missing in the release-mode.
However, since Python3.8, one probably can get away with it by providing the missing pythonXY_d.lib/pythonYX.lib as a symlink linking to another version.
The following procedure fails. Am I missing something?
Install various Ubuntu packages (prerequisites for compilation)
Get http://downloads.sourceforge.net/wxpython/wxPython-src-3.0.1.1.tar.bz2
Uncompress to wxPython-src-3.0.1.1/
Create new virtualenv called test
Activate test virtualenv
In terminal, from wxPython-src-3.0.1.1/:
./configure --prefix=/home/username/.virtualenvs/test --with-gtk2 --enable-unicode --with-opengl
#lots of output, confirms "Configured wxWidgets 3.0.1 for `x86_64-unknown-linux-gnu'"
make install
#lots of output, confirms:
# The installation of wxWidgets is finished. On certain
# platforms (e.g. Linux) you'll now have to run ldconfig
# if you installed a shared library and also modify the
# LD_LIBRARY_PATH (or equivalent) environment variable.
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.virtualenvs/test/lib
# don't run ldconfig since that is a system tool, not appropriate for virtualenv?
cd wxPython
python setup.py install
# lots of output, starting:
# WARNING: WXWIN not set in environment. Assuming '..'
# Found wx-config: /home/username/.virtualenvs/test/bin/wx-config
# Using flags: --toolkit=gtk2 --unicode=yes --version=3.0
# Preparing CORE...
# Preparing STC...
# Preparing GLCANVAS...
# Preparing GIZMOS...
# running install
# etc
The final command fails with:
src/gtk/_core_wrap.cpp:20407:7: note: ‘arg3’ was declared here
int arg3 ;
^
src/gtk/_core_wrap.cpp: In function ‘PyObject* _wrap_Image_SetAlphaBuffer(PyObject*, PyObject*, PyObject*)’:
src/gtk/_core_wrap.cpp:3747:13: warning: ‘arg3’ may be used uninitialized in this function [-Wmaybe-uninitialized]
if (ALPHASIZE != self->GetWidth() * self->GetHeight()) {
^
src/gtk/_core_wrap.cpp:20474:7: note: ‘arg3’ was declared here
int arg3 ;
^
cc1plus: some warnings being treated as errors
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
I used python setup.py install &> ~/error.txt to pass on the error messages to knowledgeable colleague who identified that C compilation was using the -Werror=format-security flag. This version of wxPython (and maybe others) cannot compile with that flag.
My $CPPFLAGS and $CFLAGS environment variables were empty. It turns out that this flag is triggered by hardening-wrapper.
So, I overrode the flag by invoking the final step as follows, and wxPython was installed successfully:
CFLAGS=-Wno-error=format-security CPPFLAGS=-Wno-error=format-security python setup.py install