We Keep Coding

Linking to system python gives 'binary is not an allowed client' error

I have just updated my Macbook to Monterey and installed XCode 13. I'm now seeing errors when trying to link my code - for example one library needs to link to the system python2.7, but gives the error:
Keiths-MacBook-Pro:libcdb keith$ make rm -f libcdb.1.0.0.dylib
libcdb.dylib libcdb.1.dylib libcdb.1.0.dylib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++
-stdlib=libc++ -headerpad_max_install_names -arch x86_64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk
-mmacosx-version-min=10.13 -Wl,-rpath,#executable_path/../Frameworks -Wl,-rpath,/usr/local/Qt-5.15.7/lib -single_module -dynamiclib -compatibility_version 1.0 -current_version 1.0.0 -install_name libcdb.1.dylib -o libcdb.1.0.0.dylib release/db.o release/KDTree.o release/db_Wlist.o release/db_VSeg.o
release/db_View.o release/db_ViaInst.o release/db_ViaGen.o
release/db_Via.o release/db_Vertex.o release/db_Vector.o
release/db_Utils.o release/db_Trapezoid.o release/db_Transform64.o
release/db_Transform.o release/db_Techfile.o release/db_Style.o
release/db_Signal.o release/db_Shape.o release/db_SegParam.o
release/db_Segment.o release/db_Rectangle.o release/db_Rect.o
release/db_QTree.o release/db_Property.o release/db_Polygon.o
release/db_PointList.o release/db_Point.o release/db_Pin.o
release/db_Path.o release/db_ObjList.o release/db_Obj.o
release/db_Net.o release/db_Mst.o release/db_Mpp.o release/db_Lpp.o
release/db_Line.o release/db_Library.o release/db_Layer.o
release/db_Label.o release/db_InstPin.o release/db_Inst.o
release/db_HVTree.o release/db_HSeg.o release/db_HierObj.o
release/db_Group.o release/db_Ellipse.o release/db_Edge.o
release/db_CellView.o release/db_Cell.o release/db_Array.o
release/db_Arc.o -F/usr/local/Qt-5.15.7/lib -L../libcpp/release -lcpp
-L/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config
-lpython2.7 -framework QtWidgets -framework QtGui -framework AppKit -framework Metal -framework QtNetwork -framework QtCore -framework DiskArbitration -framework IOKit -framework OpenGL -framework AGL
ld: cannot link directly with dylib/framework, your binary is not an
allowed client of
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/libpython2.7.tbd
for architecture x86_64 clang: error: linker command failed with exit
code 1 (use -v to see invocation) make: ***
[release/libcdb.1.0.0.dylib] Error 1
Given that I have recompiled (successfully) the Qt libs and the code for this library, why is it giving me this 'your binary is not an allowed client' error?
As far as I can see the python2.7 paths have not changed, so the error is baffling.

So the quick and diirty fix is to edit the Python.tdb file that is located at:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/System/Library/Frameworks/Python.framework/Versions/Current/Python.tdb
And add your library/executable targets to the clients list.
Of course, there is a reason Apple are doing this - python2 is deprecated and sooner or later they will drop it. But until they do, this works.

What fixed it for me was to drop the Python2 framework and link with the framework provided by my version of Python3 installed via Brew.
Locate your installation of Python3 by using this command in Terminal: ls -la $(which python3).
The framework is in the "Frameworks" folder located one level above "bin", for example: /usr/local/Cellar/python#3.9/3.9.7_1/Frameworks
Once your have the location of the Python3 framework, add it as a framework in your XCode project.
In the Build Settings, don't forget to add:
the Frameworks folder location in Framework Search Path (e.g. "/usr/local/Cellar/python#3.9/3.9.7_1/Frameworks")
the framework's Headers folder in Header Search Path (e.g. "/usr/local/Cellar/python#3.9/3.9.7_1/Frameworks/Python.framework/Headers")
Some functions changed in version 3 so you'll need to update some of your Python function calls.

Why does this Makefile rebuild every time?

I have a Makefile to build some simple Python bindings. Unfortunately, upon a plain make or make all it will rebuild every time, even when py11_bindings.cpp was not changed. I checked whether maybe the source file would accidentally be touched or something, but that's not the case as far as I see.
PYTHON = /Library/Frameworks/Python.framework/Versions/3.5/bin
CPP = c++
INC = -I/software/pybind11/include -I/software/eigen
PYTHONCFG = `$(PYTHON)/python3.5-config --cflags --ldflags`
SRC = py11_bindings.cpp
TARGET = _chain.so
all: $(SRC)
$(CPP) -O3 -shared -std=c++11 $(INC) $(PYTHONCFG) $^ -o $(TARGET)
clean:
rm $(TARGET)
I have absolutely no clue why this should happen.

I'm not a makefile expert, so maybe I am not using the correct terms.
However: Your all: defines to create the source file, when it should list the targets to create. Then you should, for each target, list the dependencies and describe, how to create it.
So, your makefile should look like this:
all: $(TARGET)
$(TARGET) : $(SRC)
$(CPP) -O3 -shared -std=c++11 $(INC) $(PYTHONCFG) $^ -o $(TARGET)

Installing the "ring.cx SIP client" on a Raspberry PI

The Situation
I would like to get terminal-based (headless) SIP calls working on my Raspberry Pi and I already tried this using linphone:
RaspberryPI: Making SIP outbound calls using linphonec or an alternative SIP soft phone
Since I am currently stuck there I wanted to try another option which was SFLPhone. They pointed me towards the ring software project which offers a daemon dring which allows making SIP calls using a scripting interface:
Indeed, the daemon can run standalone and be controlled using the DBus API.
Note the project have been renamed to "Ring" (version is bumped to 2.x). Experimental packages are available at http://ring.cx/en/documentation/linux-installation
A major feature of Ring
2.x is the optional "DHT" account type allowing to make calls without any SIP server.
There are many other enhancements such as ICE support, UPnP support, stability improvements etc.
(note clients are being re-written (GTK3, Qt5) and there is a new OS X client, they are not yet feature complete and in heavy development.)
The new daemon dring source Git repo URI is : https://gerrit-ring.savoirfairelinux.com/ring .
The DBus API is mostly the same as before. In the tools/dringctrl directory you will find an example python client that we use for testing (uses python3-dbus).
We are willing to fix any bugs you may find, the daemon bugtracker is here : https://projects.savoirfairelinux.com/projects/ring-daemon/issues
Also look at https://projects.savoirfairelinux.com/projects/ring/wiki for build instruction etc.
Regards and good luck for your embedded project,
A. B.
Compiling the dependencies
I tried to compile the dependencies for the project like stated in the README:
git clone https://gerrit-ring.savoirfairelinux.com/ring
cd ring
Compile the dependencies first
cd ../contrib/
rm -fr native/ && mkdir native
cd native
../bootstrap
make
I got this error:
libvpx.webm-4640a0c4804b/third_party/googletest/src/include/gtest/gtest.h
mv libvpx-4640a0c4804b49f1870d5a2d17df0c7d0a77af2f libvpx && touch libvpx
cd libvpx && CROSS= ./configure --target=armv7-linux-gcc \
--as=yasm --disable-docs --disable-examples --disable-unit-tests --disable-install-bins --disable-install-docs --enable-realtime-only --enable-error-concealment --disable-runtime-cpu-detect --disable-webm-io --enable-pic --prefix=/home/pi/ring/contrib/arm-linux-gnueabihf
disabling docs
disabling examples
disabling unit_tests
disabling install_bins
disabling install_docs
enabling realtime_only
enabling error_concealment
disabling runtime_cpu_detect
disabling webm_io
enabling pic
Configuring selected codecs
enabling vp8_encoder
enabling vp8_decoder
enabling vp9_encoder
enabling vp9_decoder
Configuring for target 'armv7-linux-gcc'
enabling armv7
enabling neon
enabling neon_asm
enabling media
Unable to invoke compiler: arm-none-linux-gnueabi-gcc -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
Configuration failed. This could reflect a misconfiguration of your
toolchains, improper options selected, or another problem. If you
don't see any useful error messages above, the next step is to look
at the configure error log file (config.log) to determine what
configure was trying to do when it died.
../../contrib/src/vpx/rules.mak:105: recipe for target '.vpx' failed
make: *** [.vpx] Error 1
Compiling ring
Despite compiling the dependencies failed I did attempt to compile ring:
git clone https://gerrit-ring.savoirfairelinux.com/ring
cd ring
./autogen.sh
./configure
make
make install
This caused the following error:
checking for PJPROJECT... no
configure: error: Missing pjproject files
pi#phone ~/ring $ make
make: *** No targets specified and no makefile found. Stop.
pi#phone ~/ring $ make install
make: *** No rule to make target 'install'. Stop.
So currently I am stuck and I fear that I will not be able to go beyond the current state of my project (🎥):
Edit: Now without the video codecs (like aberaud suggested) I run into the following error:
/bin/bash ../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I.. -I../include/opendht -I/home/pi/sip-desaster/ring/contrib/arm-linux-gnueabihf/include -fPIC -I/home/pi/sip-desaster/ring/contrib/arm-linux-gnueabihf/include -g -fPIC -O3 -std=c++0x -I/home/pi/sip-desaster/ring/contrib/arm-linux-gnueabihf/include -g -fPIC -O3 -std=c++0x -c -o libopendht_la-dht.lo `test -f 'dht.cpp' || echo './'`dht.cpp
libtool: compile: g++ -DHAVE_CONFIG_H -I. -I.. -I../include/opendht -I/home/pi/sip-desaster/ring/contrib/arm-linux-gnueabihf/include -fPIC -I/home/pi/sip-desaster/ring/contrib/arm-linux-gnueabihf/include -g -fPIC -O3 -std=c++0x -I/home/pi/sip-desaster/ring/contrib/arm-linux-gnueabihf/include -g -fPIC -O3 -std=c++0x -c dht.cpp -fPIC -DPIC -o libopendht_la-dht.o
In file included from ../include/opendht/dht.h:29:0,
from dht.cpp:27:
../include/opendht/infohash.h:58:22: error: expected initializer before ‘:’ token
dht.cpp:3105:1: error: expected ‘}’ at end of input
Makefile:386: recipe for target 'libopendht_la-dht.lo' failed
make[2]: *** [libopendht_la-dht.lo] Error 1
make[2]: Leaving directory '/home/pi/sip-desaster/ring/contrib/native/opendht/src'
Makefile:395: recipe for target 'install-recursive' failed
make[1]: *** [install-recursive] Error 1
make[1]: Leaving directory '/home/pi/sip-desaster/ring/contrib/native/opendht'
../../contrib/src/opendht/rules.mak:28: recipe for target '.opendht' failed
make: *** [.opendht] Error 2

Contrib
The "contrib" part of the Ring build is about building dependencies that are not available on the target system, mostly used to build full working packages when cross-compiling or for systems without proper dependency management (basically all OSs except Linux distros).
When in the contrib/native directory, you can run make list to see the list of packages to be built. Since you are cross-compiling you will need to build almost everything.
If you somehow mess up the contrib build you can safely delete the contrib/native and contrib/{target_tuple} (if any) directories and start again.
libvpx
The libvpx library is used by libav to provide the vp8 and vp9 video codecs. It's not an hard dependency and since your project doesn't use video you can safely disable it. We also encountered issues when cross compiling vpx.
In contrib/src/libav/rules.mak line 70, DEPS_libav defines the list of dependencies for libav. You can remove vpx, x264 and $(DEPS_vpx) from the list since you don't use video. You may also add speex and opus audio codecs to the list (they should be in the list but aren't, see this patch
as an example).
After cleaning contrib as described above and boostraping again, when running make list, vpx and x264 shouldn't show up in the list of "To-be-built packages". Then try to build contrib by running make.
If after trying this you encounter the same issue for other packages you may have some sort of cross compilation build path issue (I would then need more logs/details).
As a very last resort, compiling on the Pi itself (with Raspbian) is horribly slow but has the advantage to use local distro-provided dependencies and remove the hassles of cross compilation.
Good luck

I am providing an own answer which should document my steps which will hopefully lead me to the desired end result:
Download the sources
git clone https://gerrit-ring.savoirfairelinux.com/ring
cd ring
Build the contrib section
According to #aberaud's remarks I can update contrib/src/libav/rules.mak and remove any video-related dependencies (remember that I am headless):
So I changed line 70 form
DEPS_libav = zlib x264 vpx $(DEPS_vpx)
to
DEPS_libav = zlib opus speex
Now build the contrib section.
cd ../contrib/
rm -fr native/ && mkdir native
cd native
../bootstrap
make

I was hitting the same error trying to compile the contrib.
The version of Raspbian I was using came with the older version of the gcc compiler, version 4.6. After I upgraded to 4.8 it compiled instantly. Well, as instantly as anything compiles on the Pi at any rate.

how to link a Python C module

I have written a Python C module (just ffmpeg.c which depends on some FFmpeg libs and other libs) and I am wondering how to link.
I'm compiling with:
cc -std=c99 -c ../ffmpeg.c -I /usr/include/python2.7 -g
I'm trying to link right now with:
ld -shared -o ../ffmpeg.so -L/usr/local/lib -lpython2.7 -lavutil -lavformat -lavcodec -lswresample -lportaudio -lchromaprint ffmpeg.o -lc
There is no error. However, when I try to import ffmpeg in Python, I get:
ImportError: ./ffmpeg.so: undefined symbol: avio_alloc_context
Maybe this is already correct. I checked the resulting ffmpeg.so with ldd and it partly links to a wrong FFmpeg. This is strange however because of the -L/usr/local/lib which should take precedence over the default. Maybe because my custom installed FFmpeg (in /usr/local/lib) has for some reason only installed static *.a libs and *.so files have precedence over *.a files.

You should put the libraries that you're linking to after the .o file; i.e.:
ld -shared -o ../ffmpeg.so ffmpeg.o -L/usr/local/lib -lpython2.7 -lavutil -lavformat -lavcodec -lswresample -lportaudio -lchromaprint -lc
The linker is dumb, and will not link in code from static libraries that it doesn't think are needed until a dependency arises i.e. the use of avio_alloc_context happens in ffmpeg.o, and because it's not listed after the use of the library, then the linker will not consider the code in the library as needed, so it doesn't get linked in - this is the biggest reason why linking using .a files fails.
You can also use --start-group and --end-group around all the files that you are linking - this allows you to link static libraries that have cross dependencies that just seem impossible to resolve through other means:
ld -shared -o ../ffmpeg.so -L/usr/local/lib -lpython2.7 --start-group -lavutil -lavformat -lavcodec -lswresample -lportaudio -lchromaprint ffmpeg.o --end-group -lc
using .a files is a little bit trickier than .so files, but these two items generally will work around any issues you have when linking.

What is Python's equivalent of "perl -V"

The output produced by running perl -V is packed with useful information (see example below). Is there anything like it for Python?
Example output:
% perl -V
Summary of my perl5 (revision 5 version 10 subversion 1) configuration:
Platform:
osname=linux, osvers=2.6.32-5-amd64, archname=x86_64-linux-gnu-thread-multi
uname='linux brahms 2.6.32-5-amd64 #1 smp tue jun 14 09:42:28 utc 2011 x86_64 gnulinux '
config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.10 -Darchlib=/usr/lib/perl/5.10 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.10.1 -Dsitearch=/usr/local/lib/perl/5.10.1 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.10.1 -Dd_dosuid -des'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2 -g',
cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.4.5', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib /lib64 /usr/lib64
libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
perllibs=-ldl -lm -lpthread -lc -lcrypt
libc=/lib/libc-2.11.2.so, so=so, useshrplib=true, libperl=libperl.so.5.10.1
gnulibc_version='2.11.2'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector'
Characteristics of this binary (from libperl):
Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV
PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP USE_64_BIT_ALL
USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES
USE_PERLIO USE_REENTRANT_API
Locally applied patches:
DEBPKG:debian/arm_thread_stress_timeout - http://bugs.debian.org/501970 Raise the timeout of ext/threads/shared/t/stress.t to accommodate slower build hosts
DEBPKG:debian/cpan_config_path - Set location of CPAN::Config to /etc/perl as /usr may not be writable.
<snip-- iow patches galore --you get the picture>
DEBPKG:fixes/safe-reval-rdo-cve-2010-1447 - [PATCH] Wrap by default coderefs returned by rdo and reval
DEBPKG:patchlevel - http://bugs.debian.org/567489 List packaged patches for 5.10.1-17squeeze2 in patchlevel.h
Built under linux
Compiled at Jun 30 2011 22:28:00
#INC:
/etc/perl
/usr/local/lib/perl/5.10.1
/usr/local/share/perl/5.10.1
/usr/lib/perl5
/usr/share/perl5
/usr/lib/perl/5.10
/usr/share/perl/5.10
/usr/local/lib/site_perl
/usr/local/lib/perl/5.10.0
/usr/local/share/perl/5.10.0
.
Not to be confused with the much less informative perl -v:
% perl -v
This is perl, v5.10.1 (*) built for x86_64-linux-gnu-thread-multi
(with 53 registered patches, see perl -V for more detail)
Copyright 1987-2009, Larry Wall
Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.
Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

python -c 'import sysconfig, pprint; pprint.pprint(sysconfig.get_config_vars())'

Although this is incredibly hackish, impractical, and not as detailed as perl -V, this is a one-liner that can get decent information about the environment.
python -c "import platform as p;exec('for x in vars(p):\n try:\n print ({x:vars(p)[x]()})\n except:\n pass')"
Since this is not your typical easy-to-remember command, you could save this line to Python's Lib directory as sys_info.py and then you could just run:
python -m sys_info