How do I build numpy for unicode ucs4 - python

I am cross compiling for an embedded device using yocto - so using pip install is not appropriate.
My build works, but keeps defaulting to ucs2 character type, which causes an error:
numpy.core.multiarray failed to import.

Caveat, I haven't really tried this...
As far as I can see, building numpy with ucs4 support means that you have to compile python with ucs4-support. Thus, you would need to add
EXTRA_OECONF += "--enable-unicode=ucs4"
in a python_xxx.bbappend, depending on which python (2 or 3) and which OE-release you're using.
If you're getting any other issues after this, is unknown...

Related

Building VRPN server with Python 3.4 64-bit on Windows

I'm trying to build a VRPN server with Python3 flag using Python 3.4 64-bit on Windows 7 64-bit but there seems to be a problem. I need this for BlenderVR software.
This is my procedure:
1) I use CMake to create makefiles (I'm using 3.4.0 version but I've also tried different ones). I do it with this command (those flags should be there but the result seems to be the same without them anyway):
cmake -G"MinGW Makefiles" -HD:\My\BlenderVR\plugins\vrpn
-BD:\My\BlenderVR\plugins\cmake -DVRPN_BUILD_PYTHON=OFF -DVRPN_BUILD_PYTHON_HANDCODED_2X=OFF -DVRPN_BUILD_PYTHON_HANDCODED_3X=ON
I used to add those flags as well but it seems that it can find Python without them
-DPYTHON_INCLUDE_DIR=D:\My\BlenderVR\Required\Python3\include
-DPYTHON_LIBRARY=D:\My\BlenderVR\Required\Python3\libs\python34.lib
Python is correctly found and this operation doesn't throw any error.
2) Then I use mingw32-make.exe to build it and I get this error:
[ 90%] Linking CXX shared module vrpn.pyd D:/My/BlenderVR/Required/Python3/libs/python34.lib: error adding
symbols: File f ormat not recognized collect2.exe: error: ld
returned 1 exit status
python\CMakeFiles\vrpn-python.dir\build.make:505: recipe for
target 'python/vrpn .pyd' failed mingw32-make[2]: * * *
[python/vrpn.pyd] Error 1 CMakeFiles\Makefile2:3247: recipe for
target 'python/CMakeFiles/vrpn-python.dir/ all' failed
mingw32-make[1]: * * * [python/CMakeFiles/vrpn-python.dir/all]
Error 2 Makefile:159: recipe for target 'all' failed
mingw32-make: [all] Error 2
vprn.pyd is the crucial thing for my future work.
I figured out that it needs libpython34.a file (probably). When I created it and copied to Python3/libs folder it worked and finished without errors but the crated vprn.pyd didn't worked as it should.
What I need is to get import vrpn to work with this simple test in python (appending path where vrpn.pyd was build):
import sys
sys.path.append('D:/My/BlenderVR/plugins/cmake/python')
import vrpn
It lags my whole computer for a while and then pops out that Python has stop working.
I suspect that problem is in the libpython34.a file that I created doing this:
gendef python34.dll (in Windows/System32)
dlltool -D python34.dll -d python34.def -l libpython34.a
I don't how else should I get the libpython file. I've tried various versions of CMake and MinGW (like MinGWPy, TDM, w64) with many CMake flags. I was able to make it work using 32-bit Python but I need 64-bit version otherwise it is not working with BlenderVR enviroment.
I know this is very specific problem and probably kind of confusing at first but I didn't know how else to put it. I'll be glad for anything that could help. Thank you.
mingwpy should be installed with pip (until it is officially released at PYPI):
pip install -i https://pypi.anaconda.org/carlkl/simple mingwpy
all necessary import files are atomatically copied into the python\libs folder.
If python\Scripts is in the PATH it should work out of the box.
You have to make sure, that Blender Python is equiped with two import files
D:\My\BlenderVR\Required\Python3\libs\libpython\libpython34.dll.a
D:\My\BlenderVR\Required\Python3\libs\libpython\libmsvcr100.a

import error: ephem/_libastro.so undefined symbol: PyUnicodeUCS2_AsUTF8String

I just successfully installed PyEphem using pip in a pyenv. However, on import I receive:
ImportError: /python2.7/site-packages/ephem/_libastro.so: undefined symbol: PyUnicodeUCS2_AsUTF8String
In looking around I've seen it mentioned that some modules are built "against Python" in regards to Unicode and suggest recompiling. I'm quite new to Python and Ubuntu 14.04, and although I believe this is the answer to my issue, I do not know what recompiling means or how to do it.
The symbol PyUnicode_AsUTF8String(value) is used once in _libastro.c and is defined on my system in the file:
/usr/include/python2.7/unicodeobject.h
There it can be aliased one of two ways:
#ifndef Py_UNICODE_WIDE
# ...
# define PyUnicode_AsUTF8String PyUnicodeUCS2_AsUTF8String
# ...
#else
# ...
# define PyUnicode_AsUTF8String PyUnicodeUCS4_AsUTF8String
Your error message makes it sound as though your system Python is compiled to use 4-byte-wide Unicode strings (hence why the linker cannot find a UCS2 version of this function inside of it), but that the version of PyEphem that auto-compiled on your system when you ran pip install somehow got confused and unset Py_UNICODE_WIDE and thus generated C code that was expected a UCS2 symbol.
Do you have several compiled versions of Python on your system, where the Unicode setting of one version could accidentally be affecting how this compile for your system Python takes place?

Failed to import Numeric: No module named Numeric in F2PY

When I am trying to use F2PY, I'll get the error:
Failed to import Numeric: No module named Numeric
I know that numeric is dead and instead we should use numpy. But files:
/usr/local/lib/python2.7/dist-packages/f2py2e/src/fortranobject.h and
/usr/local/lib/python2.7/dist-packages/f2py2e/f2py2e.py both use the Numeric package. I tried to replace it with numpy, but I was not successful.
I used to use f2py without any problem, but after I formatted my computer and got a fresh copy of Ubuntu, I have this problem.
I also tried to use the option --2d-numpy for f2py like:
f2py -c --fcompiler=intel --2d-numpy -m processoutput processoutput.f
But it didn't work, and it is still looking for numpy.
Thank you for your help.
I ran into a similar situation using msys under Windows, and indeed I was trying to use an outdated version of f2py. The newer version is included with numpy (and doesn't need to be installed separately). And can be found in the site-packages/numpy/f2py directory. Although my setup is a bit different, I was able to compile from python using this script:
import numpy.f2py.f2py2e as f2py2e
import sys
sys.argv += "-c -m hello hello.f".split()
f2py2e.main()
You can download old versions of Numeric here: http://sourceforge.net/projects/numpy/files/Old%20Numeric/24.2/
If you install that, I think f2py will be satisfied.

undefined _PyUnicodeUCS4_IsWhitespace in compiled cython+numpy

i work on ubuntu 10.04 and used cython to compile my python code.
i then tried to copy 2 of my binaries (one with numpy, and one without) to another distribution with supported kernel and etc... the only thing i did which is not so cool is that i used the python that comes with that distribution (2.6), and copy from my ubuntu the numpy libraries.
when i exec the one without numpy, it works. when i exec the one with the 'from numpy import ...' i get an error like: undefined symbol: _PyUnicodeUCS4_IsWhitespace.
i thought that the numpy just compiled for UCS4 where as the python version in the new dist is UCS2. but for my surprise, when i exec the same python code with the numpy import - as python and not compiled - it works.
so basically i can say that if i open 'python' and import numpy libraries it works and i can use them. but if i'm using the compiled version - i get that UCS4 error..
any ideas?
(the new dist is not so much in my control and i can't really just compile anything i want on that dist)
Thanks.
well, it goes like this:
when running python interpreter and imports the numpy library it tries to load from libpython.so the symbol Python is compiled with (i guess so). this is why it works with the interpreter. so the request for that unicode function doesn't come from numpy - but from Python - so it uses the UCS2 functions it compiled with (probably).
but when running the compiled version, and again, it tries to load that function - it can't find it because it searches for a UCS4 version..
i did a small check: grep "_PyUnicode" in libpython, in the first dist, and in the second - and there was the different: one printed UCS4 functions, and the other printed UCS2 functions..
so the "easy" solution here i guess is to compile on my first dist a UCS2 version Python, then setting Cython to compile with UCS2.. i believe that will do the job.

Install libxml2 bindings for python3

The Problem
I’m trying to build libxml (libxml2-2.7.8) bindings for Python 3.2. When I run the following:
./configure --with-python=/usr/bin/python3.2
The compilation of libxml2-2.7.8 works...but the Python bindings for libxml2-2.7.8 don’t!
Attempted Fixes
Ported setup.py and generator.py (using 2to3, and some basic patches on generator.py)
Patched libxml.c and types.c as follows:
replace PyInt* by PyLong*
replace PyString* by PyBytes*
Current Situation
Unfortunately, it still wasn’t enough. I ran python3.2 setup.py build and got the following error:
types.c:594:17: error: ‘PyInstanceObject’ undeclared (first use in this function)
I can’t find the Python3 equivalent of PyInstanceObject!
Has anyone managed to compile libxml2 bindings for Python 3?
Did I miss something??? Can anybody help? :(
The PyInstanceObject is a part of the support for old style types, which is gone in Python 3, and has been deprecated since Python 2.2. The trick here is to first update the bindings to use new-style classes, and then port to Python 3.
(Or use lxml, which wraps the libxml2 in a Pythonic XML classes).
I'm not 100% sure what the replacement is, I've never done old-style classes in C, but I think it's simply a PyObject.

Categories