How to use .so modules from Python 2 in Python 3? - python

probably my question is obvious but I was not able to find an obvious decision.
There are Python 2.6+ extensions called audit and auparse. These are dynamical libraries distributed with audit-libs-python package:
[vitaly#thermaltake tmp]$ repoquery -lq audit-libs-python
/usr/lib64/python2.7/site-packages/_audit.so
/usr/lib64/python2.7/site-packages/audit.py
/usr/lib64/python2.7/site-packages/audit.pyc
/usr/lib64/python2.7/site-packages/audit.pyo
/usr/lib64/python2.7/site-packages/auparse.so
I would like to use this extension in the up-to-date Python interpreter because of suspicions about the incorrect handling of dynamic memory in python 2.6+. For some reason I cannot load them from Python 3.3:
[vitaly#thermaltake ~]$ python3.3
Python 3.3.2 (default, Mar 5 2014, 08:21:05)
[GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append("/usr/lib64/python2.7/site-packages/")
>>> import auparse
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /usr/lib64/python2.7/site-packages/auparse.so: undefined symbol: _Py_ZeroStruct
>>> import audit
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python2.7/site-packages/audit.py", line 28, in <module>
_audit = swig_import_helper()
File "/usr/lib64/python2.7/site-packages/audit.py", line 24, in swig_import_helper
_mod = imp.load_module('_audit', fp, pathname, description)
File "/usr/lib64/python3.3/imp.py", line 183, in load_module
return load_dynamic(name, filename, file)
ImportError: /usr/lib64/python2.7/site-packages/_audit.so: undefined symbol: PyInstance_Type
I would be glad if anyone could clarify the procedure of importing the modules of this kind into the modern Python interpreter. It's hard to believe that backward compatibility between 2nd and 3rd branches was broken in this case too. Thank you.

.so modules have to be compiled for each specific Python version - you can't even reuse an .so module built for Python 2.6 with Python 2.7.
When crossing over to Python 3 it gets worse, since there are some API changes, and the SO simply won't build unchanged from the .C file (with possible exceptions).
One workaround is to serve the functions you want to use in the 2.6 module with xmlrpc, then call then from a separate Python process runing Python 3.x - that should be the simplest way.

Related

Cannot import logging.py after I renamed one of my source files logging.py in my Eclipse project

I thought, I broke Eclipse by renaming one of my source file logging.py, I quickly changed the name to something else but Eclipse cannot find the original python standard file ... I reinstalled my Anaconda install but it did not correct the problem ... and I later discovered it was likely a Python problem (see Edit 1)
When I ask Eclipse to find or open the file from the 'import logging' line, it seems it cannot find it ...
Any idea to correct this problem without reinstalling Eclipse?
Traceback (most recent call last):
File "C:\Users\ailete\workspace\landema\main.py", line 56, in <module>
import requests
File "C:\Users\ailete\Anaconda2\Lib\site-packages\requests\__init__.py", line 53, in <module>
from .packages.urllib3.contrib import pyopenssl
File "C:\Users\ailete\Anaconda2\Lib\site-packages\requests\packages\__init__.py", line 27, in <module>
from . import urllib3
File "C:\Users\ailete\Anaconda2\Lib\site-packages\requests\packages\urllib3\__init__.py", line 8, in <module>
from .connectionpool import (
File "C:\Users\ailete\Anaconda2\Lib\site-packages\requests\packages\urllib3\connectionpool.py", line 35, in <module>
from .connection import (
File "C:\Users\ailete\Anaconda2\Lib\site-packages\requests\packages\urllib3\connection.py", line 43, in <module>
from .util.ssl_ import (
File "C:\Users\ailete\Anaconda2\Lib\site-packages\requests\packages\urllib3\util\__init__.py", line 19, in <module>
from .retry import Retry
File "C:\Users\ailete\Anaconda2\Lib\site-packages\requests\packages\urllib3\util\retry.py", line 15, in <module>
log = logging.getLogger(__name__)
AttributeError: 'module' object has no attribute 'getLogger'
Edit 1
The problem happens with the executable generated by pyinstaller too ?! :
...
LOADER: Running main.py
Traceback (most recent call last):
File "<string>", line 43, in <module>
File "c:\users\ailete\appdata\local\temp\pip-build-iulqaq\pyinstaller\PyInstaller\loader\pyimod03_importers.py", line 363, in load_module
File "C:\Users\ailete\workspace\landema\entities.py", line 7, in <module>
from config import locale, ribbon_menu
File "c:\users\ailete\appdata\local\temp\pip-build-iulqaq\pyinstaller\PyInstaller\loader\pyimod03_importers.py", line 363, in load_module
File "C:\Users\ailete\workspace\landema\config.py", line 7, in <module>
import logging
ImportError: No module named logging
main returned -1
...
First, define modules path, where built-in module logging is placed:
user#host:$ python
Python 2.7.9 (default, Mar 1 2015, 12:57:24)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['/usr/lib/python2.7', ... ]
Second, ensure that there is a logging package. In my OS i have /usr/lib/python2.7/logging. I think you renamed the package accidentally; If so, rename it back.
I came across a very similar problem. The error I was receiving was: "AttributeError: ‘flask.logging’ object has no attribute ‘getLogger’". And the solution was downgrade the package Flask (settings -> project -> project interpreter).
Sorry, I solved it myself.
I thought Eclipse delete the .pyc files when I told it to clean the project, but it is not how it works apparently, So after 2 reinstalls of anaconda and configuration tweaks for this project (I have others ...), I finally remembered to check that ... the .pyc for the file was still there with the old name 'logging.pyc' ... thanks Eclipse as always, for wasting my time with your quirks!

Python3 cannot find libdnet - Scapy port

I am trying to use the Python3 fork of Scapy in my project, but I am having trouble getting it running with all of its dependencies. I am currently running OSX Yosemite. In particular, it seems Python3 cannot find libdnet.so. I have Scapy working in Python2, so libdnet exists on my system - how do I get it working in Python3? Is there a supported version of libdnet for Python3? Error is as follows:
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 23 2015, 02:52:03)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from scapy.all import *
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/scapy/all.py", line 16, in <module>
from .arch import *
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/scapy/arch/__init__.py", line 75, in <module>
from .bsd import *
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/scapy/arch/bsd.py", line 12, in <module>
from .unix import *
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/scapy/arch/unix.py", line 21, in <module>
from .pcapdnet import *
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/scapy/arch/pcapdnet.py", line 22, in <module>
from .cdnet import *
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/scapy/arch/cdnet.py", line 17, in <module>
raise OSError("Cannot find libdnet.so")
OSError: Cannot find libdnet.so
Thanks
I ran in to this same issue. I installed libdnet and libpcap with homebrew (e.g., in /usr/local/lib), but ctypes could never find them using find_library(). This is not a solution, but a hack to get scapy to import. I modified the following two files in my scapy installation to specify the full path to the libraries:
I changed find_library('dnet') in scapy/arch/cdnet.py to find_library('/usr/local/lib/libdnet')
I changed find_library('pcap') in scapy/arch/winpcapy.py to find_library('/usr/local/lib/libpcap')
Another less invasive idea is to just link these two libraries from /usr/local/lib to /usr/lib...
This is the code fragment that generates the exception.
from ctypes import *
from ctypes.util import find_library
_lib_name = find_library('dnet')
if not _lib_name:
raise OSError("Cannot find libdnet.so")
_lib=CDLL(_lib_name)
Apparently, python ctypes cannot find dnet library on your computer. Once you can get ctypes find dnet, it should work with scapy.
Also, usage of dnet is not mandatory. Try scapy with dnet disabled. You do not need it for parsing packets. And depending on the system, for some limited sending scapy can use pcap, too.
Please, file an issue on https://github.com/phaethon/scapy

Python3 on Ubuntu giving errors on help() command

I used help() in the python3 shell on Ubuntu 14.04
I got this output
Please help , don't know whats wrong.
Python 3.4.0 (default, Apr 11 2014, 13:05:11)
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> help()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.4/_sitebuiltins.py", line 98, in __call__
import pydoc
File "/usr/lib/python3.4/pydoc.py", line 65, in <module>
import platform
File "/home/omega/platform.py", line 2, in <module>
print("System : ",platform.uname().system)
AttributeError: 'module' object has no attribute 'uname'
>>>
The problem is that platform is the name of a stdlib module, which help uses. By creating a module of your own with the same name that occurs before the stdlib in your sys.path, you're preventing Python from using the standard one.
The fact that your own platform module tries to use the stdlib module of the same name just compounds the problem. That isn't going to work; your import platform inside that module is just importing itself.
The solution is to not collide names like this. Look at the list of the standard modules, and don't create anything with the same name as any of them if you want to use features from that module, directly or indirectly.
In other words: Rename your platform.py to something else, or put it inside a package.
File "/home/omega/platform.py", line 2, in <module>
print("System : ",platform.uname().system)
This is the problem, go to platform.py and fix it, it will be ok. It says, platform has not any method called uname you probably misstyped.

inconsistent behaviour of interactive session and script in Python

When I run the script, I had this ImportError:
$ python ~/Dropbox/code/py/ZoteroFindOrphanedFiles.py
Traceback (most recent call last):
File "/home/zane/Dropbox/code/py/ZoteroFindOrphanedFiles.py", line 1, in <module>
import sqlite3
File "/usr/lib/python3.2/sqlite3/__init__.py", line 23, in <module>
from sqlite3.dbapi2 import *
File "/usr/lib/python3.2/sqlite3/dbapi2.py", line 23, in <module>
import datetime
File "/usr/lib/python3.2/datetime.py", line 20, in <module>
import math as _math
File "/home/zane/Dropbox/code/py/math.py", line 3, in <module>
from nzmath.rational import Integer, Rational
ImportError: No module named nzmath.rational
But I don't have it when running the interactive session:
$ python
Python 3.2.3 (default, Apr 23 2012, 23:14:44)
[GCC 4.7.0 20120414 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>>
Why is that?
Here is the problem:
import math as _math
File "/home/zane/Dropbox/code/py/math.py", line 3, in <module>
You have your own module called math.py, but this is the same as a standard Python module of the same name. This is not recommended.
The solution is to rename your math.py to something else, and don't forget to delete the math.pyc in the same directory (otherwise you'll still have the same problem).
You have a local file
/home/zane/Dropbox/code/py/math.py
which is getting imported instead of the standard lib math module.
The solution is to rename your /home/zane/Dropbox/code/py/math.py to something else.
The problem occurs when you call a script in the /home/zane/Dropbox/code/py directory since this adds this directory to the beginning of sys.path and thus this directory gets searched first when Python tries to import modules.
You have a file called math.py in the script's directory, and this shadows the stdlib math module. Rename the file.

"from _json import..." - python

I am inspecting the JSON module of python 3.1, and am currently in /Lib/json/scanner.py. At the top of the file is the following line:
from _json import make_scanner as c_make_scanner
There are five .py files in the module's directory: __init__ (two leading and trailing underscores, it's formatting as bold), decoder, encoder, scanner and tool. There is no file called "json".
My question is: when doing the import, where exactly is "make_scanner" coming from?
Yes, I am very new to Python!
It's coming from a C-compiled _json.pyd (or _json.so, etc, etc, depending on the platform) that lives elsewhere on the sys.path. You can always find out where that is in your specific Python installation by importing the module yourself and looking at its __file__, e.g.:
>>> import _json
>>> _json.__file__
'/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload/_json.so'
As you see, in my installation of Python 2.6, _json comes from the lib-dynload subdirectory of lib/python2.6, and the extension used on this platform is .so.
It may be coming from a file, or it may be built-in. On Windows, it appears to be built-in.
Python 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import _json
>>> _json.__file__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute '__file__'
and there is no _json.pyd or _json.dll in the offing.
If you want to see the source, having a binary file on your machine or not is irrelevant -- you'll need the SVN browser.

Categories