How do I install a Python extension module using distutils? - python

I'm working on a Python package named "lehmer" that includes a bunch of extension modules written in C. Currently, I have a single extension module, "rng". I am using Python's Distutils to build and install the module. I can compile and install the module, but when I try to import the module using import lehmer.rng or from lehmer import rng, the Python interpreter throws an ImportError exception. I can import "lehmer" fine.
Here are the contents of my setup.py file:
from distutils.core import setup, Extension
exts = [Extension("rng", ["lehmer/rng.c"])]
setup(name="lehmer",
version="0.1",
description="A Lehmer random number generator",
author="Steve Park, Dave Geyer, and Michael Dippery",
maintainer="Michael Dippery",
maintainer_email="mpd#cs.wm.edu",
packages=["lehmer"],
ext_package="lehmer",
ext_modules=exts)
When I list the contents of Python's site-packages directory, I see the following:
th107c-4 lehmer $ ls /scratch/usr/lib64/python2.5/site-packages/lehmer
__init__.py __init__.pyc rng.so*
My PYTHONPATH environment variable is set correctly, so that's not the problem (and as noted before, I can import lehmer just fine, so I know that PYTHONPATH is not the issue). Python uses the following search paths (as reported by sys.path):
['', '/scratch/usr/lib64/python2.5/site-packages', '/usr/lib/python25.zip', '/usr/lib64/python2.5', '/usr/lib64/python2.5/plat-linux2', '/usr/lib64/python2.5/lib-tk', '/usr/lib64/python2.5/lib-dynload', '/usr/lib64/python2.5/site-packages', '/usr/lib64/python2.5/site-packages/Numeric', '/usr/lib64/python2.5/site-packages/PIL', '/usr/lib64/python2.5/site-packages/SaX', '/usr/lib64/python2.5/site-packages/gtk-2.0', '/usr/lib64/python2.5/site-packages/wx-2.8-gtk2-unicode', '/usr/local/lib64/python2.5/site-packages']
Update
It works when used on an OpenSUSE 10 box, but the C extensions still fail to load when tested on Mac OS X. Here are the results from the Python interpreter:
>>> sys.path
['', '/usr/local/lib/python2.5/site-packages', '/opt/local/lib/python25.zip', '/opt/local/lib/python2.5', '/opt/local/lib/python2.5/plat-darwin', '/opt/local/lib/python2.5/plat-mac', '/opt/local/lib/python2.5/plat-mac/lib-scriptpackages', '/opt/local/lib/python2.5/lib-tk', '/opt/local/lib/python2.5/lib-dynload', '/opt/local/lib/python2.5/site-packages']
>>> from lehmer import rng
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name rng
>>> import lehmer.rngs
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named rngs
>>> import lehmer.rng
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named rng
>>> from lehmer import rngs
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name rngs

For the record (and because I am tired of seeing this marked as unanswered), here were the problems:
Since the current directory is automatically added to the Python packages path, the interpreter was first looking in the current directory for packages; since some C modules were not compiled in the current directory, the interpreter couldn't find them. Solution: Don't launch the interpreter from the same directory in which your working copy of the code is stored.
Distutils did not install the module with the correct permissions on OS X. Solution: Fix the permissions.

Related

Python3.7 - module not found even though it is listed in sys.path

I try to import some custom modules/packages but get the error "ModuleNotFoundError: No module named 'reader'".
I made sure that the absolute path to my custom package directory is listed in sys.path:
>>> os.getcwd()
'C:\\Python Projects\\reader'
>>> sys.path.append('C:\\Python Projects\\reader')
>>>
>>> sys.path
['', 'C:\\Users\\User\\AppData\\Local\\Programs\\Python\\Python37\\python37.zip', 'C:\\Users\\User\\AppData\\Local\\Programs\\Python\\Python37\\DLLs', 'C:\\Users\\User\\AppData\\Local\\Programs\\Python\\Python37\\lib', 'C:\\Users\\User\\AppData\\Local\\Programs\\Python\\Python37', 'C:\\Users\\User\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages', 'C:\\Python Projects\\reader']
>>>
>>> import reader
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'reader'
Also, all the paths listed in sys.path (except for '' and C:\Python Projects\reader) have been added in Environment variables > System variables (I am using Win10).
Is there something else I should to in order to successfully import custom packages in my projects?
Thank you
sys.path.append('C:\\Python Projects')
Then import reader will look for reader under 'C:\Python Projects', i.e. it will test for 'C:\Python Projects\reader'. Currently it's testing for 'C:\Python Projects\reader\reader'.

How to add python library to Spyder

I am trying to add an external python library from a third party software into Spyder so I can work with it. I have already tried the following:
Adding library path containing .py files to Tools>PYTHONPATH manager
Synchronizing the path
Updating module names list through Tools>Update Module names list
However, when I try to import modules from this library I get two types of errors:
import easy
Traceback (most recent call last):
File "<ipython-input-2-685519d35f15>", line 1, in <module>
import easy
File "C:\Program Files (x86)\Plaxis\PLAXIS 2D\plxscripting\easy.py", line 24, in <module>
from .server import Server, InputProcessor
ValueError: Attempted relative import in non-package
The second type of error as follows:
from plxscripting.easy import *
Traceback (most recent call last):
File "<ipython-input-1-a40c101d3bb0>", line 1, in <module>
from plxscripting.easy import *
ImportError: No module named plxscripting.easy
I don't understand why Spyder is not recognizing these libraries. The path has been added and shows up on the manager. What constitutes a python module? Is it not just the .py file with module name prefix? Is not the path sufficient to work with the library through the IDE?

How to change path in Ninja IDE for windows?

Im a newbie to python and starting to learn Ninja IDE seemed good. but when trying to import "telnetlib" module from within the NINJA I get error:
>>> import telnetlib
Traceback (most recent call last):
File "<console>", line 1, in <module>
ImportError: No module named telnetlib
>>>
I successfully import modules like "sys" and "time".
Results of sys.path:
>>> sys.path
['C:\\Program Files (x86)\\Ninja\\Ninja.exe']
>>>
I already have C:\python27\ folders in path and also created a system environment variable called PYTHONPATH as mentioned here:
How to add to the pythonpath in windows 7?
It Works fine when using python for windows (from python.org).
Thanks

Distinguish local module with installed module

I installed django python module on my machine, and used it like this
a.py:
import django.core
...
then, I created an new file django.py in the same folder of file a.py, and re-run a.py, it throw import error, since it just imported my local django.py
File "a.py", line 1, in <module>
import django.core
ImportError: No module named core
So, how to distinguish them when importing python module?
You can use the imp module to import your script directly from a local path:
mymodule = imp.load_source('mymodule', 'django.py')
You can then use mymodule as if you had imported it normally.
Use caution, however; the python import internals must be subverted responsibly.
You could temporarily remove the current directory from PYTHONPATH:
$ python -c'import numpy.core' # works
$ touch numpy.py # add conflicting module
$ python -c'import numpy.core' # it fails now
Traceback (most recent call last):
File "<frozen importlib._bootstrap>", line 2218, in _find_and_load_unlocked
AttributeError: 'module' object has no attribute '__path__'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named 'numpy.core'; 'numpy' is not a package
$ python -c'import sys; sys.path.remove(""); import numpy.core' # works again
Needless to say, you should use such hacks sparingly if at all -- avoid manipulating sys.path manually. Rename your local module, to avoid the conflict instead (move it inside a package at least e.g.: your_package/numpy.py).

package on sys.path, with init importing names, can be used internally, but not from outside it

I have a package my_scripting_library I want to use anywhere on machine. It has an init.py:
from silo_functions import *
and looks like
my_scripting_library
-__init__.py
-silo_functions.py
-test_stuff.py
test_stuff.py looks like:
#!/usr/bin/env python
from silo_functions import *
lines = read_lines('filepath.py')
print lines
in bashrc:
export PYTHONPATH="${PYTHONPATH}:$LIBRARY"
where LIBRARY is a correct filepath to my_scripting_library
In [1]: import sys
In [2]: sys.path
Out[2]:
['',
'/usr/bin',
'/usr/lib/python2.7/site-packages/lxml-3.3.3-py2.7-linux-x86_64.egg',
'/home/cchilders/scripts/python/my_scripting_library',
...
'/home/cchilders/.ipython']
running test_stuff with from .silo_functions import * causes:
Traceback (most recent call last):
File "./test_stuff.py", line 3, in <module>
from .silo_functions import *
ValueError: Attempted relative import in non-package
running test_stuff with from my_scripting_library.silo_functions import * causes:
Traceback (most recent call last):
File "./test_stuff.py", line 3, in <module>
from my_scripting_library.silo_functions import *
ImportError: No module named my_scripting_library.silo_functions
but running test_stuff with from silo_functions import * works:
it prints the lines
Of course I can't use this package from other folders, which is the real issue- I don't want to be forced into throwing all scripts in this one place. This is causing huge problems as I am constantly reusing dozens of functions each script, and over 5 tutorials on making a folder a python package never have worked yet. Why is something on the python path with an init not a package? Thank you
May be it is because you've added '.../python/my_scripting_library' to your path. But there are no 'my_scripting_library.py' at this folder.
If you want to use 'my_scripting_library.silo_functions', try to add '/home/cchilders/scripts/python' (not '/home/cchilders/scripts/python/my_scripting_library') to path.
Because 'my_scripting_library' is module. Python will find this folder, find __init__.py in this folder and mark it as module.

Categories