I have a strange import problem with python. The same code works on OSX, but it doesn't work on precise 12.04.
test.py
lib/
twitter/
twitterstream.py
__init__.py
The twitterstream.py has a class TwitterStream.
test.py:
import sys
sys.path.append('./lib')
import twitter.twitterstream
tw = twitter.twitterstream.TwitterStream()
tweets = tw.fetchsamples()
print tweets
I am getting following error:
> python test.py
Traceback (most recent call last):
File "test.py", line 3, in
import twitter.twitterstream
ImportError: No module named twitterstream
You have another module twitter installed on your Ubuntu machine.
You put the ./lib path at the end of the module search path; the other twitter module is installed in a location listed on the sys.path search path before that.
Insert ./lib at the start instead:
sys.path.insert(0, './lib')
Related
I am trying to write a test for my GAE programme which uses the datastore.
Following Google's Documentation, I see that I should be adding the path to my SDK into my PYTHONPATH.
I did this using:
import sys
sys.path.remove('/usr/local/lib/python2.7/dist-packages') # Has a 'google' module, which I want to be sure isn't interfering.
sys.path.insert(1,'/home/olly/google-cloud-sdk/platform/google_appengine')
sys.path.insert(1, '/home/olly/google-cloud-sdk/platform/google_appengine/lib/yaml/lib')
Then when the file is run:
Traceback (most recent call last):
File "myapp_tests.py", line 20, in <module>
from google.appengine.ext import ndb
ImportError: No module named appengine.ext
I have installed the SDK in the location above, and looking in /home/olly/google-cloud-sdk/platform/google_appengine/ I found the google folder, which has an __init__.py in it, along with appengine. Basically, the folder structure looks good to me, with them all being named correctly and having __init__.py files.
In an interactive console, after running the commands above, I found that I could run:
import google
no problem, but when I tried
import google.appengine
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named appengine
It was my understanding that having the __init__.py() files in the directories meant that they could be imported as above.
I also did a sudo find / --name "google", and the only thing that showed up that is also in my PYTHONPATH was the /usr/local/lib/python2.7/dist-packages, which I explicitly removed, and also inserted the rest of my paths in front of anyway.
I tried using GAE's own method of:
import dev_appserver
dev_appserver.fix_sys_path()
which added a whole lot of paths to sys.path, but still didn't help me make it work.
I also found that when I add '/home/olly/Servers/google_appengine/google' to my paths, I can run:
import appengine.ext
but running:
from appengine.ext import ndb
causes:
Traceback (most recent call last):
File "booking_function_tests.py", line 16, in <module>
from appengine.ext import ndb
File "/home/olly/Servers/google_appengine/google/appengine/ext/ndb/__init__.py", line 7, in <module>
from tasklets import *
File "/home/olly/Servers/google_appengine/google/appengine/ext/ndb/tasklets.py", line 69, in <module>
from .google_imports import apiproxy_stub_map
File "/home/olly/Servers/google_appengine/google/appengine/ext/ndb/google_imports.py" , line 11, in <module>
from google3.storage.onestore.v3 import entity_pb
ImportError: No module named google3.storage.onestore.v3
Am I missing something really obvious? How should I go about importing ndb?
EDIT:
I'm running the latest SDK (1.9.34), but I have the following code in my google_imports.py:
try:
from google.appengine.datastore import entity_pb
normal_environment = True
except ImportError:
try:
from google3.storage.onestore.v3 import entity_pb
normal_environment = False
except ImportError:
# If we are running locally but outside the context of App Engine.
try:
set_appengine_imports()
from google.appengine.datastore import entity_pb
normal_environment = True
except ImportError:
raise ImportError('Unable to find the App Engine SDK. '
'Did you remember to set the "GAE" environment '
'variable to be the path to the App Engine SDK?')
Also, google.__path__ gives me the '/usr/local/lib/python2.7/dist-packages' path which I thought I removed earlier. Here is an excerpt of how I'm removing it:
import sys
sys.path.insert(1, '/home/olly/Servers/google_appengine')
sys.path.insert(1, '/home/olly/Servers/google_appengine/lib/yaml/lib')
sys.path.remove('/usr/local/lib/python2.7/dist-packages')
import google
print google.__path__
print sys.path
['/usr/local/lib/python2.7/dist-packages/google']
['/home/olly/Servers/google_appengine/myapp', '/home/olly/Servers/google_appengine/lib/yaml/lib', '/home/olly/Servers/google_appengine/google', '/home/olly/Servers/google_appengine', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7', '/usr/lib/python2.7/dist-packages/ubuntu-sso-client', '/usr/lib/python2.7/dist-packages/wx-2.8-gtk2-unicode']
So my sys.path is updated, but import google seems to still be importing from the no-longer-there path, which would be the crux of my problem I guess. Do I need to reload the path or something?
I run into these problems a lot less by always running inside a virtualenv.
I agree with snakecharmerrb you should get print google.__file__ or google.__path_ to figure out exactly what you're importing.
This snippet might also solve your problem:
import google
gae_dir = google.__path__.append('/path/to/appengine_sdk//google_appengine/google')
sys.path.insert(0, gae_dir) # might not be necessary
import google.appengine # now it's on your import path`
Which version of app engine SDK are you using? I am using the latest SDK (1.9.34). I find the below snippet in my ~/google_appengine/google/appengine/ext/ndb/google_imports.py file
try:
from google3.storage.onestore.v3 import entity_pb
normal_environment = False
except ImportError:
# If we are running locally but outside the context of App Engine.
try:
set_appengine_imports()
from google.appengine.datastore import entity_pb
normal_environment = True
except ImportError:
raise ImportError('Unable to find the App Engine SDK. '
'Did you remember to set the "GAE" environment '
'variable to be the path to the App Engine SDK?')
But in your stack trace, after google3.storage import it doesn't seem to go inside the except clause.
So try the same code with latest SDK.
I have a package my_scripting_library I want to use anywhere on machine. It has an init.py:
from silo_functions import *
and looks like
my_scripting_library
-__init__.py
-silo_functions.py
-test_stuff.py
test_stuff.py looks like:
#!/usr/bin/env python
from silo_functions import *
lines = read_lines('filepath.py')
print lines
in bashrc:
export PYTHONPATH="${PYTHONPATH}:$LIBRARY"
where LIBRARY is a correct filepath to my_scripting_library
In [1]: import sys
In [2]: sys.path
Out[2]:
['',
'/usr/bin',
'/usr/lib/python2.7/site-packages/lxml-3.3.3-py2.7-linux-x86_64.egg',
'/home/cchilders/scripts/python/my_scripting_library',
...
'/home/cchilders/.ipython']
running test_stuff with from .silo_functions import * causes:
Traceback (most recent call last):
File "./test_stuff.py", line 3, in <module>
from .silo_functions import *
ValueError: Attempted relative import in non-package
running test_stuff with from my_scripting_library.silo_functions import * causes:
Traceback (most recent call last):
File "./test_stuff.py", line 3, in <module>
from my_scripting_library.silo_functions import *
ImportError: No module named my_scripting_library.silo_functions
but running test_stuff with from silo_functions import * works:
it prints the lines
Of course I can't use this package from other folders, which is the real issue- I don't want to be forced into throwing all scripts in this one place. This is causing huge problems as I am constantly reusing dozens of functions each script, and over 5 tutorials on making a folder a python package never have worked yet. Why is something on the python path with an init not a package? Thank you
May be it is because you've added '.../python/my_scripting_library' to your path. But there are no 'my_scripting_library.py' at this folder.
If you want to use 'my_scripting_library.silo_functions', try to add '/home/cchilders/scripts/python' (not '/home/cchilders/scripts/python/my_scripting_library') to path.
Because 'my_scripting_library' is module. Python will find this folder, find __init__.py in this folder and mark it as module.
In test.py, I am trying to import test_data:
import unittest2
import re
from test_data import receipt1_example
test_data.py is in the same directory as test.py. I get the following error:
/Users/ahammond/.virtualenvs/ric2.6/bin/python2.6
/Applications/PyCharm.app/helpers/pycharm/utrunner.py
/Users/ahammond/src/hackfest_spring_2012/parse_me/test.py::test true
Testing started at 11:30 AM ... Traceback (most recent call last):
File "/Applications/PyCharm.app/helpers/pycharm/utrunner.py", line
121, in
module = loadSource(a[0]) File "/Applications/PyCharm.app/helpers/pycharm/utrunner.py", line 44, in
loadSource
module = imp.load_source(moduleName, fileName) File "/Users/ahammond/src/hackfest_spring_2012/parse_me/test.py", line 4,
in
from test_data import receipt1_example ImportError: No module named test_data
Process finished with exit code 1
As you can see, I am running this under pycharm using a virtualenv. Here's a screenshot of the configuration:
The work around i use is:
import sys
import os
try:
import test_data
except ImportError:
sys.path.append(os.path.dirname(__file__))
try:
import test_data
finally:
sys.path.remove(os.path.dirname(__file__))
A friend told me that one can also add the directory entries to some include directories.
Please try PyCharm 2.5.1 RC, there was a bug with sys.path building (it contained incorrect, duplicated project source directory).
If it's not the case, you can mark additional directories as Source in Preferences | Project Structure or add them to the Paths in the Python Interpreters.
I decided to develop my home project in python 3.x rather than 2.x. So I decided to check, if it works under 3.1. I run python3.1 above my package directory and then:
>>> import fathom
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "fathom/__init__.py", line 3, in <module>
from schema import Database
ImportError: No module named schema
when I enter fathom directory however schema can be imported:
>>> import schema
Also when I run python2.6 above my package directory I can do this:
>>> import fathom
My __init__.py has following import:
from schema import Database
from inspectors import PostgresInspector, SqliteInspector, MySqlInspector
Should I add something for python3.1?
Did you try a relative import?
from . import schema
from .inspectors import PostgresInspector
Works in Python 2.6 as well.
The 2to3 script can help you pinpoint more of these problems.
I'm working on a Python package named "lehmer" that includes a bunch of extension modules written in C. Currently, I have a single extension module, "rng". I am using Python's Distutils to build and install the module. I can compile and install the module, but when I try to import the module using import lehmer.rng or from lehmer import rng, the Python interpreter throws an ImportError exception. I can import "lehmer" fine.
Here are the contents of my setup.py file:
from distutils.core import setup, Extension
exts = [Extension("rng", ["lehmer/rng.c"])]
setup(name="lehmer",
version="0.1",
description="A Lehmer random number generator",
author="Steve Park, Dave Geyer, and Michael Dippery",
maintainer="Michael Dippery",
maintainer_email="mpd#cs.wm.edu",
packages=["lehmer"],
ext_package="lehmer",
ext_modules=exts)
When I list the contents of Python's site-packages directory, I see the following:
th107c-4 lehmer $ ls /scratch/usr/lib64/python2.5/site-packages/lehmer
__init__.py __init__.pyc rng.so*
My PYTHONPATH environment variable is set correctly, so that's not the problem (and as noted before, I can import lehmer just fine, so I know that PYTHONPATH is not the issue). Python uses the following search paths (as reported by sys.path):
['', '/scratch/usr/lib64/python2.5/site-packages', '/usr/lib/python25.zip', '/usr/lib64/python2.5', '/usr/lib64/python2.5/plat-linux2', '/usr/lib64/python2.5/lib-tk', '/usr/lib64/python2.5/lib-dynload', '/usr/lib64/python2.5/site-packages', '/usr/lib64/python2.5/site-packages/Numeric', '/usr/lib64/python2.5/site-packages/PIL', '/usr/lib64/python2.5/site-packages/SaX', '/usr/lib64/python2.5/site-packages/gtk-2.0', '/usr/lib64/python2.5/site-packages/wx-2.8-gtk2-unicode', '/usr/local/lib64/python2.5/site-packages']
Update
It works when used on an OpenSUSE 10 box, but the C extensions still fail to load when tested on Mac OS X. Here are the results from the Python interpreter:
>>> sys.path
['', '/usr/local/lib/python2.5/site-packages', '/opt/local/lib/python25.zip', '/opt/local/lib/python2.5', '/opt/local/lib/python2.5/plat-darwin', '/opt/local/lib/python2.5/plat-mac', '/opt/local/lib/python2.5/plat-mac/lib-scriptpackages', '/opt/local/lib/python2.5/lib-tk', '/opt/local/lib/python2.5/lib-dynload', '/opt/local/lib/python2.5/site-packages']
>>> from lehmer import rng
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name rng
>>> import lehmer.rngs
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named rngs
>>> import lehmer.rng
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named rng
>>> from lehmer import rngs
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name rngs
For the record (and because I am tired of seeing this marked as unanswered), here were the problems:
Since the current directory is automatically added to the Python packages path, the interpreter was first looking in the current directory for packages; since some C modules were not compiled in the current directory, the interpreter couldn't find them. Solution: Don't launch the interpreter from the same directory in which your working copy of the code is stored.
Distutils did not install the module with the correct permissions on OS X. Solution: Fix the permissions.