When I try to use boto library on App Engine, I get the next error:
Traceback (most recent call last):
File "C:\Program Files\Google\google_appengine\google\appengine\ext\webapp\_webapp25.py", line 701, in __call__
handler.get(*groups)
File "E:\Probes\pruebas\pruebasAWS\main.py", line 26, in get
conn = S3Connection('<KEY1>','<KEY2>')
File "E:\Probes\pruebas\pruebasAWS\boto\s3\connection.py", line 148, in __init__
path=path, provider=provider)
File "E:\Probes\pruebas\pruebasAWS\boto\connection.py", line 231, in __init__
self.http_unretryable_exceptions.append(ssl.SSLError)
AttributeError: 'module' object has no attribute 'SSLError'
I´ve installed OpenSSL and Python 2.7. OpenSSL and SSL library for python are running, and when i deploy the App to the Google Infrastructure, it works fine. The problem comes when I try to execute the app on my local machine.
The code is:
from google.appengine.ext import webapp
from google.appengine.ext.webapp import util
from boto.s3.connection import S3Connection
import hashlib
class MainHandler(webapp.RequestHandler):
def get(self):
conn = S3Connection('<KEY1>','<KEY2>')
bucket = conn.create_bucket(hashlib.md5('noTRePeaTedBuCket').hexdigest()+"probe")
if bucket:
self.response.out.write('Bucket creado')
else:
self.response.out.write('Bucket NO creado')
The actual issue here is that AppEngine mucks around with things to make it impossible to import certain standard, builtin python modules such as ssl.
There was some conversation about this on the boto IRC and one of the users came up with this patch:
https://github.com/samba/boto/commit/6f1ab73d92ff6fb2589362bbdadf6bbe66811e7e
Some form of this will probably be merged into boto master soon.
Related
i tried to run my Python Program on my Linux maschine but when i try to run it it is having a hard time finding all the nessasary modules needed for it to work.
i installed CanOpen to my project
import threading
import sys
sys.path.append('venv/Lib/site-packages')
sys.path.append('venv/Lib/site-packages/can/interfaces/kvaser')
sys.path.append('/test/Python39/Lib')
import canopen
from canopen.network import MessageListener, NodeScanner
from canopen.nmt import NmtMaster
from canopen.sync import SyncProducer
from canopen.timestamp import TimeProducer
if __name__ == '__main__':
network = canopen.Network()
network.connect(bustype='kvaser', channel=0, bitrate=250000)
it is showing me following error in the Terminal:
Kvaser canlib is unavailable.
Traceback (most recent call last):
File "/test/Can_T1/main.py", line 60, in <module>
network.connect(bustype='kvaser', channel=0, bitrate=250000)
File "/test/Can_T1/venv/Lib/site-packages/canopen/network.py", line 112, in connect
self.bus = can.interface.Bus(*args, **kwargs)
File "/test/Can_T1/venv/Lib/site-packages/can/interface.py", line 120, in __new__
bus = cls(channel, *args, **kwargs)
File "/test/Can_T1/venv/Lib/site-packages/can/interfaces/kvaser/canlib.py", line 437, in __init__
canGetNumberOfChannels(ctypes.byref(num_channels))
NameError: name 'canGetNumberOfChannels' is not defined
Is there any way to define 'canGetNumberOfChannels'?
You have to install the canlib on your machine as well. Right now, you just have installed the SDK. If you are running on linux you have to install libcanlib.so on your machine. Take a look at the part that is related to your operating system on this link: https://pypi.org/project/canlib/
I am currently trying to zip a large file (> 1GB) using python on google app engine and I have used the following solution due to the limitations google app engine places on the memory cache for a process. Create a zip file from a generator in Python?
When I run the code on the app engine, I get the following error
Traceback (most recent call last):
File "/base/data/home/apps/s~whohasfiles/frontend.379535120592235032/gluon/restricted.py", line 212, in restricted
exec ccode in environment
File "/base/data/home/apps/s~whohasfiles/frontend.379535120592235032/applications/onefile/controllers/page.py", line 742, in <module>
File "/base/data/home/apps/s~whohasfiles/frontend.379535120592235032/gluon/globals.py", line 194, in <lambda>
self._caller = lambda f: f()
File "/base/data/home/apps/s~whohasfiles/frontend.379535120592235032/applications/onefile/controllers/page.py", line 673, in download
zip_response = page_store.gcs_zip_page(page, visitor)
File "applications/onefile/modules/page_store.py", line 339, in gcs_zip_page
w = z.start_entry(ZipInfo('%s-%s' %(file.created_on, file.name) ))
File "applications/onefile/modules/page_store.py", line 481, in start_entry
r, w = os.pipe()
OSError: [Errno 38] Function not implemented
Does the google app engine not support the OS.pipe() function?
How can I get a work around please?
The 'os' module is available but with unsupported features disabled such as pipe() as it operates on file objects [1]. You would need to use a Google Cloud Storage bucket as a temporary object as there is no concept of file objects you can use for storage local to the App Engine runtime. The GCS Client Library will give you file-like access to a bucket which you can use for this purpose [2]. Every app has access to a default storage bucket which you may need to first activate [3].
[1] https://cloud.google.com/appengine/docs/python/#Python_Pure_Python
[2] https://cloud.google.com/appengine/docs/python/googlecloudstorageclient/
[3] https://cloud.google.com/appengine/docs/python/googlecloudstorageclient/activate
I'm getting the error:
Exception: imported schema (http://www.w3.org/2001/XMLSchema) at
(http://www.w3.org/2001/XMLSchema.x sd), failed
when passing a Doctor (constructed with ImportDoctor) to the suds Client constructor.
I'm working on two Windows machines, both of them got the same version of suds installed, but only one of them rises the error above.
Could someone guide me here to know why this error rises?, so I can figure out what's missing on the machine where it happens?.
Thanks in advance!!!.
UPDATE: I don't really know if this is important, but it's worth noting that my Windows machine that is rising the error is an Amazon Web Services instance. At my local machine everything's working well!.
UPDATE: Here's some code I ran at the python interpreter of the machine I mentioned. Here you can detail how the error is rising...
>>> from suds.client import Client
>>> from suds.xsd.doctor import ImportDoctor, Import
>>> missing_import = Import("http://www.w3.org/2001/XMLSchema")
>>> missing_import.filter.add("http://tempuri.org/")
>>> doctor = ImportDoctor(missing_import)
>>> client = Client("http://etcfulfill.ebooks.com/Fulfillment.asmx?wsdl")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "suds\client.py", line 112, in __init__
self.wsdl = reader.open(url)
File "suds\reader.py", line 152, in open
d = self.fn(url, self.options)
File "suds\wsdl.py", line 159, in __init__
self.build_schema()
File "suds\wsdl.py", line 220, in build_schema
self.schema = container.load(self.options)
File "suds\xsd\schema.py", line 95, in load
child.dereference()
File "suds\xsd\schema.py", line 323, in dereference
midx, deps = x.dependencies()
File "suds\xsd\sxbasic.py", line 422, in dependencies
raise TypeNotFound(self.ref)
suds.TypeNotFound: Type not found: '(schema, http://www.w3.org/2001/XMLSchema, )'
>>> client = Client("http://etcfulfill.ebooks.com/Fulfillment.asmx?wsdl", doctor=doctor)
No handlers could be found for logger "suds.xsd.sxbasic"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "suds\client.py", line 112, in __init__
self.wsdl = reader.open(url)
File "suds\reader.py", line 152, in open
d = self.fn(url, self.options)
File "suds\wsdl.py", line 159, in __init__
self.build_schema()
File "suds\wsdl.py", line 220, in build_schema
self.schema = container.load(self.options)
File "suds\xsd\schema.py", line 93, in load
child.open_imports(options)
File "suds\xsd\schema.py", line 305, in open_imports
imported = imp.open(options)
File "suds\xsd\sxbasic.py", line 542, in open
result = self.download(options)
File "suds\xsd\sxbasic.py", line 567, in download
raise Exception(msg)
Exception: imported schema (http://www.w3.org/2001/XMLSchema) at (http://www.w3.org/2001/XMLSchema.xsd), failed
UPDATE:
I realized that suds connections always open in TCP increasing ports, and if it reaches the maximum TCP port (65535) then it starts opening again from the minimum TCP port available, so there's no problem with this.
The problem shows up when using suds ImportDoctor, because it has to open a previous connection to the location where the import should be retrieved, and for some reason, if the system reaches the maximum TCP port count, then suds somehow assumes that there's no TCP port available to open the connection for obtaining the import, and in consecuence it throws the exception:
Exception: imported schema (http://www.w3.org/2001/XMLSchema) at (http://www.w3.org/2001/XMLSchema.xsd), failed
I repeat, this only happens if suds has to open this previous connection for obtaining the import. If ImportDoctor is not used, then suds has no problem if the TCP port count reaches its maximum, it just restarts at the minimum port available.
Does anyone has any clue on how to resolve this issue???. I'd really appreciate the help!!!.
I've figured out what the problem was. The schema that was missing from the WSDL I was trying to use with suds was:
http://www.w3.org/2001/XMLSchema
And the XSD file for this schema is at:
http://www.w3.org/2001/XMLSchema.xsd
So when I used suds ImportDoctor to add this schema import, sometimes the w3.org domain was denying my access (don't know why really) and that's why this error was rising:
Exception: imported schema (http://www.w3.org/2001/XMLSchema) at (http://www.w3.org/2001/XMLSchema.xsd), failed
What did I do to solve this problem?. I just downloaded this schema to my machine and used suds ImportDoctor to retrieve this import locally.
And that was it!!!. Confusing bug!!!. But SOLVED.
This is in my application file head:
import os
import sys
from cgi import parse_qs, escape
import pymongo
from pymongo import MongoClient
I have the mongoDB 2.4 gear installed, and am trying to connect via
client = MongoClient('mongodb:$OPENSHIFT_MONGODB_DB_HOST:$OPENSHIFT_MONGODB_DB_PORT/')
I get the errors:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/var/lib/openshift/531b77fd500446980900010d/python/virtenv/lib/python2.7/site-packages/pymongo/mongo_client.py", line 239, in __init__
res = uri_parser.parse_uri(entity, port)
File "/var/lib/openshift/531b77fd500446980900010d/python/virtenv/lib/python2.7/site-packages/pymongo/uri_parser.py", line 269, in parse_uri
nodes = split_hosts(hosts, default_port=default_port)
File "/var/lib/openshift/531b77fd500446980900010d/python/virtenv/lib/python2.7/site-packages/pymongo/uri_parser.py", line 209, in split_hosts
nodes.append(parse_host(entity, port))
File "/var/lib/openshift/531b77fd500446980900010d/python/virtenv/lib/python2.7/site-packages/pymongo/uri_parser.py", line 137, in parse_host
raise ConfigurationError("Port number must be an integer.")
pymongo.errors.ConfigurationError: Port number must be an integer.
looks like OPENSHIFT_MONGODB_DB_PORT isn't set
print OPENSHIFT_MONGODB_DB_PORT --> NameError: name 'OPENSHIFT_MONGODB_DB_PORT' is not defined
Same with OPENSHIFT_MONGODB_DB_HOST
What would I need to do, to set up a connection?
Update:
I was able to connect directly via client by hardcoding info from rockmongo
client = MongoClient('mongodb://admin:password#[ip addr]:[port]/')
but when I do
client = MongoClient('mongodb:admin:password#%s:%s/' % os.environ['OPENSHIFT_MONGODB_DB_HOST'], os.environ['OPENSHIFT_MONGODB_DB_PORT']))
I get
[error] (<type 'exceptions.KeyError'>, KeyError('OPENSHIFT_MONGODB_DB_HOST',), <traceback object at 0x7f7bc8367248>)
The OpenShift connection variables are defined as environment variables, they cannot be accessed as normal Python variables. So the print statement you gave does not work, the following should;
import os
print os.environ['OPENSHIFT_MONGODB_DB_PORT']
You should change your code to;
client = MongoClient('mongodb:%s:%s/' % (os.environ['OPENSHIFT_MONGODB_DB_HOST'], os.environ['OPENSHIFT_MONGODB_DB_PORT))
You can refer to an example here.
I recently upgraded my GAE Python app to Python 2.7. Since then, I periodically get the following error with the dev server and the dev server serves up a blank page:
Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 168, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 206, in _LoadHandler
handler = __import__(path[0])
[...]
File "/Users/joneill/OpenSTV/OpenSTV/trunk/OpaVote-HR/main.py", line 2, in <module>
import views
[...]
File "/Users/joneill/OpenSTV/OpenSTV/trunk/OpaVote-HR/views.py", line 3, in <module>
from pytz.gae import pytz
[...]
File "/Users/joneill/OpenSTV/OpenSTV/trunk/OpaVote-HR/pytz/__init__.py", line 34, in <module>
from pkg_resources import resource_stream
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 662, in Decorate
return func(self, *args, **kwargs)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1818, in load_module
return self.FindAndLoadModule(submodule, fullname, search_path)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 662, in Decorate
return func(self, *args, **kwargs)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1690, in FindAndLoadModule
description)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 662, in Decorate
return func(self, *args, **kwargs)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1615, in LoadModuleRestricted
return source_file.load_module(submodule_fullname)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/dist/py_zipimport.py", line 246, in load_module
submodname, is_package, fullpath, source = self._get_source(fullmodname)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/dist/py_zipimport.py", line 207, in _get_source
source = self.zipfile.read(relpath.replace(os.sep, '/'))
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/zipfile.py", line 867, in read
return self.open(name, "r", pwd).read()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/zipfile.py", line 882, in open
zef_file = open(self.filename, 'rb')
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 578, in __init__
raise IOError(errno.EACCES, 'file not accessible', filename)
IOError: [Errno 13] file not accessible: '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg'
INFO 2012-01-21 20:50:44,222 dev_appserver.py:2832] "POST /manage HTTP/1.1" 500 -
Some notes:
This doesn't happen on the production server.
On the dev server, my app will work for a few minutes and then this error happens.
If I stop and restart my app on the dev server, it will work again for a few minutes.
I am using the latest version of gae-pytz and you can see that it fails in an import there.
The [...] that I removed are similar to the stuff you see near the end.
I don't know why setuptools is being invoked at the end.
I'm using a Mac with Lion.
I can use the dev server, but it is really annoying to stop and restart every few minutes. Any ideas how to fix this?
The actual problem from the stack trace, is your code is trying to import setup tools from site-packages, which the dev server won't do.
'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg'
You will need to include setuptools in you application code base. The fact that it works sometimes, suggests that you code paths through various modules vary, and maybe (depending on what your tesing in dev) different import orders mean setup tools has been imported somewhere else, or is only required at certain points in your code.
Have a look at line 4th line of the stack trace where pytz is imported, the next line is from pkg_resources import resource_stream thats whats triggering the rest of the import issue. I use a fake truncated pkg_resources at the root of my project, that doesn't end up trying to import stuff from setup tools. You can run the dev server in debug import mode which will tell you a lot more
Here is a fake pkg_resources.
"""Package resource API
--------------------
A resource is a logical file contained within a package, or a logical
subdirectory thereof. The package resource API expects resource names
to have their path parts separated with ``/``, *not* whatever the local
path separator is. Do not use os.path operations to manipulate resource
names being passed into the API.
The package resource API is designed to work with normal filesystem packages,
.egg files, and unpacked .egg files. It can also work in a limited way with
.zip files and with custom PEP 302 loaders that support the ``get_data()``
method.
"""
import sys, os, zipimport, time, re, imp, new
try:
frozenset
except NameError:
from sets import ImmutableSet as frozenset
from os import utime #, rename, unlink # capture these to bypass sandboxing
from os import open as os_open
There are probably other/better ways of doing this, but it works for me.
Oh, I would also suggest you use http://code.google.com/p/gae-pytz/ instead of pytz.
Cheers
An alternative answer to the above that I prefer.
The __init__.py file for pytz contains the following lines:
#try:
# from pkg_resources import resource_stream
#except ImportError:
resource_stream = None
I commented out the first three lines and that fixed the problem.
The problem is a bug in the App Engine dev server with Python 2.7. The solution is here:
File not accesible error (setuptools) in logs