Urllib2 - Django - Broken Pipe - python

I am new to Django and I am experiencing some troubles posting data to a django webapp.
On the django side I have a form (backed by couchdb - couchdbkit django ext.).
On the client side I post data with urllib2, in bulk (going through a list of data to post one by one)
data = urllib.urlencode(param)
req = urllib2.Request(OBJECT_SERVICE_URL + '/objects/create/', data)
req.add_header('Content-type', 'application/x-www-form-urlencoded')
res = urllib2.urlopen(req)
res.close()
After one record (one post passing), it get the message below, running again the next line passes and I have again this message :
res = urllib2.urlopen(req)
File "/usr/lib/python2.6/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/usr/lib/python2.6/urllib2.py", line 397, in open
response = meth(req, response)
File "/usr/lib/python2.6/urllib2.py", line 510, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.6/urllib2.py", line 435, in error
return self._call_chain(*args)
File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
result = func(*args)
File "/usr/lib/python2.6/urllib2.py", line 518, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 500: INTERNAL SERVER ERROR
The server says :
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/django/core/servers/basehttp.py", line 284, in run
self.finish_response()
File "/usr/local/lib/python2.6/dist-packages/django/core/servers/basehttp.py", line 324, in finish_response
self.write(data)
File "/usr/local/lib/python2.6/dist-packages/django/core/servers/basehttp.py", line 420, in write
self._write(data)
File "/usr/lib/python2.6/socket.py", line 318, in write
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 59571)
self.flush()
File "/usr/lib/python2.6/socket.py", line 297, in flush
self._sock.sendall(buffer(data, write_offset, buffer_size))
error: [Errno 104] Connection reset by peer
Traceback (most recent call last):
File "/usr/lib/python2.6/SocketServer.py", line 283, in _handle_request_noblock
----------------------------------------
self.process_request(request, client_address)
File "/usr/lib/python2.6/SocketServer.py", line 309, in process_request
self.finish_request(request, client_address)
File "/usr/lib/python2.6/SocketServer.py", line 322, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/local/lib/python2.6/dist-packages/django/core/servers/basehttp.py", line 570, in __init__
BaseHTTPRequestHandler.__init__(self, *args, **kwargs)
File "/usr/lib/python2.6/SocketServer.py", line 618, in __init__
self.finish()
File "/usr/lib/python2.6/SocketServer.py", line 661, in finish
self.wfile.flush()
File "/usr/lib/python2.6/socket.py", line 297, in flush
self._sock.sendall(buffer(data, write_offset, buffer_size))
error: [Errno 32] Broken pipe
I must state that I am working on the embedded django development web server...
And here is the output of the test :
======================================================================
ERROR: test_create_object (objects.tests.ObjectAppTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/patrick/workspace_py/objects/tests.py", line 21, in test_create_object
rs = c.post('/objects/create/', param)
File "/usr/local/lib/python2.6/dist-packages/django/test/client.py", line 455, in post
response = super(Client, self).post(path, data=data, content_type=content_type, **extra)
File "/usr/local/lib/python2.6/dist-packages/django/test/client.py", line 256, in post
return self.request(**r)
File "/usr/local/lib/python2.6/dist-packages/django/test/client.py", line 387, in request
response = self.handler(environ)
File "/usr/local/lib/python2.6/dist-packages/django/test/client.py", line 84, in __call__
response = self.get_response(request)
File "/usr/local/lib/python2.6/dist-packages/django/core/handlers/base.py", line 169, in get_response
response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
File "/usr/local/lib/python2.6/dist-packages/django/core/handlers/base.py", line 218, in handle_uncaught_exception
return callback(request, **param_dict)
File "/home/patrick/workspace_py/env/lib/python2.6/site-packages/coffin/views/defaults.py", line 34, in server_error
content = render_to_string(template_name, Context({}))
File "/home/patrick/workspace_py/env/lib/python2.6/site-packages/coffin/template/loader.py", line 50, in render_to_string
template = get_template(template_name)
File "/home/patrick/workspace_py/env/lib/python2.6/site-packages/coffin/template/loader.py", line 24, in get_template
return env.get_template(template_name)
File "/home/patrick/workspace_py/env/lib/python2.6/site-packages/jinja2/environment.py", line 719, in get_template
return self._load_template(name, self.make_globals(globals))
File "/home/patrick/workspace_py/env/lib/python2.6/site-packages/jinja2/environment.py", line 693, in _load_template
template = self.loader.load(self, name, globals)
File "/home/patrick/workspace_py/env/lib/python2.6/site-packages/jinja2/loaders.py", line 115, in load
source, filename, uptodate = self.get_source(environment, name)
File "/home/patrick/workspace_py/env/lib/python2.6/site-packages/jinja2/loaders.py", line 377, in get_source
raise TemplateNotFound(template)
TemplateNotFound: 500.html
----------------------------------------------------------------------
Does anyone know how to fix ?
Patrick

Some times I found weird errors and quick connection resets using development server (like in your log). If your application is about http, REST and webservices you should use Apache or Gunicorn so you feel a more realistic enviroment while developing.

Related

Getting error code when trying to download using pytube

Not sure what it is, but its been driving me nuts and im such a newbie I tried so many things but im not sure what it means, ive tried installing and reinstalling pytube but nothing changed.
Here's my code
from pytube import YouTube
link = input("Enter in a link: ")
video = YouTube(link)
print(video.title)
print(video.streams.filter(progressive=True))
stream = video.streams.get_by_itag(22)
stream.download()
print("Video downloaded")
The entire error message:
Traceback (most recent call last):
- File "c:\PytubeDownloader\YoutubeDownloader2.py", line 5, in <module>
video = YouTube(link)
- File "C:\Python\lib\site-packages\pytube\__main__.py", line 91, in __init__
self.prefetch()
- File "C:\Python\lib\site-packages\pytube\__main__.py", line 181, in prefetch
self.vid_info_raw = request.get(self.vid_info_url)
- File "C:\Python\lib\site-packages\pytube\request.py", line 36, in get
return _execute_request(url).read().decode("utf-8")
- File "C:\Python\lib\site-packages\pytube\request.py", line 24, in _execute_request
return urlopen(request) # nosec
- File "C:\Python\lib\urllib\request.py", line 216, in urlopen
return opener.open(url, data, timeout)
- File "C:\Python\lib\urllib\request.py", line 525, in open
response = meth(req, response)
- File "C:\Python\lib\urllib\request.py", line 634, in http_response
response = self.parent.error(
- File "C:\Python\lib\urllib\request.py", line 557, in error
result = self._call_chain(*args)
- File "C:\Python\lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
- File "C:\Python\lib\urllib\request.py", line 749, in http_error_302
return self.parent.open(new, timeout=req.timeout)
- File "C:\Python\lib\urllib\request.py", line 525, in open
response = meth(req, response)
- File "C:\Python\lib\urllib\request.py", line 634, in http_response
response = self.parent.error(
- File "C:\Python\lib\urllib\request.py", line 563, in error
return self._call_chain(*args)
- File "C:\Python\lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
- File "C:\Python\lib\urllib\request.py", line 643, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 410: Gone
Ok so I did what was recommended in Pytube: urllib.error.HTTPError: HTTP Error 410: Gone and now im getting a completely different error
Traceback (most recent call last):
File "c:\PytubeDownloader\YoutubeDownloader2.py", line 9, in
print(video.streams.filter(progressive=True))
File "C:\Python\lib\site-packages\pytube_main_.py", line 292, in streams
return StreamQuery(self.fmt_streams)
File "C:\Python\lib\site-packages\pytube_main_.py", line 184, in fmt_streams
extract.apply_signature(stream_manifest, self.vid_info, self.js)
File "C:\Python\lib\site-packages\pytube\extract.py", line 409, in apply_signature
cipher = Cipher(js=js)
File "C:\Python\lib\site-packages\pytube\cipher.py", line 33, in init
raise RegexMatchError(
pytube.exceptions.RegexMatchError: init: could not find match for ^\w+\W
Experienced same error. In cipher.py replace the regexs in pattern in get_initial_function_name with:
r'\bc\s*&&\s*d\.set\([^,]+\s*,\s*\([^)]*\)\s*\(\s*(?P<sig>[a-zA-Z0-9$]+)\('.

yahoo_finance api throws 404. urllib2.HTTPError: HTTP Error 400: Bad Request

$ pip install yahoo-finance
$ python
Then in python console,
>>> from yahoo_finance import Share
>>> yahoo = Share('YHOO')
Traceback (most recent call last): File "", line 1, in File
"/Users/ishandutta2007/virtualenvironment/googlfin/lib/python2.7/site-packages/yahoo_finance/init.py",
line 178, in init
self.refresh() File "/Users/ishandutta2007/virtualenvironment/googlfin/lib/python2.7/site-packages/yahoo_finance/init.py",
line 142, in refresh
self.data_set = self._fetch() File "/Users/ishandutta2007/virtualenvironment/googlfin/lib/python2.7/site-packages/yahoo_finance/init.py",
line 181, in _fetch
data = super(Share, self)._fetch() File "/Users/ishandutta2007/virtualenvironment/googlfin/lib/python2.7/site-packages/yahoo_finance/init.py",
line 134, in _fetch
data = self._request(query) File "/Users/ishandutta2007/virtualenvironment/googlfin/lib/python2.7/site-packages/yahoo_finance/init.py",
line 118, in _request
response = yql.YQLQuery().execute(query) File "/Users/ishandutta2007/virtualenvironment/googlfin/lib/python2.7/site-packages/yahoo_finance/yql.py",
line 61, in execute
'env': DATATABLES_URL File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout) File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 435, in open
response = meth(req, response) File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 548, in http_response
'http', request, response, code, msg, hdrs) File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 473, in error
return self._call_chain(*args) File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args) File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 556, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) urllib2.HTTPError: HTTP Error 400: Bad Request

python ssl eof occurred in violation of protocol, wantwriteerror, zeroreturnerror

I'm running many celery tasks (20,000) using gevent for the pool (also monkey patching all). Each of these tasks hit 3rd party services like adwords to pull data.
I keep having tasks fail because of underlying SSL errors. Below are the stack-traces from a few of the exceptions (in no particular order, these are failures from separate tasks). I also get WantWriteError and ZeroReturnError occasionally but the EOF error seems to come up the most.
These errors happen while using different client libraries like googleads (suds library for soap communication) as well as requests and elasticsearch. I'm guessing some of these libraries use urllib3 while others use urllib2 etc.
There has been a lot of info on the EOF issue and forcing TLSv1 but I can't seem to find a resolution that works.
I'm not sure if I'm running too many requests at once, if somethings blocking or what; any help would be greatly appreciated, I'm pulling my hair out over this one.
Traceback (most recent call last):
...
File "/srv/reporting/src/reporting/stats/adwords/client.py", line 58, in _awql_report
downloader = self._get_client(client_id).GetReportDownloader(version=self.REPORT_DOWNLOADER_VERSION)
File "/usr/local/lib/python2.7/dist-packages/googleads/adwords.py", line 283, in GetReportDownloader
return ReportDownloader(self, version, server)
File "/usr/local/lib/python2.7/dist-packages/googleads/adwords.py", line 400, in __init__
proxy=proxy_option, cache=self._adwords_client.cache).wsdl.schema
File "/usr/local/lib/python2.7/dist-packages/suds/client.py", line 115, in __init__
self.wsdl = reader.open(url)
File "/usr/local/lib/python2.7/dist-packages/suds/reader.py", line 150, in open
d = self.fn(url, self.options)
File "/usr/local/lib/python2.7/dist-packages/suds/wsdl.py", line 136, in __init__
d = reader.open(url)
File "/usr/local/lib/python2.7/dist-packages/suds/reader.py", line 74, in open
d = self.download(url)
File "/usr/local/lib/python2.7/dist-packages/suds/reader.py", line 92, in download
fp = self.options.transport.open(Request(url))
File "/usr/local/lib/python2.7/dist-packages/suds/transport/https.py", line 62, in open
return HttpTransport.open(self, request)
File "/usr/local/lib/python2.7/dist-packages/suds/transport/http.py", line 67, in open
return self.u2open(u2request)
File "/usr/local/lib/python2.7/dist-packages/suds/transport/http.py", line 132, in u2open
return url.open(u2request, timeout=tm)
File "/usr/lib/python2.7/urllib2.py", line 400, in open
response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 418, in _open
'_open', req)
File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 1216, in https_open
return self.do_open(httplib.HTTPSConnection, req)
File "/usr/lib/python2.7/urllib2.py", line 1178, in do_open
raise URLError(err)
URLError: <urlopen error [Errno 8] _ssl.c:504: EOF occurred in violation of protocol>
Traceback (most recent call last):
...
File "/srv/reporting/src/reporting/stats/analytics/client.py", line 57, in get_access_token
response = requests.post('https://accounts.google.com/o/oauth2/token', data)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 88, in post
return request('post', url, data=data, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 456, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 559, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 382, in send
raise SSLError(e, request=request)
SSLError: [Errno bad handshake] (-1, 'Unexpected EOF')
Traceback (most recent call last):
...
self.es.index(index=self.INDICE, doc_type=self.ROOT_CLASS.__name__, body=self.export(obj), id=obj.id)
File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/utils.py", line 68, in _wrapped
return func(*args, params=params, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/__init__.py", line 213, in index
_make_path(index, doc_type, id), params=params, body=body)
File "/usr/local/lib/python2.7/dist-packages/elasticsearch/transport.py", line 284, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_requests.py", line 44, in perform_request
response = self.session.request(method, url, data=body, timeout=timeout or self.timeout)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 456, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 559, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 327, in send
timeout=timeout
File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py", line 493, in urlopen
body=body, headers=headers)
File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py", line 319, in _make_request
httplib_response = conn.getresponse(buffering=True)
File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 407, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 365, in _read_status
line = self.fp.readline()
File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 273, in readline
data = self._sock.recv(self._rbufsize)
File "/usr/local/lib/python2.7/dist-packages/OpenSSL/SSL.py", line 995, in recv
self._raise_ssl_error(self._ssl, result)
File "/usr/local/lib/python2.7/dist-packages/OpenSSL/SSL.py", line 851, in _raise_ssl_error
raise ZeroReturnError()
ZeroReturnError
So let's break this down by each traceback block. The first ends with:
File "/usr/lib/python2.7/urllib2.py", line 1178, in do_open
raise URLError(err)
URLError: <urlopen error [Errno 8] _ssl.c:504: EOF occurred in violation of protocol>
This is coming from urllib2. The fact that this receives an EOF makes me think that the server closed the connection while you were waiting for that "thread" to read from the socket again. You might want to use more time.sleep(0) to yield to gevent.
The second traceback comes from requests:
File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 382, in send
raise SSLError(e, request=request)
SSLError: [Errno bad handshake] (-1, 'Unexpected EOF')
The [Errno bad handshake] would make me tend to think this is a problem establishing the connection which could be caused by an unexpected EOF. Is that caused by using gevent? I'm uncertain.
The final traceback is definitely from requests as well but it also is coming out of PyOpenSSL and isn't being caught by urllib3 or requests.
File "/usr/local/lib/python2.7/dist-packages/OpenSSL/SSL.py", line 851, in _raise_ssl_error
raise ZeroReturnError()
ZeroReturnError
I did some searching and found that "According to the pyOpenSSL docs ZeroReturnError means that the SSL connection has been closed cleanly." This says to me that the server again closed the connection because you took to long to read anything from the socket.
In short, I think you need to explicitly yield more often just to ensure that these socket problems don't arise. That's just a guess though, so take it with a grain of salt.

Neo4Django create node not working in manage.py shell

I have neo4j-2.1.3 installed and server running on my Linux system . I created model "publisher" in my app . And then in manage.py shell , whenever I save a node with
from BooksGraph.models import Publisher
p=Publisher.objects.create(name='Sunny',address='b-1/196')
a long error pops up with:
Traceback (most recent call last):
File "", line 1, in File
"/usr/local/lib/python2.7/dist-packages/neo4django/db/models/manager.py",
line 42, in create return self.get_query_set().create(**kwargs) File
"/usr/local/lib/python2.7/dist-packages/neo4django/db/models/query.py",
line 1052, in create return super(NodeQuerySet, self).create(**kwargs)
File
"/usr/local/lib/python2.7/dist-packages/django/db/models/query.py",
line 377, in create obj.save(force_insert=True, using=self.db) File
"/usr/local/lib/python2.7/dist-packages/neo4django/db/models/base.py",
line 325, in save return super(NodeModel, self).save(using=using,
**kwargs) File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py",
line 463, in save self.save_base(using=using,
force_insert=force_insert, force_update=force_update) File
"/usr/local/lib/python2.7/dist-packages/neo4django/db/models/base.py",
line 341, in save_base self._save_neo4j_node(using) File "",
line 2, in _save_neo4j_node File
"/usr/local/lib/python2.7/dist-packages/neo4django/db/models/base.py",
line 111, in trans_method
len(connections[args[0].using]._transactions) < 1: File
"/usr/local/lib/python2.7/dist-packages/neo4django/utils.py", line
313, in getitem **db['OPTIONS']) File
"/usr/local/lib/python2.7/dist-packages/neo4django/neo4jclient.py",
line 29, in init super(EnhancedGraphDatabase,
self).init(*args, **kwargs) File
"/usr/local/lib/python2.7/dist-packages/neo4jrestclient/client.py",
line 74, in init response = Request(**self._auth).get(self.url)
File
"/usr/local/lib/python2.7/dist-packages/neo4jrestclient/request.py",
line 63, in get return self._request('GET', url, headers=headers) File
"/usr/local/lib/python2.7/dist-packages/neo4django/db/init.py",
line 60, in _request headers) File
"/usr/local/lib/python2.7/dist-packages/neo4jrestclient/request.py",
line 198, in _request auth=auth, verify=verify) File
"/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
468, in get return self.request('GET', url, **kwargs) File
"/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
456, in request resp = self.send(prep, **send_kwargs) File
"/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
559, in send r = adapter.send(request, **kwargs) File
"/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line
378, in send raise ProxyError(e) ProxyError: ('Cannot connect to
proxy.', error(113, 'No route to host'))
This sounds like a network setup problem. Can you check what URL the library is trying to connect to and that that one really goes to your local Neo4j Server?

Moved Permanently error when running dev_appserver.py

I am trying the basic helloworld example (https://developers.google.com/appengine/docs/python/gettingstartedpython27/helloworld) and keep getting the HTTP Error 301: Moved Permanently error whenever I try to test my code using dev_appserver.py
The 2 files I have are copied and pasted exactly from the developers.google.com site.
I have included the location of where dev_appserver.py is in both PATH and PYTHONPATH
I am running this on Linux with python 2.7.3 and appengine v1.8.4
The output on the terminal when I run this is...
[verma#localhost python]$ dev_appserver.py helloworld/
WARNING 2013-09-11 04:45:49,988 api_server.py:327] Could not initialize images API; you are likely missing the Python "PIL" module.
INFO 2013-09-11 04:45:49,999 api_server.py:138] Starting API server at: http://localhost:57128
INFO 2013-09-11 04:45:50,021 dispatcher.py:164] Starting module "default" running at: http://localhost:8080
INFO 2013-09-11 04:45:50,023 admin_server.py:117] Starting admin server at: http://localhost:8000
HTTPError()
HTTPError()
Traceback (most recent call last):
File "/home/verma/Documents/gae/google_appengine/lib/cherrypy/cherrypy/wsgiserver/wsgiserver2.py", line 1302, in communicate
req.respond()
File "/home/verma/Documents/gae/google_appengine/lib/cherrypy/cherrypy/wsgiserver/wsgiserver2.py", line 831, in respond
self.server.gateway(self).respond()
File "/home/verma/Documents/gae/google_appengine/lib/cherrypy/cherrypy/wsgiserver/wsgiserver2.py", line 2115, in respond
response = self.req.server.wsgi_app(self.env, self.start_response)
File "/home/verma/Documents/gae/google_appengine/google/appengine/tools/devappserver2/wsgi_server.py", line 256, in __call__
return app(environ, start_response)
File "/home/verma/Documents/gae/google_appengine/google/appengine/tools/devappserver2/request_rewriter.py", line 311, in _rewriter_middleware
response_body = iter(application(environ, wrapped_start_response))
File "/home/verma/Documents/gae/google_appengine/google/appengine/tools/devappserver2/python/request_handler.py", line 97, in __call__
self._flush_logs(response.get('logs', []))
File "/home/verma/Documents/gae/google_appengine/google/appengine/tools/devappserver2/python/request_handler.py", line 233, in _flush_logs
apiproxy_stub_map.MakeSyncCall('logservice', 'Flush', request, response)
File "/home/verma/Documents/gae/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 94, in MakeSyncCall
return stubmap.MakeSyncCall(service, call, request, response)
File "/home/verma/Documents/gae/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 328, in MakeSyncCall
rpc.CheckSuccess()
File "/home/verma/Documents/gae/google_appengine/google/appengine/api/apiproxy_rpc.py", line 156, in _WaitImpl
self.request, self.response)
File "/home/verma/Documents/gae/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 200, in MakeSyncCall
self._MakeRealSyncCall(service, call, request, response)
File "/home/verma/Documents/gae/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 226, in _MakeRealSyncCall
encoded_response = self._server.Send(self._path, encoded_request)
File "/home/verma/Documents/gae/google_appengine/google/appengine/tools/appengine_rpc.py", line 393, in Send
f = self.opener.open(req)
File "/usr/lib64/python2.7/urllib2.py", line 410, in open
response = meth(req, response)
File "/usr/lib64/python2.7/urllib2.py", line 523, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python2.7/urllib2.py", line 448, in error
return self._call_chain(*args)
File "/usr/lib64/python2.7/urllib2.py", line 382, in _call_chain
result = func(*args)
File "/usr/lib64/python2.7/urllib2.py", line 531, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 301: Moved Permanently
Traceback (most recent call last):
File "/home/verma/Documents/gae/google_appengine/lib/cherrypy/cherrypy/wsgiserver/wsgiserver2.py", line 1302, in communicate
req.respond()
File "/home/verma/Documents/gae/google_appengine/lib/cherrypy/cherrypy/wsgiserver/wsgiserver2.py", line 831, in respond
self.server.gateway(self).respond()
File "/home/verma/Documents/gae/google_appengine/lib/cherrypy/cherrypy/wsgiserver/wsgiserver2.py", line 2115, in respond
response = self.req.server.wsgi_app(self.env, self.start_response)
File "/home/verma/Documents/gae/google_appengine/google/appengine/tools/devappserver2/wsgi_server.py", line 256, in __call__
return app(environ, start_response)
File "/home/verma/Documents/gae/google_appengine/google/appengine/tools/devappserver2/request_rewriter.py", line 311, in _rewriter_middleware
response_body = iter(application(environ, wrapped_start_response))
File "/home/verma/Documents/gae/google_appengine/google/appengine/tools/devappserver2/python/request_handler.py", line 97, in __call__
self._flush_logs(response.get('logs', []))
File "/home/verma/Documents/gae/google_appengine/google/appengine/tools/devappserver2/python/request_handler.py", line 233, in _flush_logs
apiproxy_stub_map.MakeSyncCall('logservice', 'Flush', request, response)
File "/home/verma/Documents/gae/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 94, in MakeSyncCall
return stubmap.MakeSyncCall(service, call, request, response)
File "/home/verma/Documents/gae/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 328, in MakeSyncCall
rpc.CheckSuccess()
File "/home/verma/Documents/gae/google_appengine/google/appengine/api/apiproxy_rpc.py", line 156, in _WaitImpl
self.request, self.response)
File "/home/verma/Documents/gae/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 200, in MakeSyncCall
self._MakeRealSyncCall(service, call, request, response)
File "/home/verma/Documents/gae/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 226, in _MakeRealSyncCall
encoded_response = self._server.Send(self._path, encoded_request)
File "/home/verma/Documents/gae/google_appengine/google/appengine/tools/appengine_rpc.py", line 393, in Send
f = self.opener.open(req)
File "/usr/lib64/python2.7/urllib2.py", line 410, in open
response = meth(req, response)
File "/usr/lib64/python2.7/urllib2.py", line 523, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python2.7/urllib2.py", line 448, in error
return self._call_chain(*args)
File "/usr/lib64/python2.7/urllib2.py", line 382, in _call_chain
result = func(*args)
File "/usr/lib64/python2.7/urllib2.py", line 531, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 301: Moved Permanently
INFO 2013-09-11 04:46:01,635 module.py:593] default: "GET / HTTP/1.1" 500 -
I have a feeling I am missing something very basic but can't find it. By the way this was working a few days back and I don't remember doing anything stupid to break this :-(
Quick Solution to Fix your Problem.
Open FILE [GAE Installation path]/google/appengine/tools/appengine_rpc.py"
Goto Line 578
Comment out the following line
# opener.add_handler(fancy_urllib.FancyProxyHandler())
It appears that dev_apserver needs to contact the google app engine servers on the internet every once in a while (even if you do not ask it to check for updates). The problem I had was that it cannot access the google servers when behind a proxy (even though I have properly exported http_proxy and https_proxy).
When I use a wired network directly connected to the internet I have no problems at all.
Once dev_appserver has 'called home', it will continue to work without issue behind a proxy for a few more days, and then I get this problem again.
In the past I have noticed similar behavior when connected on wifi as well.
It appears that dev_appserver works best when on a wired network without proxies.
This is based simply on my observations; it would be great if someone could give a definitive answer to this question.

Categories