So I've been trying to implement the Janrain Engage plugin to the application I'm working on (Google App Engine (Python 2.7)), using GAESessions as the sessions library.
Following the instructions given on the GAESessions page, I created the "gaesessions" folder (containing "__init__.py") as well as "appengine_config.py" in my application's root directory, as well as the relevant files to process the plugin.
Attempting to login via Janrain, however, threw me a 500 error and gave me this traceback in the GAE logs:
E 2013-03-25 07:06:55.535
'thread._local' object has no attribute 'current_session'
Traceback (most recent call last):
File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1536, in __call__
rv = self.handle_exception(request, response, e)
File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1530, in __call__
rv = self.router.dispatch(request, response)
File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~k-sketch-test/10.366190612177083948/rpx.py", line 56, in post
session = get_current_session()
File "/base/data/home/apps/s~k-sketch-test/10.366190612177083948/gaesessions/__init__.py", line 38, in get_current_session
return _tls.current_session
AttributeError: 'thread._local' object has no attribute 'current_session'
I have searched through the other posts regarding the "get_current_session()" issue, but they seem to be referencing 'local' instead of 'thread._local'.
Any ideas on what's going on here? Thanks in advance!
========
{root folder}/appengine_config.py
from gaesessions import SessionMiddleware
import os
COOKIE_KEY = '<hidden - was generated through os.urandom(64)>'
def webapp_add_wsgi_middleware(app):
from google.appengine.ext.appstats import recording
app = SessionMiddleware(app, cookie_key=COOKIE_KEY)
app = recording.appstats_wsgi_middleware(app)
return app
========
{root folder}/appengine_config.py Is the file appengine_config.py in the root location?
aaps/app.yaml
/main.py
/appengine_config.py
/gaesessions #folder
Can look a basic tutorial on gaesessions
Related
I want to setup a machine learning pipeline that is callable by flask but I am facing some issues, these links are for the documentations I have read so far:
https://exploreflask.com/en/latest/views.html#view-decorators
https://flask.palletsprojects.com/en/1.1.x/api/#flask.Flask
Let me explain the pipeline I have in mind:
pull a dataframe from a PostgreSQL database
encode said dataframe to make it ready for most algorithms
split up the data
feed to a pipeline and determine accuracy
store the model in a pickle file
What is working so far:
All parts are working as a regular script
I can just slap all the steps into one huge flask file with one decorator and it would run as well (my emergency solution)
The File Structure
The encoder script:
#Flask main thread
#makes flask start this part as application and not as module
app = Flask('encoder_module')
#app.route('/df_encoder')
def df_encoder(rng = 4):
encoding stuff
`return df`
The Pipeline script (random forest regressor here)
app = Flask('pipeline_module')
#app.route('/pipeline_rfr')
def pipeline_rfr():
pipeline stuff
`return grid_search_rfr`
The pickle module:
app = Flask('pickle_module')
#app.route('/store_reg_pickle')
def store_pickle():
"""
Usage of a Pickle Model -Storage of a trained Model
"""
model = grid_search_rfr
#specify file name in letter strings
model_file = "regression_model"
with open(model_file, mode='wb') as m_f:
pickle.dump(model, m_f)
print(f"Model saved in: {os.getcwd()}")
return model_file
The Main Flask File
#packages
from flask import Flask
from encoder_main_thread import df_encoder
from rfr_pipeline_function import pipeline_rfr
from pickle_call import store_pickle
app = Flask(__name__.split('.')[0])
#app.route('/regression_pipe')
#df_encoder
#pipeline_rfr
#store_reg_pickle
def regression_pipe():
`return 'pipeline done`
The problem iss that the return value of the encoder cannot be a dataframe, only a string, tuple, etc.
Is there a workaround for this?
I actually want it to be a flawless passing of the dataframe to the pipeline and eventually storing it in the pickle file which is then saved in the folder.
For some reason it cannot detect the pickle file import and throws following error:
Use a production WSGI server instead.
* Debug mode: off
Traceback (most recent call last):
File "C:\ANACONDA3\Scripts\flask-script.py", line 9, in <module>
sys.exit(main())
File "C:\ANACONDA3\lib\site-packages\flask\cli.py", line 967, in main
cli.main(args=sys.argv[1:], prog_name="python -m flask" if as_module else None)
File "C:\ANACONDA3\lib\site-packages\flask\cli.py", line 586, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "C:\ANACONDA3\lib\site-packages\click\core.py", line 782, in main
rv = self.invoke(ctx)
File "C:\ANACONDA3\lib\site-packages\click\core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\ANACONDA3\lib\site-packages\click\core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\ANACONDA3\lib\site-packages\click\core.py", line 610, in invoke
return callback(*args, **kwargs)
File "C:\ANACONDA3\lib\site-packages\click\decorators.py", line 73, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "C:\ANACONDA3\lib\site-packages\click\core.py", line 610, in invoke
return callback(*args, **kwargs)
File "C:\ANACONDA3\lib\site-packages\flask\cli.py", line 848, in run_command
app = DispatchingApp(info.load_app, use_eager_loading=eager_loading)
File "C:\ANACONDA3\lib\site-packages\flask\cli.py", line 305, in __init__
self._load_unlocked()
File "C:\ANACONDA3\lib\site-packages\flask\cli.py", line 330, in _load_unlocked
self._app = rv = self.loader()
File "C:\ANACONDA3\lib\site-packages\flask\cli.py", line 388, in load_app
app = locate_app(self, import_name, name)
File "C:\ANACONDA3\lib\site-packages\flask\cli.py", line 240, in locate_app
__import__(module_name)
File "C:\Users\bill-\OneDrive\Dokumente\Docs Bill\TA_files\functions_scripts_storage\flask_test\flask_regression_pipeline.py", line 18, in <module>
#store_reg_pickle
NameError: name 'store_reg_pickle' is not defined
If you wish I could upload the entire scripts but that is a lot to look through and since it is working as a long regular pice of code, the mistake needs to be somewhere with my flask setup.
I successfully duplicated the Tasks API from this video, but I am unable to successfully translate this format to using the YouTube API.
Here is my .py file:
import httplib2
import os
import sys
import jinja2
import webapp2
import logging
import pprint
from oauth2client import tools
from oauth2client.client import flow_from_clientsecrets
from oauth2client.client import OAuth2WebServerFlow
from oauth2client.client import AccessTokenRefreshError
from oauth2client.tools import argparser, run_flow
from oauth2client.appengine import OAuth2Decorator
from apiclient.discovery import build
from google.appengine.ext.webapp import util
from google.appengine.ext.webapp import template
decorator = OAuth2Decorator(
client_id = '*my client ID*',
client_secret = '*my client secret*',
scope='https://www.googleapis.com/auth/youtube')
service = build("youtube", "v3")
class MainHandler(webapp2.RequestHandler):
#decorator.oauth_required
def get (self):
self.response.headers['Content-Type'] = 'text/plain'
channels_list = service.channels().list(
mine=True,
part="id"
).execute(http = decorator.http())
self.response.out.write (pprint.pformat(channels_list))
app = webapp2.WSGIApplication (
[
('/', MainHandler),
(decorator.callback_path, decorator.callback_handler()),
],
debug=True)
Here is my Traceback:
Traceback (most recent call last):
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1102, in __call__
return handler.dispatch()
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "C:\Users\...\testapp\oauth2client\appengine.py", line 733, in check_oauth
resp = method(request_handler, *args, **kwargs)
File "C:\Users\...\testapp\testapp.py", line 35, in get
).execute(http = decorator.http())
File "C:\Users\...\testapp\oauth2client\util.py", line 129, in positional_wrapper
return wrapped(*args, **kwargs)
File "C:\Users\...\testapp\apiclient\http.py", line 723, in execute
raise HttpError(resp, content, uri=self.uri)
HttpError: <HttpError 403 when requesting https://www.googleapis.com/youtube/v3/channels?part=id&mine=true&alt=json returned "Insufficient Permission">
INFO 2015-06-13 12:27:54,515 module.py:788] default: "GET / HTTP/1.1" 500 2278
I have checked and double-checked that I have both the YouTube Data API and YouTube Analytics API enabled this client ID. I have even disabled them and re-enabled to check, but I am still getting this error.
I am new to GAE and its methods, so maybe I am not understanding an error in the Traceback.
One note is that I was getting "ImportError: No module named..." for apiclient, httplib2, oauth2client, and uritemplate, so I moved those folders directly into my app file (and didn't get that error again). Not sure if moving those directly into the folder is causing errors.
Well I hope I didn't waste anyone's time, but if anyone else has this issue, I found this question, and although I didn't have a memcache issue, I revoked permission for my app from the user account I was using, refreshed the app and gave permission again and now it seems to be working.
I guess this was a Google-side issue. Thanks to anyone who spent the time to read my question.
I don't know why my project show the next cookie error. Could someone help me?
PATH
->test
->lib
->public
->templates
- app.yaml
- main.py
- client_secrets.json
- session-secret (python -c "import os;print os.urandom(64)" > session.secret)
When I use my App Engine Launcher (release: "1.7.5") and check out my localhost web page
I chose my Google account to add permissions in accounts.google.com/AccountChooser?service....... (redirect) and then accept conditions of the scopes
The log console shows the next error:
Traceback (most recent call last):
File "C:\Program Files\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "C:\Program Files\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "C:\Program Files\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "C:\Program Files\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1102, in __call__
return handler.dispatch()
File "C:\Program Files\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "C:\Program Files\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "C:\Program Files\Google\google_appengine\pruebasDocs\main.py", line 320, in get
creds = self.GetCodeCredentials()
File "C:\Program Files\Google\google_appengine\pruebasDocs\main.py", line 194, in GetCodeCredentials
session.set_secure_cookie(name='userid', value=userid)
File "lib\sessions.py", line 160, in set_secure_cookie
self.set_cookie(name, value, expires_days=expires_days, **kwargs)
File "lib\sessions.py", line 141, in set_cookie
self.response.headers._headers.append(('Set-Cookie', str(vals.OutputString(None))))
**AttributeError: ResponseHeaders instance has no attribute '_headers'**
lo.....t:8080/?code=4/00VfZ4DJ8d0P99v1kwn0yjBofcbq.gn6ceL8RBx0XYKs_1NgQtmXj_6WohwI
MAIN.PY
def GetCodeCredentials(self):
# Other frameworks use different API to get a query parameter.
code = self.request.get('code')
if not code:
# returns None to indicate that no code was passed from Google Drive.
return None
# Auth flow is a controller that is loaded with the client information,
# including client_id, client_secret, redirect_uri etc
oauth_flow = self.CreateOAuthFlow()
# Perform the exchange of the code. If there is a failure with exchanging
# the code, return None.
try:
creds = oauth_flow.step2_exchange(code)
except FlowExchangeError:
return None
# Create an API service that can use the userinfo API. Authorize it with our
# credentials that we gained from the code exchange.
users_service = CreateService('oauth2', 'v2', creds)
# Make a call against the userinfo service to retrieve the user's information.
# In this case we are interested in the user's "id" field.
userid = users_service.userinfo().get().execute().get('id')
# Store the user id in the user's cookie-based session.
session = sessions.LilCookies(self, SESSION_SECRET)
session.set_secure_cookie(name='userid', value=userid)
SESSIONS.PY
# output all their cookies to the headers at once before a response flush.
for vals in new_cookie.values():
self.response.headers._headers.append(('Set-Cookie', vals.OutputString(None)))
I have this piece of code which is running perfectly on localhost but throws up this obscure error on GAE:
import_string() failed for 'webapp2_extras.appengine.auth.models.User' . Possible reasons are: - missing __init__.py in a package; - package or module
My import statements:
from webapp2_extras import auth
from webapp2_extras import sessions
from webapp2_extras.auth import InvalidAuthIdError
from webapp2_extras.auth import InvalidPasswordError
Usage of auth's user model:
user = self.auth.store.user_model.create_user(username, password_raw = password, email = email)
if not user[0]: #returns a tuple with [boolean, user_info]
return 'Create user error'
else:
self.set_flash("Thank you for registering. Please login!")
self.redirect(self.auth_config['login_url'])
Full code
UPDATE (Full stack trace)
import_string() failed for 'webapp2_extras.appengine.auth.models.User'. Possible reasons are:
- missing __init__.py in a package;
- package or module path not included in sys.path;
- duplicated package or module name taking precedence in sys.path;
- missing module, class, function or variable;
Original exception:
ImportError: No module named ndb
Debugged import:
- 'webapp2_extras' found in '/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2_extras/__init__.pyc'.
- 'webapp2_extras.appengine' found in '/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2_extras/appengine/__init__.pyc'.
- 'webapp2_extras.appengine.auth' found in '/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2_extras/appengine/auth/__init__.pyc'.
- 'webapp2_extras.appengine.auth.models' not found.
Traceback (most recent call last):
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1511, in __call__
rv = self.handle_exception(request, response, e)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__
rv = self.router.dispatch(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__
return handler.dispatch()
File "/base/data/home/apps/s~webapp-auth/1.358936463581927371/main.py", line 34, in dispatch
response = super(BaseHandler, self).dispatch()
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 547, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~webapp-auth/1.358936463581927371/main.py", line 127, in post
user = self.auth.store.user_model.create_user(username, password_raw = password, email = email)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 701, in __get__
value = self.func(obj)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2_extras/auth.py", line 131, in user_model
cls = self.config['user_model'] = webapp2.import_string(cls)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1824, in import_string
return getattr(__import__(module, None, None, [obj]), obj)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2_extras/appengine/auth/models.py", line 13, in <module>
from ndb import model
Look similar to this issue which was already fixed by webapp 2.5.1
Make sure you import the latest version of webapp2, by adding those line to your app.yaml file:
libraries:
- name: webapp2
version: latest
As a workaround you can add the following lines to your application:
import sys
from google.appengine.ext import ndb
sys.modules['ndb'] = ndb
I am running into an error that I am unable to isolate the root cause of. The error is the following: "ReferenceProperty failed to be resolved: [u'StatusLog', STATUSLOGSID]". This error only occurs sometimes, about once or twice a day. The scripts that generate this error succeed way more often than they fail. The strangest thing about the error is that it is failing to resolve a reference property, which should never be the case (in regards to this situation) because the entities that are being referenced are never deleted by my webapp. Furthermore, I am not generating the keys that are being referenced, the Google App Engine is. The relevant code is listed below.
THE GAE TRANSACTION:
def updateStatus(key):
hbo = HBO.get(key)
hbo.updateStatus()
hbo.put()
class HBOCRON(webapp2.RequestHandler):
def get(self):
keys = db.Query(HBO, keys_only = True).filter("inactive = ", False)
XG_ON = db.create_transaction_options(xg=True)
for key in keys: db.run_in_transaction_options(XG_ON, updateStatus, key)
app = webapp2.WSGIApplication([('/cron/hbo', HBOCRON)],debug=True)
Two other relevant functions...
def logStatus(self):
self.status = StatusLog(
hbo = self,
prev = self.status,
date = datetime.datetime.now(),
on = self.online(),
up = self.upToDate(),
dns = self.DNS_update_needed,
dis = self.manually_disabled).put()
def updateStatus(self):
status = self.status
if status is None \
or status.on != self.online() \
or status.up != self.upToDate() \
or status.dns != self.DNS_update_needed:
self.logStatus()
self.flagged = True
elif status.dis != self.manually_disabled:
self.logStatus()
Traceback:
ReferenceProperty failed to be resolved: [u'StatusLog', 248327L]
Traceback (most recent call last):
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1511, in __call__
rv = self.handle_exception(request, response, e)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__
rv = self.router.dispatch(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__
return handler.dispatch()
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 547, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~hs-hbo/1.357660268729453201/api/hbo/getCheckin.py", line 88, in post
(hbo, data) = db.run_in_transaction_options(XG_ON, checkinTransaction, self.request)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/api/datastore.py", line 2476, in RunInTransactionOptions
ok, result = _DoOneTry(new_connection, function, args, kwargs)
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/api/datastore.py", line 2498, in _DoOneTry
result = function(*args, **kwargs)
File "/base/data/home/apps/s~hs-hbo/1.357660268729453201/api/hbo/getCheckin.py", line 33, in checkinTransaction
hbo.updateStatus()
File "/base/data/home/apps/s~hs-hbo/1.357660268729453201/shared/datastore.py", line 116, in updateStatus
return self.logStatus()
File "/base/data/home/apps/s~hs-hbo/1.357660268729453201/shared/datastore.py", line 102, in logStatus
prev = self.status,
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/db/__init__.py", line 3597, in __get__
reference_id.to_path())
ReferencePropertyResolveError: ReferenceProperty failed to be resolved: [u'StatusLog', 248327L]
Thanks for any insight/help/answers/suggestions!
This happens when you attempt to resolve a reference property (by dereferencing it - for instance, (MyModel.MyReferenceProp.foo), and the property being referenced no longer exists - because it has been deleted.
You need to modify your code to catch this exception when you dereference an entity that may have been deleted, and handle it appropriately.