I am setting up an app in GAE. The app relies on up to three successive AJAX calls from the client that produces an individual string in each request. I would like to retain and combine the data on the server side in order to perform some more parsing on it.
I have been told that using global variables is really really bad, but I'm not sure how I would structure this flow without using globals to combine the data. Any suggestions? I don't want to do one single AJAX call because I would like to continue to update the user as to the progress. Abstractly it looks something like this-
finalData = ""
class func1(webapp2.RequestHandler):
def get(self):
func1data = some.command()
global finalData
finalData += func1data
class func2(webapp2.RequestHandler):
def get(self):
func2data = some.command()
global finalData
finalData += func2data
class func3(webapp2.RequestHandler):
def get(self):
func3data = some.command()
global finalData
finalData += func3data
It's a terrible idea, as in, it won't work. Or worse, it might appear to work sometimes when your requests happen to hit the same instance, but it won't work otherwise.
Store the data in an entity in the the datastore/memcache (if you use ndb it'll automatically be put in memcache), and use a get() to fetch the data on each successive request.
Alternatives are to use sessions, or return the data in a cookie to the client so it's sent back to the server with the next request (though that would insecurely allow the client to modify the value).
Related
I am adding new data into my Database by doing a POST-request on my eve-API.
Since there need to be added some data from the Python side I thought I could add these data by using a pre-request event hook.
So is there a way to modify the data contained in the POST-request using a pre-request hook before inserting the data into the database? I already understood how to implement such a hook but do not have any clue about how to modify data before inserting to DB.
You probably want to look at database hooks, specifically at insert hooks:
When a POST requests hits the API and new items are about to be stored in the database, these vents are fired:
on_insert for every resource endpoint.
on_insert_<resource_name> for the specific resource endpoint.
Callback functions could hook into these events to arbitrarily add new fields or edit existing ones.
In the code below:
def before_insert(resource_name, documents):
if resource_name == 'myresource':
for document in documents:
document['field'] = 'value'
app = Eve()
app.on_insert += before_insert
app.run()
Every time a POST hits the API the before_insert function is invoked. The function updates field1 for every document. Since this callback is invoked before the payload is sent to the database, changes will be persisted to the database.
An interesting alternative would be:
def before_insert(resource_name, documents):
for document in documents:
document['field'] = 'value'
app = Eve()
app.on_insert_myresource += before_insert
app.run()
In the callback we are not testing the endpoint name anymore. This is because we hooked our callback to the on_insert_myresoure event so the function will only be called when POST request are performed on the myresource endpoint. Better separation of concerns, code is simpler and also, improved performance since the callback is not going to be hit an all API inserts. Side note, eventually you can hook multiple callbacks to the same event (hence the use of the addition operator +=).
In my case I wanted to duplicate documents if a given property is in data.
I have to use pre_POST event hook to do that.
def pre_notifications(request):
data = json.loads(request.get_data())
if 'payload' in data and 'condition' in data:
notification = data['payload']
documents = []
users = app.data.pymongo().db.users.find()
for user in users:
copy_notification = copy(notification)
copy_notification['user_email'] = user['user_email']
documents.append(copy_notification)
request._cached_data = json.dumps(documents).encode('utf-8')
First, I tried to replace request.data but it does not work. Doing some search on code I figured out the _cached_data property. Then it works.
Just to complement the answer of #Gustavo (I cannot leave a comment in his answer). You can update the request._cached_json property without serializing your data.
Using his example:
def pre_notifications(request):
data = json.loads(request.get_data())
if 'payload' in data and 'condition' in data:
notification = data['payload']
documents = []
users = app.data.pymongo().db.users.find()
for user in users:
copy_notification = copy(notification)
copy_notification['user_email'] = user['user_email']
documents.append(copy_notification)
request._cached_json = documents
Let's say I have a class which stores important, dynamic data which I need to render my sites. This class should be created individually per user, and the data from the class needs to get updated according to some user input, so I have at least two views:
#app.route('/')
def index():
myclass = MyClass()
return render_template('index.html')
#app.route('/_update', methods=['GET'])
def update():
ret_data = {"value": request.args.get('c', type=float)}
a = myclass.calculate(ret_data['value'])
return jsonify(result=a)
Ofcourse it can't work this way, because myclass wouldn't exist in update() - so a working solution would be to make myclass global on creation. But this doesn't seem clean and it ruins the possibility for individual classes per session.
So, is there any clean way to access a class instance in different views, or how should I handle this, it doesn't feel like an uncommon scenario to me.
Secondly, I would like to have the class instance created for every user, but also closed when every a user closes his browser window etc. - I don't really get how to do this, I have a __del__() function in my class, but it won't be used if I set the instance to global.
Thank you very much!
You have a fundamental misunderstanding about how web applications work. They are stateless: nothing is shared between requests for any particular user. On the contrary, any global variable will be accessible to whichever user happens to hit the application next.
The way to store state for a user is to put it in the database. You can use a session cookie to associate an individual user with their particular data.
As Daniel Rosemann pointed out, it's probably not how one should design a web application. There is however a way to reach that functionality using global variables plus multiple instances. I don't know enough about python to estimate how wrong (or even dangerous) the use of global variables is, but it seems working for me - I'm happy about every comment on this solution:
Setting two global dicts, one to store the class instances, one for keep track if the instance is still relevant:
global session_instances, session_alive
session_instances = {}
session_alive = {}
In my root view I create a uuid and save the class instance with it in the dict and start a thread which should close my class after some time:
#app.route('/')
def index():
if not session.get('uid'):
session['uid'] = uuid.uuid4()
session_instances.update({session['uid'] : pyd2d.d2d()})
session_alive.update({session['uid'] : 0})
t = Thread(target=close_session, args = (session['uid'], ))
t.start()
return render_template('index.html')
The thread responsible for closing (e.g. 15 seconds after the last request):
def close_session(uid):
global session_alive
while True:
time.sleep(15)
if session_alive[uid] == 0:
session_instances[uid].stop()
break
session_alive[uid] = 0
And finally, to update the timer anytime a request is send:
#app.before_request
def before_request():
global session_alive
if session.get('uid'):
session_alive[session['uid']] = 1
This seems to work just fine. Should I feel bad about using global variables, or is it ok in cases like this? I appreciate any input!
my question is quite hard to describe, so I will focus on explaining the situation. So let's say I have 2 different entities, which may run on different machines. Let's call the first one Manager and the second one Generator. The manager is the only one which can be called via the user.
The manager has a method called getVM(scenario_Id), which takes the ID of a scenario as a parameter, and retrieve a BLOB from the database corresponding to the ID given as a parameter. This BLOB is actually a XML structure that I need to send to the Generator. Both have a Flask running.
On another machine, I have my generator with a generateVM() method, which will create a VM according to the XML structure it recieves. We will not talk about how the VM is created from the XML.
Currently I made this :
Manager
# This method will be called by the user
#app.route("/getVM/<int:scId>", methods=['GET'])
def getVM(scId):
xmlContent = db.getXML(scId) # So here is what I want to send
generatorAddr = sgAdd + "/generateVM" # sgAdd is declared in the Initialize() [IP of the Generator]
# Here how should I put my data ?
# How can I transmit xmlContent ?
testReturn = urlopen(generatorAddr).read()
return json.dumps(testReturn)
Generator
# This method will be called by the user
#app.route("/generateVM", methods=['POST'])
def generateVM():
# Retrieve the XML content...
return "Whatever"
So as you can see, I am stuck on how to transmit the data itself (the XML structure), and then how to treat it... So if you have any strategy, hint, tip, clue on how I should proceed, please feel free to answer. Maybe there are some things I do not really understand about Flask, so feel free to correct everything wrong I said.
Best regards and thank you
PS : Lines with routes are commented because they mess up the syntax coloration
unless i'm missing something couldn't you just transmit it in the body of a post request? Isn't that how your generateVM method is setup?
#app.route("/getVM/<int:scId>", methods=['GET'])
def getVM(scId):
xmlContent = db.getXML(scId)
generatorAddr = sgAdd + "/generateVM"
xml_str = some_method_to_generate_xml()
data_str = urllib.urlencode({'xml': xml_str})
urllib.urlopen(generatorAddr, data=data_str).read()
return json.dumps(testReturn)
http://docs.python.org/2/library/urllib.html#urllib.urlopen
I noticed a strange behaviour today: It seems that, in the following example, the config.CLIENT variable stays persistent accross requests – even if the view gets passed an entirely different client_key, the query that gets the client is only executed once (per many requests), and then the config.CLIENT variable stays assigned.
It does not seem to be a database caching issue.
It happens with mod_python as well as with the test server (the variable is reassigned when the test server is restarted).
What am I missing here?
#views.py
from my_app import config
def get_client(client_key=None):
if config.CLIENT == None:
config.CLIENT = get_object_or_404(Client, key__exact=client_key, is_active__exact=True)
return config.CLIENT
def some_view(request, client_key):
client = get_client(client_key)
...
return some_response
# config.py
CLIENT = None
Multiple requests are processed by the same process and global variables like your CLIENT live as long, as process does. You shouldn't rely on global variables, when processing requests - use either local ones, when you need to keep a variable for the time of building response or put data into the database, when something must persist across multiple requests.
If you need to keep some value through the request you can either add it to thread locals (here you should some examples, that adds user info to locals) or simply pass it as a variable into other functions.
OK, just to make it slightly clearer (and in response to the comment by Felix), I'm posting the code that does what I needed. The whole problem arose from a fundamental misunderstanding on my part and I'm sorry for any confusion I might have caused.
import config
# This will be called once per request/view
def init_client(client_key):
config.CLIENT = get_object_or_404(Client, key__exact=client_key, is_active__exact=True)
# This might be called from other modules that are unaware of requests, views etc
def get_client():
return config.CLIENT
I love CherryPy's API for sessions, except for one detail. Instead of saying cherrypy.session["spam"] I'd like to be able to just say session["spam"].
Unfortunately, I can't simply have a global from cherrypy import session in one of my modules, because the cherrypy.session object isn't created until the first time a page request is made. Is there some way to get CherryPy to initialize its session object immediately instead of on the first page request?
I have two ugly alternatives if the answer is no:
First, I can do something like this
def import_session():
global session
while not hasattr(cherrypy, "session"):
sleep(0.1)
session = cherrypy.session
Thread(target=import_session).start()
This feels like a big kludge, but I really hate writing cherrypy.session["spam"] every time, so to me it's worth it.
My second solution is to do something like
class SessionKludge:
def __getitem__(self, name):
return cherrypy.session[name]
def __setitem__(self, name, val):
cherrypy.session[name] = val
session = SessionKludge()
but this feels like an even bigger kludge and I'd need to do more work to implement the other dictionary functions such as .get
So I'd definitely prefer a simple way to initialize the object myself. Does anyone know how to do this?
For CherryPy 3.1, you would need to find the right subclass of Session, run its 'setup' classmethod, and then set cherrypy.session to a ThreadLocalProxy. That all happens in cherrypy.lib.sessions.init, in the following chunks:
# Find the storage class and call setup (first time only).
storage_class = storage_type.title() + 'Session'
storage_class = globals()[storage_class]
if not hasattr(cherrypy, "session"):
if hasattr(storage_class, "setup"):
storage_class.setup(**kwargs)
# Create cherrypy.session which will proxy to cherrypy.serving.session
if not hasattr(cherrypy, "session"):
cherrypy.session = cherrypy._ThreadLocalProxy('session')
Reducing (replace FileSession with the subclass you want):
FileSession.setup(**kwargs)
cherrypy.session = cherrypy._ThreadLocalProxy('session')
The "kwargs" consist of "timeout", "clean_freq", and any subclass-specific entries from tools.sessions.* config.