I have a 'Calculation' entity in my datastore with say, 7 million objects under that entity. It has the following properties (python runtime):
class Calculation(db.Model):
question = db.StringProperty(required=True)
answer = db.StringProperty()
suppose examples of the 'question' property are things like '1+1', '2+2', '3+3' (not really important). All the calculation objects start out with empty answer properties.
When a user connects to the app, an ajax call is made and my app is supposed to [1] fetch a Calculation object with an empty answer property and send it to the user's browser. The user's browser then evaluates the question and sends it back to a different server handler.
How do I update that specific Calculation object's answer property[2]?
If someone could provide me with code for [1] and [2] that would be great. Not really experienced with App Engine and the query stuff is confusing. What is the best solution for this if I want to conserve as much server-CPU as possible?
Thanks!
I don't know if i have really understand. But you simply need to return for your first ajax call the key of the entity Calculation and the question. When the user makes a response you firstly get the entity by the key and update the property answer.
Step 1, The ajax call return a question in JSON (for example):
# To fetch an empty answered question
qry = Calculation.All().filter('answer =', None)
ref = qry.get()
# The Json response
{ 'key': unicode(ref.key()),
'question': ref.question}
Step 2, You update the entity by the key:
# key and answer are variable from an http GET or POST request.
ref = db.get(key)
ref.answer = answer
ref.put()
Related
I am trying to make use of a column's value as a radio button's choice using below code
Forms.py
#retreiving data from database and assigning it to diction list
diction = polls_datum.objects.values_list('poll_choices', flat=True)
#initializing list and dictionary
OPTIONS1 = {}
OPTIONS = []
#creating the dictionary with 0 to no of options given in list
for i in range(len(diction)):
OPTIONS1[i] = diction[i]
#creating tuples from the dictionary above
#OPTIONS = zip(OPTIONS1.keys(), OPTIONS1.values())
for i in OPTIONS1:
k = (i,OPTIONS1[i])
OPTIONS.append(k)
class polls_form(forms.ModelForm):
#retreiving data from database and assigning it to diction list
options = forms.ChoiceField(choices=OPTIONS, widget = forms.RadioSelect())
class Meta:
model = polls_model
fields = ['options']
Using a form I am saving the data or choices in a field (poll_choices), when trying to display it on the index page, it is not reflecting until a server restart.
Can someone help on this please
of course "it is not reflecting until a server restart" - that's obvious when you remember that django server processes are long-running processes (it's not like PHP where each script is executed afresh on each request), and that top-level code (code that's at the module's top-level, not in a function) is only executed once per process when the module is first imported. As a general rule: don't do ANY db query at a module's top-level or at the top-level of a class statement - at best you'll get stale data, at worse it will crash your server process (if you're doing query before everything has been properly setup by django, or if you're doing query based on a schema update before the migration has been applied).
The possible solutions are either to wait until the form's initialisation to setup your field's choices, or to pass a callable as the formfield's choices options, cf https://docs.djangoproject.com/en/2.1/ref/forms/fields/#django.forms.ChoiceField.choices
Also, the way you're building your choices list is uselessly complicated - you could do it as a one-liner:
OPTIONS = list(enumerate(polls_datum.objects.values_list('poll_choices', flat=True))
but it's also very brittle - you're relying on the current db content and ordering for the choice value when you should use the polls_datum's pk instead (which is garanteed to be stable).
And finally: since you're working with what seems to be a related model, you may want to use a ModelChoiceField instead.
For future reference:
What version of Django are you using?
Have you read up on the documentation of ModelForms? https://docs.djangoproject.com/en/2.1/topics/forms/modelforms/
I'm not sure what you're trying to do with diction to dictionary to tuple. I think you could skip a step there and your future self will thank you for that.
Try to follow some tutorials and understand why certain steps are being taken. I can see from your code that you're rather new to coding or Python and there's room for improvement. Not trying to talk you down, but I'm trying to push you into the direction of becoming a better developer ;-)
REAL ANSWER:
That being said, I think the solution is to write the loading of the data somewhere in your form model, rather than 'loose' in forms.py. See bruno's answer for more information on this.
If you want to reload the data on each request that loads the form, you should create a function that gets called every time the form is loaded (for example in the form's __init__ function).
I am adding new data into my Database by doing a POST-request on my eve-API.
Since there need to be added some data from the Python side I thought I could add these data by using a pre-request event hook.
So is there a way to modify the data contained in the POST-request using a pre-request hook before inserting the data into the database? I already understood how to implement such a hook but do not have any clue about how to modify data before inserting to DB.
You probably want to look at database hooks, specifically at insert hooks:
When a POST requests hits the API and new items are about to be stored in the database, these vents are fired:
on_insert for every resource endpoint.
on_insert_<resource_name> for the specific resource endpoint.
Callback functions could hook into these events to arbitrarily add new fields or edit existing ones.
In the code below:
def before_insert(resource_name, documents):
if resource_name == 'myresource':
for document in documents:
document['field'] = 'value'
app = Eve()
app.on_insert += before_insert
app.run()
Every time a POST hits the API the before_insert function is invoked. The function updates field1 for every document. Since this callback is invoked before the payload is sent to the database, changes will be persisted to the database.
An interesting alternative would be:
def before_insert(resource_name, documents):
for document in documents:
document['field'] = 'value'
app = Eve()
app.on_insert_myresource += before_insert
app.run()
In the callback we are not testing the endpoint name anymore. This is because we hooked our callback to the on_insert_myresoure event so the function will only be called when POST request are performed on the myresource endpoint. Better separation of concerns, code is simpler and also, improved performance since the callback is not going to be hit an all API inserts. Side note, eventually you can hook multiple callbacks to the same event (hence the use of the addition operator +=).
In my case I wanted to duplicate documents if a given property is in data.
I have to use pre_POST event hook to do that.
def pre_notifications(request):
data = json.loads(request.get_data())
if 'payload' in data and 'condition' in data:
notification = data['payload']
documents = []
users = app.data.pymongo().db.users.find()
for user in users:
copy_notification = copy(notification)
copy_notification['user_email'] = user['user_email']
documents.append(copy_notification)
request._cached_data = json.dumps(documents).encode('utf-8')
First, I tried to replace request.data but it does not work. Doing some search on code I figured out the _cached_data property. Then it works.
Just to complement the answer of #Gustavo (I cannot leave a comment in his answer). You can update the request._cached_json property without serializing your data.
Using his example:
def pre_notifications(request):
data = json.loads(request.get_data())
if 'payload' in data and 'condition' in data:
notification = data['payload']
documents = []
users = app.data.pymongo().db.users.find()
for user in users:
copy_notification = copy(notification)
copy_notification['user_email'] = user['user_email']
documents.append(copy_notification)
request._cached_json = documents
Sorry for noobster question again.
But I'm trying to do some very easy stuff here, and I don't know how. Documentation gives me hints which do not work, or apply.
I recieve a POST request and grab a variable out of it. It says "name".
I have to search all over my entities Object (for example) and find out if there's one that has the same name. Is there's none, I must create a new Entity with this name. Easy it may look, but I keep Failing.
Would really appreciate any help.
My code currently is this one:
objects_qry = Object.query(Object.name == data["name"])
if (not objects_qry ):
obj = Object()
obj .name = data["name"]
obj .put()
class Object(ndb.Model):
name = ndb.StringProperty()
Using a query to perform this operation is really inefficient.
In addition your code is possibly unreliable, if name doesn't exist and you have two requests at the same time for name you could end up with two records. And you can't tell because your query only returns the first entity with the name property equal to some value.
Because you expect only one entity for name a query is expensive and inefficient.
So you have two choices you can use get_or_insert or just do a get, and if you have now value create a new entity.
Any way here is a couple of code samples using the name as part of the key.
name = data['name']
entity = Object.get_or_insert(name)
or
entity = Object.get_by_id(name)
if not entity:
entity = Object(id=name)
entity.put()
Calling .query just creates a query object, it doesn't execute it, so trying to evaluate is as a boolean is wrong. Query object have methods, fetch and get that, respectively, return a list of matching entities, or just one entity.
So your code could be re-written:
objects_qry = Object.query(Object.name == data["name"])
existing_object = objects_qry.get()
if not existing_object:
obj = Object()
obj.name = data["name"]
obj.put()
That said, Tim's point in the comments about using the ID instead of a property makes sense if you really care about names being unique - the code above wouldn't stop two simultaneous requests from creating entities with the same name.
my question is quite hard to describe, so I will focus on explaining the situation. So let's say I have 2 different entities, which may run on different machines. Let's call the first one Manager and the second one Generator. The manager is the only one which can be called via the user.
The manager has a method called getVM(scenario_Id), which takes the ID of a scenario as a parameter, and retrieve a BLOB from the database corresponding to the ID given as a parameter. This BLOB is actually a XML structure that I need to send to the Generator. Both have a Flask running.
On another machine, I have my generator with a generateVM() method, which will create a VM according to the XML structure it recieves. We will not talk about how the VM is created from the XML.
Currently I made this :
Manager
# This method will be called by the user
#app.route("/getVM/<int:scId>", methods=['GET'])
def getVM(scId):
xmlContent = db.getXML(scId) # So here is what I want to send
generatorAddr = sgAdd + "/generateVM" # sgAdd is declared in the Initialize() [IP of the Generator]
# Here how should I put my data ?
# How can I transmit xmlContent ?
testReturn = urlopen(generatorAddr).read()
return json.dumps(testReturn)
Generator
# This method will be called by the user
#app.route("/generateVM", methods=['POST'])
def generateVM():
# Retrieve the XML content...
return "Whatever"
So as you can see, I am stuck on how to transmit the data itself (the XML structure), and then how to treat it... So if you have any strategy, hint, tip, clue on how I should proceed, please feel free to answer. Maybe there are some things I do not really understand about Flask, so feel free to correct everything wrong I said.
Best regards and thank you
PS : Lines with routes are commented because they mess up the syntax coloration
unless i'm missing something couldn't you just transmit it in the body of a post request? Isn't that how your generateVM method is setup?
#app.route("/getVM/<int:scId>", methods=['GET'])
def getVM(scId):
xmlContent = db.getXML(scId)
generatorAddr = sgAdd + "/generateVM"
xml_str = some_method_to_generate_xml()
data_str = urllib.urlencode({'xml': xml_str})
urllib.urlopen(generatorAddr, data=data_str).read()
return json.dumps(testReturn)
http://docs.python.org/2/library/urllib.html#urllib.urlopen
I've been struggling for this issue for a few hours - I know there's probably a simple solution that I'm overlooking.
I have a one to many relationship with my models.
I have need to return all rows of one object with the rows for the related object.
In a sense I have this:
object
object
object_relationship.property
object_relationship.property
object
object_relationship.property
object
Now - I can run through all of these fine, but I run into an issue when I want to send these back to the html template.
I can send the object back - but how do I send the object_relationship back in the order that I have it above?
Does this make sense?
You might not need to worry too much about this, acutally... look at these models:
class Venue(base.NamedEntity, HasPerformances, HasUrl, HasLocation):
city = db.ReferenceProperty(City, collection_name='venues')
url = db.StringProperty(required=True, validator=validators.validate_url)
location = db.GeoPtProperty()
class Performance(base.Entity):
show = db.ReferenceProperty(Show, collection_name='performances', required=True)
utc_date_time = db.DateTimeProperty(required=True)
venue = db.ReferenceProperty(Venue, collection_name='performances', required=True)
In a case like this, nothing stops you from using venue.performances from either code or templates and treating it as a list. The API will automatically fire queries as needed to fetch the actual objects. The same thing goes for performance.venue.
The only problem here is performance - you've got a variant of the n+1 problem to deal with. There are workarounds, though, like this article by Nick Johnson. I'd suggest reading the API code too... it makes for interesting reading how the property get is captured and dereferenced.
My first suggestion is to denormalize the data if you are going to do many reports like that. For example, maybe you could include object.name on the object_relationship entity.
That said, you could send a list of dicts to your template, so maybe something like:
data = []
for entity in your_query:
children = [{'name': child.name} for child in entity.object_relation]
data.append({'name': object.name,
'children': children,
...
})
Then pass the data list to your template, and process it.
Please note, this will perform very badly. It will execute another query for every one of the items in your first query. Use Appstats to profile your app.