This is my GAE datastore:
class Search(ndb.Model):
city = ndb.StringProperty()
counter = ndb.IntegerProperty(indexed = True)
date = ndb.DateTimeProperty(auto_now_add=True)
When I run this part of code:
keys = Search.query(Search.city == city).fetch()
if (len(keys)==0):
luogo = Search(city = city, counter = 1)
luogo.put()
else:
for key in keys:
luogo_1 = key.get()
luogo_1.counter = luogo_1.counter+1
luogo_1.put()
my terminal says that is missing in object Search attribute get
Do you know why?
Running
keys = Search.query(Search.city == city).fetch()
fetches a list of model instances
so
for key in keys:
luogo_1 = key.get()
fails because instances don't have a get method.
You need to do:
keys = Search.query(Search.city == city).fetch(keys_only=True)
to fetch a list of keys, or treat keys as a list of instances rather than keys, and omit the key.get() call.
Maybe because key object doesn't have get() method:
luogo_1 = key.get()
Use python dir() function, it helps me a lot
Related
I have a class `Collection' that looks like this:
class Collection():
def __init__(self, db, collection_name):
self.db = db
self.collection_name = collection_name
if not hasattr(self.__class__, 'client'):
self.__class__.client = MongoClient()
self.data_base = getattr(self.client, self.db)
self.collection = getattr(self.data_base, self.collection_name)
def getCollectionKeys(self):
....etc.
I cleverly created a function to create a dictionary of class instances as follows:
def getCollections():
collections_dict = {}
for i in range(len(db_collection_names)):
collections_dict[db_collection_names[i]] = Collection(database_name, db_collection_names[i])
return collections_dict
it works. however, whenever I want to access a class instance, I have to go through the dictionary:
agents_keys = collections_dict['agents'].getCollectionKeys()
I would love to just write:
agents_keys = agents.getCollectionKeys()
Is there a simple way to get those instances "out" of the dict?
You can get a reference to items in a vanilla python dictionary using a generator object in a for loop, or by using a list expression.
agent_keys = [x.getCollectionKeys() for x in collections_dict.values()]
or this
agent_keys = []
for name in db_collection_names:
#do something with individual item
#there could also be some logic here about which keys to append
agent_keys.append(collections_dict[name].getCollectionKeys())
#now agent_keys is full of all the keys
My mental model of how objects are interacted with in python. Feel free to edit if you actually know how it works.
You cannot "take" items of the dictionary per say unless you call the del operator which removes the association of a variable name (that is what you type in the editor like "foo" and "bar") with an object ( the actual collections of bits in the program your machine sees). What you can do is get a reference to the object, which in python is a symbol that for all your intents and purposes is the object you want.
The dictionary just holds a bunch of references to your database objects.
The expression collections_dict['agents'] is equivalent to your original database object that you put into the dictionary like this
collections_dict['agents'] = my_particular_object
From google documentation:
"A model instance's key includes the instance's entity kind along with a unique identifier. The identifier may be either a key name string, assigned explicitly by the application when the instance is created, or an integer numeric ID, assigned automatically by App Engine when the instance is written (put) to the Datastore. "
so in the example:
name = "John"
idd = 11
person = Person(name, idd)
person.put()
How do i get the "integer numeric ID, assigned automatically by App Engine"?
if you are using ndb put() returns the new key... call the id function on the key:
name = "John"
idd = 11
person = Person(name, idd)
new_key = person.put()
auto_assigned_id = new_key.id()
from https://developers.google.com/appengine/docs/python/ndb/entities :
To store the object as a persistent entity in the Datastore, use the
put() method. This returns a key for retrieving the entity from the
Datastore later:
sandy_key = sandy.put()
and:
https://developers.google.com/appengine/docs/python/ndb/keyclass#Key_id
Have you tried the
print person.id()
or if you have provided the unique identifuer
print person.id_or_name()
Also the put() method returns the key
key = person.put()
I'm using the bottle framework together with mongoengine.
I have an orders model :
class OrderDetail(Option):
orderDetailsQty = FloatField()
def to_dict(self):
return mongo_to_dict_helper(self)
class Order(Document):
userName = StringField(required=True)
orderDate = DateTimeField()
orderStatus = ListField(EmbeddedDocumentField(Status))
orderDetails = ListField(EmbeddedDocumentField(OrderDetail))
orderComments = ListField(EmbeddedDocumentField(Comment))
isActive = BooleanField()
def to_dict(self):
orderObj = mongo_to_dict_helper(self)
orderDetailList = []
for orderDetail in orderObj["orderDetails"]:
orderDetailList.append(orderDetail.__dict__)
orderObj["OrderDetails"] = orderDetailList
return (self)
When mongodb is queried I get an object which is then converted in to a dict by using the following function :
def mongo_to_dict_helper(obj):
return_data = []
for field_name in obj._fields:
if field_name in ("id",):
continue
data = obj._data[field_name]
if isinstance(obj._fields[field_name], StringField):
return_data.append((field_name, str(data)))
elif isinstance(obj._fields[field_name], FloatField):
return_data.append((field_name, float(data)))
elif isinstance(obj._fields[field_name], IntField):
return_data.append((field_name, int(data)))
elif isinstance(obj._fields[field_name], ListField):
return_data.append((field_name, int(data)))
else:
# You can define your logic for returning elements
pass
return dict(return_data)
I found this function after a long search in the internet. Later found out that this function also fails while defining a member as the ListField(EmbeddedDocumentField(obj)).
I also tried writing a condition for catching the specific case of EmbeddedDocumentField :
elif isinstance(obj._fields[field_name], EmbeddedDocumentField):
return_data.append(mongo_to_dict_helper(data))
but that didn't do any good either.
Anyone have a workaround for this issue ?
What about just using to_mongo method of an object to convert it to a dict?
object.to_mongo()
Expanding on #alexvassel's and #z0r's answers, calling .to_mongo() converts the object to a SON instance. Once you have it, you can call its .to_dict() method to convert it to a dictionary.
For example... (qset is a queryset that's returned from mongoengine, after e.g. Posts.objects.all()).
sons = [ob.to_mongo() for ob in qset]
for son in sons:
print str(son.to_dict())
import json
json.loads(yourobj.to_json())
Extending on #alexvassel's answer, to_mongo() method returns SON object, which you can convert to dict by calling its to_dict() method
object.to_mongo().to_dict()
you can custom method to convert object to dict
class Order(Document):
userName = StringField(required=True)
orderDate = DateTimeField()
orderStatus = ListField(EmbeddedDocumentField(Status))
orderDetails = ListField(EmbeddedDocumentField(OrderDetail))
orderComments = ListField(EmbeddedDocumentField(Comment))
isActive = BooleanField()
def as_dict(self):
return {
"user_name": self.userName,
"order_date": self.orderDate.strftime("%Y-%m-%d %H:%M:%S"),
}
now you can use obj.as_dict() to dict
orders = Order.objects.all()
datas = [each.as_dict() for each in orders]
combining all other answers,
import json
dict = {'data':[json.loads(ob.to_json()) for ob in qset]}
There can be two scenario.
when query returns CommandCursor object
**records = list(CursorObject)**
ex - Class.objects().aggregate({...})
when query returns BaseQuerySet object
**import json**
**records = json.loads(BaseQuerySetObject.to_json())**
ex - Class.objects().filter(..)
I have a set of objects stored in a variable called subs. The column subscribed_to is a foreign object.
Is there any way to do check if a related_object is in this list in a simpler way:
def check_subscription_status(user, related_object):
subs = get_user_subscriptions(user) # returns filter queryset
subscribed = False
for s in subs:
if s.subscribed_to == related_object: #related object is the potential match
subscribed = True
break
return subscribed
return (related_object in [s.subscribed_to for s in subs])
ETA: A better way:
return subs.filter(subscribed_to=related_object).count() > 0
In a Python Google App Engine app I'm writing, I have an entity stored in the datastore that I need to retrieve, make an exact copy of it (with the exception of the key), and then put this entity back in.
How should I do this? In particular, are there any caveats or tricks I need to be aware of when doing this so that I get a copy of the sort I expect and not something else.
ETA: Well, I tried it out and I did run into problems. I would like to make my copy in such a way that I don't have to know the names of the properties when I write the code. My thinking was to do this:
#theThing = a particular entity we pull from the datastore with model Thing
copyThing = Thing(user = user)
for thingProperty in theThing.properties():
copyThing.__setattr__(thingProperty[0], thingProperty[1])
This executes without any errors... until I try to pull copyThing from the datastore, at which point I discover that all of the properties are set to None (with the exception of the user and key, obviously). So clearly this code is doing something, since it's replacing the defaults with None (all of the properties have a default value set), but not at all what I want. Suggestions?
Here you go:
def clone_entity(e, **extra_args):
"""Clones an entity, adding or overriding constructor attributes.
The cloned entity will have exactly the same property values as the original
entity, except where overridden. By default it will have no parent entity or
key name, unless supplied.
Args:
e: The entity to clone
extra_args: Keyword arguments to override from the cloned entity and pass
to the constructor.
Returns:
A cloned, possibly modified, copy of entity e.
"""
klass = e.__class__
props = dict((k, v.__get__(e, klass)) for k, v in klass.properties().iteritems())
props.update(extra_args)
return klass(**props)
Example usage:
b = clone_entity(a)
c = clone_entity(a, key_name='foo')
d = clone_entity(a, parent=a.key().parent())
EDIT: Changes if using NDB
Combining Gus' comment below with a fix for properties that specify a different datastore name, the following code works for NDB:
def clone_entity(e, **extra_args):
klass = e.__class__
props = dict((v._code_name, v.__get__(e, klass)) for v in klass._properties.itervalues() if type(v) is not ndb.ComputedProperty)
props.update(extra_args)
return klass(**props)
Example usage (note key_name becomes id in NDB):
b = clone_entity(a, id='new_id_here')
Side note: see the use of _code_name to get the Python-friendly property name. Without this, a property like name = ndb.StringProperty('n') would cause the model constructor to raise an AttributeError: type object 'foo' has no attribute 'n'.
If you're using the NDB you can simply copy with:
new_entity.populate(**old_entity.to_dict())
This is just an extension to Nick Johnson's excellent code to address the problems highlighted by Amir in the comments:
The db.Key value of the ReferenceProperty is no longer retrieved via an unnecessary roundtrip to the datastore.
You can now specify whether you want to skip DateTime properties with the auto_now and/or auto_now_add flag.
Here's the updated code:
def clone_entity(e, skip_auto_now=False, skip_auto_now_add=False, **extra_args):
"""Clones an entity, adding or overriding constructor attributes.
The cloned entity will have exactly the same property values as the original
entity, except where overridden. By default it will have no parent entity or
key name, unless supplied.
Args:
e: The entity to clone
skip_auto_now: If True then all DateTimeProperty propertes will be skipped which have the 'auto_now' flag set to True
skip_auto_now_add: If True then all DateTimeProperty propertes will be skipped which have the 'auto_now_add' flag set to True
extra_args: Keyword arguments to override from the cloned entity and pass
to the constructor.
Returns:
A cloned, possibly modified, copy of entity e.
"""
klass = e.__class__
props = {}
for k, v in klass.properties().iteritems():
if not (type(v) == db.DateTimeProperty and ((skip_auto_now and getattr(v, 'auto_now')) or (skip_auto_now_add and getattr(v, 'auto_now_add')))):
if type(v) == db.ReferenceProperty:
value = getattr(klass, k).get_value_for_datastore(e)
else:
value = v.__get__(e, klass)
props[k] = value
props.update(extra_args)
return klass(**props)
The first if expression is not very elegant so I appreciate if you can share a better way to write it.
I'm neither Python nor AppEngine guru, but couldn't one dynamically get/set the properties?
props = {}
for p in Thing.properties():
props[p] = getattr(old_thing, p)
new_thing = Thing(**props).put()
A variation inspired in Nick's answer which handles the case in which your entity has a (repeated) StructuredProperty, where the StructuredProperty itself has ComputedProperties. It can probably be written more tersely with dict comprehension somehow, but here is the longer version that worked for me:
def removeComputedProps(klass,oldDicc):
dicc = {}
for key,propertType in klass._properties.iteritems():
if type(propertType) is ndb.StructuredProperty:
purged = []
for item in oldDicc[key]:
purged.append(removeComputedProps(propertType._modelclass,item))
dicc[key]=purged
else:
if type(propertType) is not ndb.ComputedProperty:
dicc[key] = oldDicc[key]
return dicc
def cloneEntity(entity):
oldDicc = entity.to_dict()
klass = entity.__class__
dicc = removeComputedProps(klass,oldDicc)
return klass(**dicc)
This can be tricky if you've renamed the underlying keys for your properties... which some people opt to do instead of making mass data changes
say you started with this:
class Person(ndb.Model):
fname = ndb.StringProperty()
lname = ndb.StringProperty()
then one day you really decided that it would be nicer to use first_name and last_name instead... so you do this:
class Person(ndb.Model):
first_name = ndb.StringProperty(name="fname")
last_name = ndb.StringProperty(name="lname")
now when you do Person._properties (or .properties() or person_instance._properties) you will get a dictionary with keys that match the underlying names (fname and lname)... but won't match the actual property names on the class... so it won't work if you put them into the constructor of a new instance, or use the .populate() method (the above examples will break)
In NDB anyways, instances of models have ._values dictionary which is keyed by the underlying property names... and you can update it directly. I ended up with something like this:
def clone(entity, **extra_args):
klass = entity.__class__
clone = klass(**extra_args)
original_values = dict((k,v) for k,v in entity._values.iteritems() if k not in clone._values)
clone._values.update(original_values)
return clone
This isn't really the safest way... as there are other private helper methods that do more work (like validation and conversion of computed properties by using _store_value() and _retrieve_value())... but if you're models are simple enough, and you like living on the edge :)
Here's the code provided by #zengabor with the if expression formatted for easier reading. It may not be PEP-8 compliant:
klass = e.__class__
props = {}
for k, v in klass.properties().iteritems():
if not (type(v) == db.DateTimeProperty and ((
skip_auto_now and getattr(v, 'auto_now' )) or (
skip_auto_now_add and getattr(v, 'auto_now_add')))):
if type(v) == db.ReferenceProperty:
value = getattr(klass, k).get_value_for_datastore(e)
else:
value = v.__get__(e, klass)
props[k] = value
props.update(extra_args)
return klass(**props)