Mongokit validate dict inside list - python

How can I validate that my desc field is required and my category field is optional?
class Mydoc(Document):
structure = {
"name": unicode,
"items": [{
"category": int,
"desc": unicode
}]
}
required_fields = ["name", "items", "items.desc"] # Error: items has no attribute
# desc, it is a list not a dict.
How can I validate the categories inside the list?
UPDATE
https://groups.google.com/forum/?fromgroups=#!topic/mongokit/GP5AgaMG6T4

The tricky point here is that we do not know how many items there are. Mongokit doesn't allow you to specify a nested object as required because it could potentially be very slow if you have many items.
So, in short, mongokit doesn't allow required_fields and default_values in nested objects.
However, Mongokit is very light and can be customized very easily if needed:
class MyDoc(Document):
structure = {
"name": unicode,
"items": [{
"category": int,
"desc": unicode
}]
}
def validate(self, *args, **kwargs):
super(MyDoc, self).validate(*args, **kwars)
for item in self["items"]:
assert item["desc"], "desc is required: %s" % item

Related

marshmallow - choose nested schema based on field value and handle fields 'starting with'

I am parsing a file and end up with a dictionary like this:
user_data = {
"title": "TestA",
"sects": [
{
"type": "cmd",
"cmd0": {
"moveX": 12,
"moveY": 34,
"action": "fire"
},
"session": 9999,
"cmd1": {
"moveX": 56,
"moveY": 78,
"action": "stop"
},
"endType": 0,
},
{
"type": "addUsers",
"user1": {
"name": "John",
"city": "London"
},
"user2": {
"name": "Mary",
"city": "New York"
},
post = "yes"
}
]
}
I am attempting to validate it using marshmallow but not sure how to handle these two things:
With sects the content of each nested dict is dependent on the type (cmd, addUser, etc). Is there a way to pick a schema based on the value of a field?
Is there such a thing as field 'startsWith' to handle the fact that I may have cmd0, cmd1...cmdN?
So, something like the following:
class CmdSchema(schema):
type = fields.Str()
cmdN = fields.Dict(...) # Allow cmd1, cmd2, etc
session = fields.Int()
endType = fields.Int()
class AddUsersSchema(schema):
type = fields.Str()
userN = fields.Dict(...) # Allow user1, user2, etc
post = fields.Str()
class ParsedFileSchema(schema):
title = fields.Str()
sects = fields.Nested(...) # Choose nested schema based on sects[i]['type']?
Note: I cannot change the format/content of the file I'm parsing and order matters (so I need to retain the order of cmd1, cmd2, etc so they can be processed in the correct order).
1/ What you want to achieve is polymorphism. You may want to check out marshmallow-oneofschema.
2/ This does not exist to my knowledge. It shouldn't be too hard to create as a custom validator, though. It might even make it to the core.

convert list of class objects into json in python or django

my_objects = []
my_objects.append(Needed("bye",9))
my_objects.append(Needed("tata",8))
my_objects.append(Needed("hi",10))
i have list of object(example 5 objects in list) like this
class Needed:
def __init__(self, name, number):
self.name = name
self.number = number
and i need to convert this into json order by count like below
{
"results":[
{ "name":"hi",
"number":"10"
},
{ "name":"bye",
"number":"9"
},
{ "name":"tata",
"number":"8"
},
...........
...........
]
}
so how to achieve this in django
First you need to make your objects json-serializable. You could provide a default encoder function or write your own JSONEncoder as mentionned in the FineManual, but for a simple one-off case like your example building dicts from your objects is probably the simplest solution:
class Needed(object):
def __init__(self, name, number):
self.name = name
self.number = number
def to_dict(self):
return {"name": self.name, "number": self.number}
Then build a list of dicts from your objects:
results = [obj.to_dict() for obj in my_objects]
sort it based on number:
results.sort(key=lambda obj: obj["number"])
and serialize the whole thing:
jsdata = json.dumps({"results": results})
This is pretty ordinary Python stuff - building dictionaries out of objects, making a list of them and sorting the list.
Beside the fact that this is not django related, here is some Python code that can help you:
from operator import attrgetter
# sort objects by attribute number in reverse order
result = {'results': [obj.__dict__ for obj in sorted(my_objects, key=attrgetter('number'), reverse=True)]}
# `indent=4` returns a json string "prettyfied"
print(json.dumps(result, indent=4))
Which prints:
{
"results": [
{
"name": "hi",
"number": 10
},
{
"name": "bye",
"number": 9
},
{
"name": "tata",
"number": 8
}
...
]
}
Hope it helps.

Adding additional data to django rest framework viewset response

Curently I have a viewset of an example Warehouse and I want to pass additional 'filter' list to each dictionary that is returned.
My WarehouseViewSet:
class WarehouseViewSet(viewsets.ReadOnlyModelViewSet):
filters = [{'date': 'Date/Time'}]
queryset = Warehouse.objects.all()
serializer_class = WarehouseSerializer
WarehouseSerializer:
class WarehouseSerializer(serializers.ModelSerializer):
class Meta:
model = Warehouse
field = ('name', 'address', 'action_list')
Currently I get an json list response like:
[
{
"id": 1,
"name": "Brameda Warehouse",
"address": "Bergijk"
},
{
"id": 2,
"name": "Amazon",
"address": "UK"
}
]
I would like to get:
[
{
"id": 1,
"name": "Brameda Warehouse",
"address": "Bergijk"
"filters": [
{'date': 'dateTime'}, {'actove': 'ActiveObject'}
]
},
{
"id": 2,
"name": "Amazon",
"address": "UK",
"filters": [
{'date': 'dateTime'}, {'actove': 'ActiveObject'}
]
}
]
I understand that having one filter is enough outside the objects dictionary, but I would like to know how to pass lists inside objects.
Any ideas how I can pass additional lists that would be returned as json object would be appreaciated.
I feel a bit unclear as to what you want, but if you just want to add some read-only computed field to the output, you can use SerializerMethodField:
class WarehouseSerializer(serializers.ModelSerializer):
# your other declared fields here
filters = serializers.SerializerMethodField()
# your Meta options here
def get_filters(self, obj):
return ['some', 'stuff', 'here', {'and': 'more'}]
The method has to be named get_field_name (there is an option to change it but I don't really see any point using it).
You get the instance being serialized as obj.
You may return anything that is made of regular types (numbers, strings, dicts, lists, tuples, booleans, None).
If the data has to come from outside, you should have the caller add it to the context, and it will be available on self.context['foobar'].

Can't iterate over my own object

I am new to Python and can't figure this out. I am trying to make an object from a json feed. I am trying to basically make a dictionary for each item in the json fed that has every property. The error I get is either TypeError: 'mediaObj' object is not subscriptable or not iterable
For bonus points, the array has many sub dictionaries too. What I would like is to be able to access that nested data as well.
Here is my code:
url = jsonfeedwithalotofdata.com
data = urllib.request.urlopen(url)
data = instagramData.read()
data = instagramData.decode('utf-8')
data = json.loads(data)
media = data['data']
class mediaObj:
def __init__(self, item):
for key in item:
setattr(self, key, item[key])
print(self[key])
def run(self):
return self['id']
for item in media:
mediaPiece = mediaObj(item)
This would come from a json feed that looks as follows (so data is the array that comes after "data"):
"data": [
{
"attribution": null,
"videos": {},
"tags": [],
"type": "video",
"location": null,
"comments": {},
"filter": "Normal",
"created_time": "1407423448461",
"link": "http://instagram.com/p/rabdfdIw9L7D-/",
"likes": {},
"images": {},
"users_in_photo": [],
"caption": {},
"user_has_liked": true,
"id": "782056834879232959294_1051813051",
"user": {}
}
So my hope was that I could create an object for every item in the array, and then I could, for instance, say:
print(mediaPiece['id'])
or even better
print(mediaPiece['comments'])
And see a list of comments. Thanks a million
You're having a problem because you're using attributes to store your data items, but using list/dictionary lookup syntax to try to retrieve them.
Instead of print(self[key]), use print(getattr(self, key)), and instead of return self['id'] use return self.id.

Django/python querysets - adding another "child" queryset as an item to each list object?

Apologies if the answer to this is obvious - I'm very new to django/python & haven't been able to find a solution in my searching so far.
I have a straightforward queryset, eg
members = LibraryMembers.objects.all()
with this I can do:-
for m in members:
member_books = LibraryBorrows.objects.filter(member_id=m[u'id'])
What I really want though is to be able to serialize the results into json, so it looks something like this:-
{
"members":
[
{
"id" : "1",
"name" : "Joe Bloggs"
"books":
[
{
"name" : "Five Go Exploring",
"author" : "Enid Blyton",
},
{
"name" : "Princess of Mars",
"author" : "Edgar Rice Burroughs",
},
]
}
]
}
To my mind, the obvious thing to try was:-
for m in members:
m[u'books'] = LibraryBorrows.objects.filter(member_id=m[u'id'])
However I'm getting TypeError: 'LibraryBorrows' object does not support item assignment
Is there any way to achieve what I'm after?
Model instances are not indeed not dicts. Now if you want dicts instead of model instances, then Queryset.values() is your friend - you get a list of dicts with only the required fields, and you avoid the overhead of retrieving unneeded fields from the database and building full-blown model instances.
>> members = LibraryMember.objects.values("id", "name")
>> print members
[{"id" : 1, "name" : "Joe Bloggs"},]
Then you code would look like:
members = LibraryMember.objects.values("id", "name")
for m in members:
m["books"] = LibraryBorrows.objects.filter(
member_id=m['id']
).values("name", "author")
Now you still have to issue one additionnal db query for each parent row which may not be that efficient, depending on the number of LibraryMember. If you have hundreds or more LibraryMember, a better approach would be to query on the LibraryBorrow instead, including the related fields from LibraryMember, then regroup the rows based on LibraryMember id, ie:
from itertools import group_by
def filter_row(row):
for name in ("librarymember__id", "librarymember__name"):
del row[name]
return row
members = []
rows = LibraryBorrow.objects.values(
'name', 'author', 'librarymember__id', 'librarymember__name'
).order_by('librarymember__id')
for key, group in group_by(rows, lambda r: r['librarymember__id']):
group = list(group)
member = {
'id' : group[0]['librarymember_id'],
'name':group[0]['librarymember_name']
'books' = [filter_row(row) for row in group]
}
members.append(member)
NB : this can be seen as premature optimization (and would be if you only have a couple LibraryMember in your db), but trading hundreds or more queries for one single query and a bit of postprocessing usually makes a real difference for "real life" datasets.
Well m is a LibraryMember object so you won't be able to treat it as a dictionary. As a side note: Most people don't name the models in plural form since they are just a class modeling an object, not a collection of objects.
One possible solution is to make a list of dictionaries with the values that you need from both objects, something like this in a one-liner:
o = [ { "id": m.id, "name": m.name, "books": [{"name": b.name, "author": b.author} for b in m.libraryborrows_set.all()] } for m in LibraryMembers.objects.all()]
Note that you can use the related manager to get the books for a given member. For better clarity:
o = []
for m in LibraryMembers.objects.all():
member_books = [{"name": b.name, "author": b.author} for b in m.libraryborrows_set.all()]
o.append( { "id": m.id, "name": m.name, "books": member_books } )
EDIT:
To serialize all the fields:
members = []
for member in LibraryMembers.objects.all():
member_details = {}
for field in member._meta.get_all_field_names():
member_details[field] = getattr(member, field)
books = []
for book in member.librayborrows_set.all():
book_details = {}
for field in book._meta.get_all_field_names():
book_details[field] = getattr(book, field)
books.append(book_details)
member_details['books'] = books
members.append(member_details)
I also found DjangoFullSerializers which I hadn't heard about until today:
http://code.google.com/p/wadofstuff/wiki/DjangoFullSerializers

Categories