I use the following code to get my data out of the dict.
test = self.events_max_total_gross()
events = organizer.events.all()
for event in events:
test.get(event.pk, {}).values()
[...]
I use this query set to get the data. My question is: Does the transformation at the end makes sense or is there a better way to access the dict (without transforming it first). As I have several of these my approach doesn't seem to follow the DRY principle.
def events_max_total_gross(self):
events_max_total_gross = (
Event.objects.filter(
organizer__in=self.organizers,
status=EventStatus.LIVE
)
.annotate(total_gross=Sum(F('tickets__quantity') * F('tickets__price_gross')))
.values('pk', 'total_gross')
)
"""
Convert this
[
{
'pk': 2,
'total_gross': 12345
},
{
'pk': 3,
'total_gross': 54321
},
...
]
to this:
{
2: {
'total_gross': 12345,
},
3: {
'total_gross': 12345,
}
...
}
"""
events_max_total_gross_transformed = {}
for item in events_max_total_gross:
events_max_total_gross_transformed.setdefault(
item['pk'], {}
).update({'total_gross': item['total_gross']})
return events_max_total_gross_transformed
Use:
transformed = {
v['pk']: { 'total_gross': v['total_gross'] } for v in events_max_total_gross
}
This is called a python dict comprehension. Google that term if you want tutorials or examples.
If I understood it correctly, then for every organiser's event you need total_gross, you can query like this instead of going over events and events_max_total_gross in loop.
First get all the events of that particular organizer.
event_ids = Event.objects.filter(organizer=organizer).values_list('id',flat=True)
Run this then:
Event.objects.filter(
id__in=event_ids,
organizer__in=self.organizers,
status=EventStatus.LIVE
)
.annotate(total_gross=Sum(F('tickets__quantity') * F('tickets__price_gross')))
.values('pk', 'total_gross')
)
This way you will save your transformation of dict and looping it over again.
In case you need to do like this because of some other requirement then you can use python dict comprehension:
events_max_total_gross_transformed = {
event['pk']: { 'total_gross': event['total_gross'] } for event in events_max_total_gross
}
But as you have several of these, you might wanna have a look at proxy-models also which might help. Here you can write manager functions to help you with your queries.
Related
So I have a flattened tree in JSON like this, as array of objects:
[{
aid: "id3",
data: ["id1", "id2"]
},
{
aid: "id1",
data: ["id3", "id2"]
},
{
aid: "id2",
nested_data: {aid: "id4", atype: "nested", data: ["id1", "id3"]},
data: []
}]
I want to gather that tree and resolve ids into data with recursion loops into something like this (say we start from "id3"):
{
"aid":"id3",
"payload":"1",
"data":[
{
"id1":{
"aid":"id1",
"data":[
{
"id3":null
},
{
"id2":null
}
]
}
},
{
"id2":{
"aid":"id2",
"nested_data":{
"aid":"id4",
"atype":"nested",
"data":[
{
"id1":null
},
{
"id3":null
}
]
},
"data":[
]
}
}
]
}
So that we would get breadth-first search and resolve some field into "value": "object with that field" on first entrance and "value": Null
How to do such a thing in python 3?
Apart from all the problems that your structure has in terms of syntax (identifiers must be within quotes, etc.), the code below will provide you with the requested answer.
But you should carefully think about what you are doing, and have the following into account:
Using the relations expressed in the flat structure that you provide will mean that you will have an endless recursion since you have items that include other items that in turn include the first ones (like id3 including id1, which in turn include id3. So, you have to define stop criteria, or be sure that this does not occur in your flat structure.
Your initial flat structure is better to be in the form of a dictionary, instead of a list of pairs {id, data}. That is why the first thing the code below does is to transform this.
Your final, desired structure contains a lot of redundancies in terms of information contained. Consider simplifying it.
Finally, you mentioned nothing about the "nested_data" nodes, and how they should be treated. I simply assumed that in case that exist, further expansion is required.
Please, consider trying to provide a bit of context in your questions, some real data examples (I believe the data provided is not real data, therefore the inconsistencies and redundancies), and try yourself and provide your efforts; that's the only way to learn.
from pprint import pprint
def reformat_flat_info(flat):
reformatted = {}
for o in flat:
key = o["aid"]
del o["aid"]
reformatted[key] = o
return reformatted
def expand_data(aid, flat, lvl=0):
obj = flat[aid]
if obj is None: return {aid: obj}
obj.update({"aid": aid})
if lvl > 1:
return {aid: None}
for nid,id in enumerate(obj["data"]):
obj["data"][nid] = expand_data(id, flat, lvl=lvl+1)
if "nested_data" in obj:
for nid,id in enumerate(obj["nested_data"]["data"]):
obj["nested_data"]["data"][nid] = expand_data(id, flat, lvl=lvl+1)
return {aid: obj}
# Provide the flat information structure
flat_info = [
{
"aid": "id3",
"data": ["id1", "id2"]
}, {
"aid": "id1",
"data": ["id3", "id2"]
}, {
"aid": "id2",
"nested_data": {"aid": "id4", "atype": "nested", "data": ["id1", "id3"]},
"data": []
}
]
pprint(flat_info)
print('-'*80)
# Reformat the flat information structure
new_flat_info = reformat_flat_info(flat=flat_info)
pprint(new_flat_info)
print('-'*80)
# Generate the result
starting_id = "id3"
result = expand_data(aid=starting_id, flat=new_flat_info)
pprint(result)
I have a python code that interacts with multiple APIs. All of the APIs return some json but each has different structure. Let's say I'm looking for people's names in all these jsons:
json_a = {
"people": [
{"name": "John"},
{"name": "Peter"}
]
}
json_b = {
"humans": {
"names": ["Adam", "Martin"]
}
}
As you can see above the dictionaries from jsons have arbitrary structures. I'd like to define something that will serve as a "blueprint" for navigating each json, something like this:
all_jsons = {
"json_a": {
"url": "http://endpoint",
"json_structure": "people -> list -> name"
},
"json_b": {
"url": "http://someotherendpoint",
"json_structure": "humans -> names -> list"
}
}
So that if I'm working with json_a I'll just look into all_jsons["json_a"]["json_structure"] and I have an information on how to navigate this exact json. What would be the best way to achieve this?
Why not define concrete retrieval functions for each api:
def retrieve_a(data):
return [d["name"] for d in data["people"]]
def retrieve_b(data):
return data["humans"]["names"]
and store them for each endpoint:
all_jsons = {
"json_a": {
"url": "http://endpoint",
"retrieve": retrieve_a
},
"json_b": {
"url": "http://someotherendpoint",
"retrieve": retrieve_b
}
}
I have found this approach more workable than trying to express code-logic by configuration. Then you can easily collect names:
for dct in all_jsons.values():
data = ... # requests.get(dct["url"]).json() # or similar
names = dct["retrieve"](data)
To get a value from a dictionary with a key if it may not exist, dict.get(key) is used for. To distinguish which type list or dict or else, type(val) is useful for. Combination of them should achieve your problem.
I want to add a set of extendedProperties to a Google Calendar event. I want some of those properties be lists. As (in python),
event = {
..., # standard properties
"extendedProperties": {
"shared": {
"max_crew": 3,
"crew_list": [
"crew1#example.com",
"crew2#example.com",
],
}
}
...
}
This creates the max_crew property but not the crew_list property.
Any way to do this? Or do I need to use a parse-able string (max 1024 chars)?
There is a way: explained in Google Calendar's guide and reference.
And in Python first create a dictionary for your extra fields.
body = {
"extendedProperties": {
"private": {
"petsAllowed": "yes"
}
}
}
Then make a request with:
service.events().patch(calendarId='calendar_id', eventId='event_id', body=body).execute()
If it is successful, it will return the updated event.
Hybor confirms my observation that the interface does not support a list as a value. Shortsighted, imho, but so it goes.
I have such slice of loaded json tp python dictionary (size_dict):
{
"sizeOptionName":"XS",
"sizeOptionId":"1528",
"sortOrderNumber":"7017"
},
{
"sizeOptionName":"S",
"sizeOptionId":"1529",
"sortOrderNumber":"7047"
},
{
"sizeOptionName":"M",
"sizeOptionId":"1530",
"sortOrderNumber":"7095"
}
and I have products with size Id (dictionary_prod):
{
"catalogItemId":"7627712",
"catalogItemTypeId":"3",
"regularPrice":"0.0",
"sizeDimension1Id":"1528",
"sizeDimension2Id":"0",
}
I need to make such as output for any product:
result_dict = {'variant':
[{"catalogItemId":"7627712", ...some other info...,
'sizeName': 'XS', 'sizeId': '1525'}}]}
so I need to convert size ID and add it to new result object
What is the best pythonic way to do this?
I dont know how to get right data from size_dict
if int(dictionary_prod['sizeDimension1Id']) > o:
(result_dict['variant']).append('sizeName': size_dict???)
As Tommy mentioned, this is best facilitated by mapping the size id's to their respective dictionaries.
size_dict = \
[
{
"sizeOptionName":"XS",
"sizeOptionId":"1528",
"sortOrderNumber":"7017"
},
{
"sizeOptionName":"S",
"sizeOptionId":"1529",
"sortOrderNumber":"7047"
},
{
"sizeOptionName":"M",
"sizeOptionId":"1530",
"sortOrderNumber":"7095"
}
]
size_id_map = {size["sizeOptionId"] : size for size in size_dict}
production_dict = \
[
{
"catalogItemId":"7627712",
"catalogItemTypeId":"3",
"regularPrice":"0.0",
"sizeDimension1Id":"1528",
"sizeDimension2Id":"0",
}
]
def make_variant(idict):
odict = idict.copy()
size_id = odict.pop("sizeDimension1Id")
odict.pop("sizeDimension2Id")
odict["sizeName"] = size_id_map[size_id]["sizeOptionName"]
odict["sizeId"] = size_id
return odict
result_dict = \
{
"variant" : [make_variant(product) for product in production_dict]
}
print(result_dict)
Your question is a little confusing but it looks like you have a list (size_dict) of dictionaries that contain some infroamtion and you want to do a lookup to find a particular element in the list that contains the SizeOptionName you are interested in so that you can read off the SizeOptionID.
So first you could organsie your size_dict as a dictionary rather than a list - i.e.
sizeDict = {"XS":{
"sizeOptionName":"XS",
"sizeOptionId":"1528",
"sortOrderNumber":"7017"
}, "S": {
"sizeOptionName":"S",
"sizeOptionId":"1529",
"sortOrderNumber":"7047"
}, ...
You could then read off the SizeOptionID you need by doing:
sizeDict[sizeNameYouAreLookingFor][SizeOptionID]
Alternative you could keep your current structure and just search the list of dictionaries that is size_dict.
So:
for elem in size_dict:
if elem.SizeOptionID == sizeYouAreLookingFor:
OptionID = elem.SizeOptionId
Or perhaps you are asking something else?
I'm trying to figure out if there is a way to use conditionals such as $set to do more advanced updating. This is what I'm trying to do in pseudo code:
# new data to use in a possible update
newData = { 'emailAddress' : $usersEmailAddress,
'keyA' : 'valueA',
'keyB' : None,
'keyC' : '<null>' }
self.request.root.db.users.update(
{ 'emailAddress' : newData['emailAddress'] },
{ '$set': {
"Here, loop through all newData keys/values and if
notNull(newData[key]) == True and is different than the
corresponding key/value in the user
document (or if user document doesn't have that key)
than update with newData key/value"
} }, upsert = False, safe = True )
# The result should be that if the value of keyA (and ONLY key A because the
# others are null) is different in the user document
# than in newData, than the user document should be updated with the new value.
# function to catch any possible None value or equivalent string
def notNull(valueToCheck):
if thingToCheck and thingToCheck != "null" and thingToCheck != 'nil' and thingToCheck != '<null>' and thingToCheck != '' and thingToCheck != ' ':
return True
else:
return False
What's the most efficient way of doing this? Because currently I'm having to pull the entire document with find_one and, I'm told, thats rather expensive. Is there a way to do this with just $set?
No, MongoDB does not support this feature. You can either, as you say, retrieve the document, analyze it in your client-side code, and issue an update based on its contents, or you could issue a series of updates like:
db.users.update({
'emailAddress': newData['emailAddress'],
'$or': [
{ 'keyA': { '$exists': false } },
{ 'keyA': None } ] }
]
}, {
'$set': { 'keyA': newData['keyA'] }
})
The former will be more efficient, of course, since it's a single fetch and a single update. But consider whether you need to guard against multiple MongoDB clients simultaneously fetching and updating the same document.