I have a mongo database including the following collection:
"
"_id": {
"$oid": "12345"
},
"id": "333555",
"token": [
{
"access_token": "ac_33bc",
"expires_in": 3737,
"token_type": "bearer",
"expires_at": {
"$date": "2021-07-02T13:37:28.123Z"
}
}
]
}
In the next python script I'm trying to return and print only the access_token but can't figure out how to do so. I've tried various methods which none of the worked.I've given the "id" as a parameter
def con_mongo():
try:
client = pymongo.MongoClient("mongodb:localhost")
#DB name
db = client["db1"]
#Collection
coll = db["coll1"]
#1st method
x = coll.find({"id":"333555"},{"token":"access_token"})
for data in x:
print(x)
#2nd method
x= coll.find({"id":"333555"})
tok=x.distinct("access_token")
#print(x[0])
for data in tok:
print(data)
except Exception:
logging.info(Exception)
It doesn't work this way, although if I replace (or remove) the "access_token" with simply "token" it works but I get back all the informations included in the field "token" where I only need the value of the "access_token".
Since access_token is an array element, you need to qualify it's name with the name of the array, to properly access its value.
Actually you can first extract the whole document and get the desired value through simple list and dict indexing.
So, assuming you are retrieving many documents with that same id:
x = [doc["token"][0]["access_token"] for doc in coll.find({"id":"333555"})]
The above, comprehensively creates a list with the access_tokens of all the documents matching the given id.
If you just need the first (and maybe only) occurrence of a document with that id, you can use find_one() instead:
x = coll.find_one({"id":"333555"})["token"][0]["access_token"]
# returns ac_33bc
token is a list so you have to reference the list element, e.g.
x = coll.find({"id":"333555"},{"token.access_token"})
for data in x:
print(data.get('token')[0].get('access_token'))
prints:
ac_33bc
I am trying to build out a JSON export mirrored on this example:
import json
data = {}
data['people'] = []
data['people'].append({
'name': 'Scott',
'website': 'stackabuse.com',
'from': 'Nebraska'
})
with open('data.txt', 'w') as outfile:
json.dump(data, outfile)
However, I wish for there to be more nested {} than just the one above, but cannot seem to figure out how to do that. This is what I'd like the JSON to look like:
{
"people":{
"Citizens":{
"Workers":{
"wage":"34",
"id":"1 of 2"
},
"non-workers":{
"wage":"0",
"id":"2 of 2"
}
}
}
}
Can anyone help me? Thank you!
While the method Brenden Price pointed out works, there is no need for so many statements. The number of statements solely depends on how deep the dictionaries which you are assigning are.
At its shortest:
data = {}
data["people"] = {
"Citizens": {
"Workers": {"wage": "34", "id": "1 of 2"},
"Non-Workers": {"wage": "0", "id": "2 of 2"}
}
}
To create the nested {}, you're adding more dictionaries. I'm not sure the goal of this, but o achieve that exact example you've provided, your code would be something along the lines of this:
data = {}
data['people'] = {} # empty dict of people
data['people']['citizens'] = {} # empty dict of workers inside people
data['people']['citizens']['workers'] = {} # empty dict of workers inside people
data['people']['citizens']['non-workers'] = {} # empty dict of non-workers inside people
data['people']['citizens']['workers']['wage'] = "34" # set workers wage to str "34"
data['people']['citizens']['workers']['id'] = "1 of 2" # set id to str "1 of 2"
data['people']['citizens']['non-workers']['wage'] = "0" # set wage to "0"
data['people']['citizens']['non-workers']['id'] = "2 of 2" # set id to "2 of 2"
Depending on the usage, you could technically do this and create that exact example as well:
data = {
"people":{
"Citizens":{
"Workers":{
"wage":"34",
"id":"1 of 2"
},
"non-workers":{
"wage":"0",
"id":"2 of 2"
}
}
}
}
I have a member variable;
valid_add_customer_request = {
"client": {0},
"name": "CustomerName",
"account_number": "W/L141123512",
"mobile_number": "1232 414122",
"landline_number": "1234515123",
"email": "CustomerName#email.com"
}
Where the client is an id of another object. Ideally I'd like to do something like but because the variable is a dictionary this isn't possible.
valid_add_customer_request.format(1)
Is there a clean way of achieving the above.
valid_add_customer_request = lambda x: {
"client": x,
"name": "CustomerName",
"account_number": "W/L141123512",
"mobile_number": "1232 414122",
"landline_number": "1234515123",
"email": "CustomerName#email.com"
}
valid_add_customer_request(1)
you can do one of the following, depending what desired outcome is:
valid_add_customer_request['client'] = 1 # client = 1 i.e. int
valid_add_customer_request['client'] = {1} # client = {1} i.e. set with one element 1
valid_add_customer_request.setdefault('client', set()).add(1) # client = {0,1}, i.e. add element to the existing set
This may help
https://www.programiz.com/python-programming/methods/dictionary/update.
valid_add_customer_request.update(dict(client=1))
I have a dynamodb table with an attribute containing a nested map and I would like to update a specific inventory item that is filtered via a filter expression that results in a single item from this map.
How to write an update expression to update the location to "in place three" of the item with name=opel,tags include "x1" (and possibly also f3)?
This should just update the first list elements location attribute.
{
"inventory": [
{
"location": "in place one", # I want to update this
"name": "opel",
"tags": [
"x1",
"f3"
]
},
{
"location": "in place two",
"name": "abc",
"tags": [
"a3",
"f5"
]
}],
"User" :"test"
}
Updated Answer - based on updated question statement
You can update attributes in a nested map using update expressions such that only a part of the item would get updated (ie. DynamoDB would apply the equivalent of a patch to your item) but, because DynamoDB is a document database, all operations (Put, Get, Update, Delete etc.) work on the item as a whole.
So, in your example, assuming User is the partition key and that there is no sort key (I didn't see any attribute that could be a sort key in that example), an Update request might look like this:
table.update_item(
Key={
'User': 'test'
},
UpdateExpression="SET #inv[0].#loc = :locVal",
ExpressionAttributeNames={
'#inv': 'inventory',
'#loc': 'location'
},
ExpressionAttributeValues={
':locVal': 'in place three',
},
)
That said, you do have to know what the item schema looks like and which attributes within the item should be updated exactly.
DynamoDB does NOT have a way to operate on sub-items. Meaning, there is no way to tell Dynamo to execute an operation such as "update item, set 'location' property of elements of the 'inventory' array that have a property of 'name' equal to 'opel'"
This is probably not the answer you were hoping for, but it is what's available today. You may be able to get closer to what you want by changing the schema a bit.
If you need to reference the sub-items by name, perhaps storing something like:
{
"inventory": {
"opel": {
"location": "in place one", # I want to update this
"tags": [ "x1", "f3" ]
},
"abc": {
"location": "in place two",
"tags": [ "a3", "f5" ]
}
},
"User" :"test"
}
Then your query would be:
table.update_item(
Key={
'User': 'test'
},
UpdateExpression="SET #inv.#brand.#loc = :locVal",
ExpressionAttributeNames={
'#inv': 'inventory',
'#loc': 'location',
'#brand': 'opel'
},
ExpressionAttributeValues={
':locVal': 'in place three',
},
)
But YMMV as even this has limitations because you are limited to identifying inventory items by name (ie. you still can't say "update inventory with tag 'x1'"
Ultimately you should carefully consider why you need Dynamo to perform these complex operations for you as opposed to you being specific about what you want to update.
You can update the nested map as follow:
First create and empty item attribute of type map. In the example graph is the empty item attribute.
dynamoTable = dynamodb.Table('abc')
dynamoTable.put_item(
Item={
'email': email_add,
'graph': {},
}
Update nested map as follow:
brand_name = 'opel'
DynamoTable = dynamodb.Table('abc')
dynamoTable.update_item(
Key={
'email': email_add,
},
UpdateExpression="set #Graph.#brand= :name, ",
ExpressionAttributeNames={
'#Graph': 'inventory',
'#brand': str(brand_name),
},
ExpressionAttributeValues = {
':name': {
"location": "in place two",
'tag': {
'graph_type':'a3',
'graph_title': 'f5'
}
}
Updating Mike's answer because that way doesn't work any more (at least for me).
It is working like this now (attention for UpdateExpression and ExpressionAttributeNames):
table.update_item(
Key={
'User': 'test'
},
UpdateExpression="SET inv.#brand.loc = :locVal",
ExpressionAttributeNames={
'#brand': 'opel'
},
ExpressionAttributeValues={
':locVal': 'in place three',
},
)
And whatever goes in Key={}, it is always partition key (and sort key, if any).
EDIT:
Seems like this way only works when with 2 level nested properties. In this case you would only use "ExpressionAttributeNames" for the "middle" property (in this example, that would be #brand: inv.#brand.loc). I'm not yet sure what is the real rule now.
DynamoDB UpdateExpression does not search on the database for matching cases like SQL (where you can update all items that match some condition). To update an item you first need to identify it and get primary key or composite key, if there are many items that match your criteria, you need to update one by one.
then the issue to update nested objects is to define UpdateExpression,ExpressionAttributeValues & ExpressionAttributeNames to pass to Dynamo Update Api .
I use a recursive function to update nested Objects on dynamoDB. You ask for Python but I use javascript, I think is easy to see this code and implents on Python:
https://gist.github.com/crsepulv/4b4a44ccbd165b0abc2b91f76117baa5
/**
* Recursive function to get UpdateExpression,ExpressionAttributeValues & ExpressionAttributeNames to update a nested object on dynamoDB
* All levels of the nested object must exist previously on dynamoDB, this only update the value, does not create the branch.
* Only works with objects of objects, not tested with Arrays.
* #param obj , the object to update.
* #param k , the seed is any value, takes sense on the last iteration.
*/
function getDynamoExpression(obj, k) {
const key = Object.keys(obj);
let UpdateExpression = 'SET ';
let ExpressionAttributeValues = {};
let ExpressionAttributeNames = {};
let response = {
UpdateExpression: ' ',
ExpressionAttributeNames: {},
ExpressionAttributeValues: {}
};
//https://stackoverflow.com/a/16608074/1210463
/**
* true when input is object, this means on all levels except the last one.
*/
if (((!!obj) && (obj.constructor === Object))) {
response = getDynamoExpression(obj[key[0]], key);
UpdateExpression = 'SET #' + key + '.' + response['UpdateExpression'].substring(4); //substring deletes 'SET ' for the mid level values.
ExpressionAttributeNames = {['#' + key]: key[0], ...response['ExpressionAttributeNames']};
ExpressionAttributeValues = response['ExpressionAttributeValues'];
} else {
UpdateExpression = 'SET = :' + k;
ExpressionAttributeValues = {
[':' + k]: obj
}
}
//removes trailing dot on the last level
if (UpdateExpression.indexOf(". ")) {
UpdateExpression = UpdateExpression.replace(". ", "");
}
return {UpdateExpression, ExpressionAttributeValues, ExpressionAttributeNames};
}
//you can try many levels.
const obj = {
level1: {
level2: {
level3: {
level4: 'value'
}
}
}
}
I had the same need.
Hope this code helps. You only need to invoke compose_update_expression_attr_name_values passing the dictionary containing the new values.
def compose_update_expression_attr_name_values(data: dict) -> (str, dict, dict):
""" Constructs UpdateExpression, ExpressionAttributeNames, and ExpressionAttributeValues for updating an entry of a DynamoDB table.
:param data: the dictionary of attribute_values to be updated
:return: a tuple (UpdateExpression: str, ExpressionAttributeNames: dict(str: str), ExpressionAttributeValues: dict(str: str))
"""
# prepare recursion input
expression_list = []
value_map = {}
name_map = {}
# navigate the dict and fill expressions and dictionaries
_rec_update_expression_attr_name_values(data, "", expression_list, name_map, value_map)
# compose update expression from single paths
expression = "SET " + ", ".join(expression_list)
return expression, name_map, value_map
def _rec_update_expression_attr_name_values(data: dict, path: str, expressions: list, attribute_names: dict,
attribute_values: dict):
""" Recursively navigates the input and inject contents into expressions, names, and attribute_values.
:param data: the data dictionary with updated data
:param path: the navigation path in the original data dictionary to this recursive call
:param expressions: the list of update expressions constructed so far
:param attribute_names: a map associating "expression attribute name identifiers" to their actual names in ``data``
:param attribute_values: a map associating "expression attribute value identifiers" to their actual values in ``data``
:return: None, since ``expressions``, ``attribute_names``, and ``attribute_values`` get updated during the recursion
"""
for k in data.keys():
# generate non-ambiguous identifiers
rdm = random.randrange(0, 1000)
attr_name = f"#k_{rdm}_{k}"
while attr_name in attribute_names.keys():
rdm = random.randrange(0, 1000)
attr_name = f"#k_{rdm}_{k}"
attribute_names[attr_name] = k
_path = f"{path}.{attr_name}"
# recursion
if isinstance(data[k], dict):
# recursive case
_rec_update_expression_attr_name_values(data[k], _path, expressions, attribute_names, attribute_values)
else:
# base case
attr_val = f":v_{rdm}_{k}"
attribute_values[attr_val] = data[k]
expression = f"{_path} = {attr_val}"
# remove the initial "."
expressions.append(expression[1:])
I can't figure out how to loop though a JSON object that is deeper than 1 level. The object is:
{
"data":[
{
"id":"251228454889939/insights/page_fan_adds_unique/day",
"name":"page_fan_adds_unique",
"period":"day",
"values":[
{
"value":9,
"end_time":"2012-05-29T07:00:00+0000"
},
{
"value":5,
"end_time":"2012-05-30T07:00:00+0000"
}
],
"title":"Daily New Likes",
"description":"Daily The number of new people who have liked your Page (Unique Users)"
},
{
"id":"251228454889939/insights/page_fan_adds/day",
"name":"page_fan_adds",
"period":"day",
"values":[
{
"value":9,
"end_time":"2012-05-29T07:00:00+0000"
},
{
"value":5,
"end_time":"2012-05-30T07:00:00+0000"
}
],
"title":"Daily New Likes",
"description":"Daily The number of new people who have liked your Page (Total Count)"
}
]
}
Code:
def parseJsonData(data):
output_json = json.loads(data)
for i in output_json:
print i
for k in output_json[i]:
print k
How come I can't access the object like: output_json[data][id]?
I get an error if I try this:
string indice must be an integer
Being that your "data" key is actually a list of objects, you cannot access the items by their "id" field directly. You would need to access each item by a list index such as:
output_json["data"][0]["id"]
Now, if what you want to do is to be able to index the members of "data" by the "id" field as the key, you could reformat your data:
# make "data" a dict {id: item, }, instead of list [item1, item2, ...]
output_json['data'] = dict((item['id'], item) for item in json_data['data'])
print output_json['data']
# {'251228454889939/insights/page_fan_adds_unique/day': ...
print output_json['data']['251228454889939/insights/page_fan_adds_unique/day']
# {'description': 'Daily The number of new p ...
# ways to loop over "data"
for id_, item in output_json['data'].iteritems():
print id_, item
for item in output_json['data'].itervalues():
print item
Otherwise what you have to do is just loop over "data", since there is no real correlation between the index and the object:
for item in output_json["data"]:
print item['id']
# 251228454889939/insights/page_fan_adds_unique/day
# 251228454889939/insights/page_fan_adds/day
What you pasted is not valid JSON. There is an unmatched [ after "data".
Based on this, I would guess that maybe the data isn't what you think it is. If the value of output_json[data] is a list, then you won't be able to access output_json[data][id]. Instead you'll have to do something like output_json[data][0][id], where the [0] accesses the first item in the list.