I am using a python and getting the data from an API the data formatted as listed in the example I have a problem getting out Cust_id and name put of the API
Below is one of the things I tried and one of the things answered by SimonR. I am sure I am doing something really dumb right now but I get the error
typeError: the JSON object must be str, bytes or bytearray, not dict. Thank you everyone in advance for your answers
import json
a = {
"count": 5,
"Customers": {
"32759": {
"cust_id": "1234",
"name": "Mickey Mouse"
},
"11053": {
"cust_id": "1235",
"name": "Mini Mouse"
},
"21483": {
"cust_id": "1236",
"name": "Goofy"
},
"12441": {
"cust_id": "1237",
"name": "Pluto"
},
"16640": {
"cust_id": "1238",
"name": "Donald Duck"
}
}
}
d = json.loads(a)
customers = {v["cust_id"]: v["name"] for v in d["Customers"].values()}
Is this what you're trying to do ?
import json
d = json.loads(a)
customers = {v["cust_id"]: v["name"] for v in d["Customers"].values()}
outputs :
{'1234': 'Mickey Mouse',
'1235': 'Mini Mouse',
'1236': 'Goofy',
'1237': 'Pluto',
'1238': 'Donald Duck'}
Well if I understood correctly you can do this:
# d is the API response in your post
# This will give you the list of customers
customers = d['Customers']
Then you can iterate over the customers dictionary and save them to any data structure you want:
# This will print out the name and cust_id
for k, v in customers.items():
print(v['cust_id'], v['name'])
Hope it helps!
import json
# convert json to python dict
response = json.loads(json_string)
# loop through all customers
for key, customer in response['Customers'].items():
# get customer id
customer['cust_id']
# get customer name
custoemr['name']
Related
This question already has answers here:
Python list of dictionaries search
(24 answers)
Closed last month.
First, I am new to Python and working with JSON.
I am trying to extract just one value from an API request response, and I am having a difficult time parsing out the data I need.
I have done a lot of searching on how to do this, but most all the examples use a string or file that is formatted is much more basic than what I am getting.
I understand the key - value pair concept but I am unsure how to reference the key-value I want. I think it has something to do with the response having multiple objects having the same kay names. Or maybe the first line "Bookmark" is making things goofy.
The value I want is for the model name in the response example below.
That's all I need from this. Any help would be greatly appreciated.
{
"Bookmark": "<B><P><p>SerNum</p><p>Item</p></P><D><f>false</f><f>false</f></D><F><v>1101666</v><v>ADDMASTER IJ7102-23E</v></F><L><v>123456</v><v>Model Name</v></L></B>",
"Items": [
[
{
"Name": "SerNum",
"Value": "123456"
},
{
"Name": "Item",
"Value": "Model Name"
},
{
"Name": "_ItemId",
"Value": "PBT=[unit] unt.DT=[2021-07-28 08:20:33.513] unt.ID=[eae2621d-3e9f-4515-9763-55e67f65fae6]"
}
]
],
"Message": "Success",
"MessageCode": 0
}
If you want to find value of dictionary with key 'Name' and value 'Item' you can do:
import json
with open('your_data.json', 'r') as f_in:
data = json.load(f_in)
model_name = next((i['Value'] for lst in data['Items'] for i in lst if i['Name'] == 'Item'), 'Model name not found.')
print(model_name)
Prints:
Model Name
Note: if the dictionary is not found string 'Model name not found.' is returned
First, load the JSON into a python dict:
import json
x = '''{
"Bookmark": "<B><P><p>SerNum</p><p>Item</p></P><D><f>false</f><f>false</f></D><F><v>1101666</v><v>ADDMASTER IJ7102-23E</v></F><L><v>123456</v><v>Model Name</v></L></B>",
"Items": [
[
{
"Name": "SerNum",
"Value": "123456"
},
{
"Name": "Item",
"Value": "Model Name"
},
{
"Name": "_ItemId",
"Value": "PBT=[unit] unt.DT=[2021-07-28 08:20:33.513] unt.ID=[eae2621d-3e9f-4515-9763-55e67f65fae6]"
}
]
],
"Message": "Success",
"MessageCode": 0
}'''
# parse x:
y = json.loads(x)
# The result is a Python dictionary.
Now if you want the value 'Model Name', you would do:
print(y['Items'][0][1]['Value'])
I want to store key-value JSON data in aws DynamoDB where key is a date string in YYYY-mm-dd format and value is entries which is a python dictionary. When I used boto3 client to save data there, it saved it as a data type object, which I don't want. My purpose is simple: Store JSON data against a key which is a date, so that later I will query the data by giving that date. I am struggling with this issue because I did not find any relevant link which says how to store JSON data and retrieve it without any conversion.
I need help to solve it in Python.
What I am doing now:
item = {
"entries": [
{
"path": [
{
"name": "test1",
"count": 1
},
{
"name": "test2",
"count": 2
}
],
"repo": "test3"
}
],
"date": "2022-10-11"
}
dynamodb_client = boto3.resource('dynamodb')
table = self.dynamodb_client.Table(table_name)
response = table.put_item(Item = item)
What actually saved:
[{"M":{"path":{"L":[{"M":{"name":{"S":"test1"},"count":{"N":"1"}}},{"M":{"name":{"S":"test2"},"count":{"N":"2"}}}]},"repo":{"S":"test3"}}}]
But I want to save exactly the same JSON data as it is, without any conversion at all.
When I retrieve it programmatically, you see the difference of single quote, count value change.
response = table.get_item(
Key={
"date": "2022-10-12"
}
)
Output
{'Item': {'entries': [{'path': [{'name': 'test1', 'count': Decimal('1')}, {'name': 'test2', 'count': Decimal('2')}], 'repo': 'test3'}], 'date': '2022-10-12} }
Sample picture:
Why not store it as a single attribute of type string? Then you’ll get out exactly what you put in, byte for byte.
When you store this in DynamoDB you get exactly what you want/have provided. Key is your date and you have a list of entries.
If you need it to store in a different format you need to provide the JSON which correlates with what you need. It's important to note that DynamoDB is a key-value store not a document store. You should also look up the differences in these.
I figured out how to solve this issue. I have two column name date and entries in my dynamo db (also visible in screenshot in ques).
I convert entries values from list to string then saved it in db. At the time of retrival, I do the same, create proper json response and return it.
I am also sharing sample code below so that anybody else dealing with the same situation can have atleast one option.
# While storing:
entries_string = json.dumps([
{
"path": [
{
"name": "test1",
"count": 1
},
{
"name": "test2",
"count": 2
}
],
"repo": "test3"
}
])
item = {
"entries": entries_string,
"date": "2022-10-12"
}
dynamodb_client = boto3.resource('dynamodb')
table = dynamodb_client.Table(<TABLE-NAME>)
-------------------------
# While fetching:
response = table.get_item(
Key={
"date": "2022-10-12"
}
)['Item']
entries_string=response['entries']
entries_dic = json.loads(entries_string)
response['entries'] = entries_dic
print(json.dumps(response))
I have a JSON file like this below and the keys in the custom_fields can vary for each id. I need to import this data into BigQuery but they don't allow field names to begin with a number. So, using Python 3.7, I am trying to find out how can I dynamically concatenate a value to the beginning of those keys within custom_fields without manually specifying each field name?
{
"response":[{
"id": "123",
"custom_fields":{
"5c30673efc89f7000400001d":"val1",
"5e34770a8e3d1b010a757981":"val2",
"5e3477d28e3d1b0140757993":"val3"
}},
{
"id": "456",
"custom_fields":{
"5c30673efc89f7000400001d":"val1",
"5e34770a8e3d1b010a757981":"val2",
"5e3477d28e3d1b0140757993":"val3"
}}]
}
The data is coming from an API and saved to cloud storage, with the output being retrieved and formatted to JSON with this:
response = urllib.request.Request('https://www.test.com')
result = urllib.request.urlopen(response)
resulttext = result.read()
jsonResponse = json.loads(resulttext.decode('utf-8'))
Desired output would be like:
{
"response":[{
"id": "123",
"custom_fields":{
"_5c30673efc89f7000400001d":"val1",
"_5e34770a8e3d1b010a757981":"val2",
"_5e3477d28e3d1b0140757993":"val3"
}},
{
"id": "456",
"custom_fields":{
"_5c30673efc89f7000400001d":"val1",
"_5e34770a8e3d1b010a757981":"val2",
"_5e3477d28e3d1b0140757993":"val3"
}}]
}
If the jsonResponse is like what you've shown in your post then this should do the job fine.
for d in jsonResponse["response"]:
d["custom_fields"] = {f"_{k}": v for k, v in d["custom_fields"].items()}
import pprint
a_dict = {
"id": "123",
"custom_fields":{
"5c30673efc89f7000400001d":"val1",
"5e34770a8e3d1b010a757981":"val2",
"5e3477d28e3d1b0140757993":"val3"
}
}
print('before')
pprint.pprint(a_dict)
for key in a_dict['custom_fields']:
k_new = '_' + key
a_dict['custom_fields'][k_new] = a_dict['custom_fields'].pop(key)
print('after')
pprint.pprint(a_dict)
outputs:
before
{'custom_fields': {'5c30673efc89f7000400001d': 'val1',
'5e34770a8e3d1b010a757981': 'val2',
'5e3477d28e3d1b0140757993': 'val3'},
'id': '123'}
after
{'custom_fields': {'_5c30673efc89f7000400001d': 'val1',
'_5e34770a8e3d1b010a757981': 'val2',
'_5e3477d28e3d1b0140757993': 'val3'},
'id': '123'}
I have a JSON object as below that I load from a file. I want to parse the JSON object for all the CityName values.
{
"CityList": [
{
"Continent": "USA",
"CityName": "Chicago"
},
{
"Continent": "Russia",
"CityName": "Moscow"
},
{
"Continent": "Asia",
"CityName": "Beijing"
},
{
"Continent": "Australia",
"CityName": "Sydney"
}
]
}
I am using Python script to extract the CityName element from the JSON such through a FOR loop. I want to use the "name" variable downstream for some other reasons.
name=Chicago
name=Moscow
name=Beijing
name=Sydney
I have tried the following so far.
with open('city_names.json','r') as read_file:
json_data = read_file.read()
data = json.loads(json_data)
for k,v in data.items():
name=v['CityName']
print(name)
After running the above unsuccessfully for quite some time, I keep getting this error "TypeError: list indices must be integers, not
str". I know what the issue is but unfortunately I don't know the
fix. Help is really appreciated.
to get CityName, you can loop through data['CityList'] since it is a list:
for v in data['CityList']:
name=v['CityName']
print(name)
Your data is a list of dicts under the CityList key, so you should iterate over the sub-dicts of the list first and then output the value of the CityName key:
for d in data['CityList']:
print(d['CityName'])
I have an array of dictionaries like so:
myDict[0] = {'date':'today', 'status': 'ok'}
myDict[1] = {'date':'yesterday', 'status': 'bad'}
and I'm trying to export this array to a json file where each dictionary is its own entry. The problem is when I try to run:
dump(myDict, open("test.json", "w"))
It outputs a json file with a number prefix before each entry
{"0": {"date": "today", "status": "ok"}, "1": {"date": "yesterday", "status": "bad"} }
which apparently isn't legal json since my json parser (protovis) is giving me error messages
Any ideas?
Thanks
Use a list instead of a dictionary; you probably used:
myDict = {}
myDict[0] = {...}
You should use:
myList = []
myList.append({...}
P.S.: It seems valid json to me anyways, but it is an object and not a list; maybe this is the reason why your parser is complaining
You should use a JSON serializer...
Also, an array of dictionaries would better serialize to something like this:
[
{
"date": "today",
"status": "ok"
},
{
"date": "yesterday",
"status": "bad"
}
]
That is, you should just use a JavaScript array.