We Keep Coding

Creating a custom json from existing json

I have Json something like below
Input Json
"blocks": [
{
"key": "",
"text": "Contents",
"type": "unstyled",
"depth": 0,
"inlineStyleRanges": [
{
"offset": 0,
"length": 8,
"style": "BOLD"
}
],
"entityRanges": [],
"data": {}
},
{
"key": "",
"text": "1.\u00a0\u00a0\u00a0\u00a0 Completed\n Activities & Accomplishments\u00a0 1",
"type": "unstyled",
"depth": 0,
"inlineStyleRanges": [],
"entityRanges": [
{
"offset": 0,
"length": 49,
"key": 0
}
],
"data": {}
},
{
"key": "",
"text": "2.\u00a0\u00a0\u00a0\u00a0 Planned Activities for\n Next Reporting Period\u00a0 3",
"type": "unstyled",
"depth": 0,
"inlineStyleRanges": [],
"entityRanges": [
{
"offset": 0,
"length": 55,
"key": 1
}
],
"data": {}
},
I am trying to extract "text" key data and want to convert it into new json in the key value format
I am able to extract text properly now I just wanna know how to separate numeric from letters successfully
def jsontojson():
with open('C:\Extraction\Docs\TMS TO 692M15-22-F-00073 Monthly Status _ Financial Report July 2022.json') as json_file:
# json1=json_file.read()
json1=json.load(json_file)
value=""
for dict1 in json1["blocks"]:
# print(dict1)
for key in dict1:
if key=="text":
value+=dict1[key]
dict2={}
d=value.split()
print("Value of d",d)
if str(d).isdigit():
dict2['key1']=d
else:
dict2['desciption']=d
print("Dictionary is",dict2['key'])
For the above code it gives me Key error : key1
Let me know where i am wrong or what I need to do so that i can get the Output
Expected OUTPUT
[
{
"key": "",
"text": "Contents"
},
{
"key": "1.",
"text": "Completed Activities & Accomplishments"
},
{
"key": "2.",
"text": "Planned Activities for Next Reporting Period"
},
]

Try (json1 contains the data from your question):
import re
out = []
for d in json1["blocks"]:
num = re.search(r"^(\d+\.)\s*(.*)", d["text"].replace("\n", ""))
out.append(
{
"key": num.group(1) if num else "",
"text": (num.group(2) if num else d["text"]).rsplit(maxsplit=1)[0],
}
)
print(out)
Prints:
[
{"key": "", "text": "Contents"},
{"key": "1.", "text": "Completed Activities & Accomplishments"},
{"key": "2.", "text": "Planned Activities for Next Reporting Period"},
]

Another and easy way is (if your first list is called "Mylist" :
new_list =[]
for dic in Mylist:
new_list.append({'key':dic['key'],
'text':dic['text']})

How to modify a string inside a JSON object and convert it into another JSON object and thus add a key value with Python?

I would like to find a way to convert a string inside my JSON object into another JSON object (nested) with the use of Python and thus add a key to the values inside my string that are separated by the use of a ";".
My current JSON object:
{
"reference": "#############",
"messageContext": "",
"from": {
"number": "+#############",
"name": ""
},
"to": {
"number": "#############"
},
"message": {
"text": "12302;8;6245;d517666e-41ca-459a-9b35-c49386df537b;2;2;50.8447;-4.3614;2021-04-28T22:24:12.204Z;rec123;PRD",
"media": {
"mediaUri": "",
"contentType": "",
"title": ""
},
"custom": {}
},
"groupings": [
"",
"",
""
],
"time": "2021-05-02 14:03:22",
"timeUtc": "2021-05-02T12:03:22",
"channel": "Sms"
}
The format I would try to have is something like this, where I would change the text to an object while adding key values.
Result, which I try to obtain (Inside of text:" "):
{
"reference": ""#############",",
"messageContext": "",
"from": {
"number": "+"#############",",
"name": ""
},
"to": {
"number": ""#############","
},
"message": {
"text": {
"APSI": "12302",
"idVhl": 8,
"idDrv": 6245,
"GUID": "d517666e-41ca-459a-9b35-c49386df537b",
"idLne": 2,
"idSvc": 2,
"Lat": 50.8447,
"Lon": -4.3614,
"Timestamp": "2021-04-28T22:24:12.204Z",
"Rec": "rec123",
"Env": "PRD"
},
"media": {
"mediaUri": "",
"contentType": "",
"title": ""
},
"custom": {}
},
"groupings": [
"",
"",
""
],
"time": "2021-05-02 14:03:22",
"timeUtc": "2021-05-02T12:03:22",
"channel": "Sms"
}

Given your JSON object as input this function will return your expected output.
def transform(input):
text = input['message']['text']
text_list = text.split(';')
text_dict = {
'APSI': text_list[0],
'idVhl': int(text_list[1]),
'idDrv': int(text_list[2]),
'GUID': text_list[3],
'idLne': int(text_list[4]),
'idSvc': int(text_list[5]),
'Lat': float(text_list[6]),
'Lon': float(text_list[7]),
'Timestamp': text_list[8],
'Rec': text_list[9],
'Env': text_list[10]
}
input['message']['text'] = text_dict
return input
This is done by splitting your text on the ; character which returns a list of all the values you want. You can then parse the list into a dictionary and then assign that dictionary to your original text value.

Elasticsearch query in Python does not return All of the results

I am new to python and elastic search and I have written an elasticsearch python query code which has to receive some data (keyword or category) and send a query to the ES database; then retrieve and print the matching data in my console:
def SingleKeywordQuery (keyword):
response = es.search(index="main-news-test-data",body={"from":0,"size":10000,"query":{"multi_match":{
"query": keyword,
"fields": [ "content", "title", "lead" ]
}}})
return(response)
def SingleCategoryQuery (category):
response = es.search(index="main-news-test-data",body={"from":0,"size":10000,"query":{"match":{"category": category}}})
return(response)
if __name__ == '__main__':
keyword = input('Enter Keyword: ')
category = input('Enter Category: ')
mapCategory = Mapper(category)
...
elif keyword != '' and mapCategory is None:
data = SingleKeywordQuery(keyword)
print(data)
elif keyword == '' and mapCategory is not None:
mapCategory = eval(mapCategory)
print(mapCategory)
data = SingleCategoryQuery(mapCategory)
print(data)
elif keyword == '' and mapCategory is None:
print('Please select a filter')
The problem is, that my code only retrieves one of the results. like, I know for a fact that there are 3 data with the category number that I want or 2 data with the keyword that I want but it only returns one. what seems to be the problem?
Edit: Data Mapping of Elasticsearch
PUT /main-news-test-data
{
"mappings": {
"properties": {
"content": {
"type": "text"
},
"title": {
"type": "text"
},
"lead": {
"type": "text"
},
"agency": {
"type": "keyword"
},
"date_created": {
"type": "date"
},
"url": {
"type": "keyword"
},
"image": {
"type": "keyword"
},
"category": {
"type": "keyword"
},
"id":{
"type": "keyword"
}
}
}
}
and this is the BULK data that I uploaded using Bulk API:
{ "index" : { "_index" : "main-news-test-data", "_id" : "1" } }
{
"content":"\u0641\u0647\u06cc\u0645\u0647 \u062d\u0633\u0646\u200c\u0645\u06cc\u0631\u06cc: \u0627\u06af\u0631\u0686\u0647 \u062f\u0631 \u0647\u06cc\u0627\u0647\u0648\u06cc ",
"title":"\u06a9\u0627\u0631\u0647\u0627\u06cc \u0642\u0627\u0644\u06cc\u0628\u0627\u0641",
"lead":"\u062c\u0627\u0645\u0639\u0647 > \u0634\u0647\u0631\u06cc -.",
"agency":"13",
"date_created":1494518193,
"url":"http://www.khabaronline.ir/(X(1)S(bud4wg3ebzbxv51mj45iwjtp))/detail/663749/society/urban",
"image":"uploads/2017/05/11/1589793661.jpg",
"category":"15",
"id":"2981643"
}
{ "index" : { "_index" : "main-news-test-data", "_id" : "2" } }
{
....

How can I iterate over nested json dicts?

I've been trying to figure out how I can iterate over a json like object, so I could get a user id by its name.
json
{
"ApiSearchResult": [
{
"totalNumberResults": 55,
"type": "User",
"searchResults": [
{
"firstName": "shashank",
"name": "0o_shashank._o0",
"uid": 81097836
},
{
"firstName": "Shahnawaz",
"name": "0shahnawaz.0",
"uid": 83697589
},
{
"firstName": "Ashu",
"name": "ashu.-3",
"uid": 83646061
},
{
"bgImage": "photoalbum_491396460_user82597906-1-jpeg.jpg",
"firstName": "Garfield",
"name": "beast.boy",
"uid": 82597906
},
{
"firstName": "Bharath",
"name": "bharath_mohan69",
"uid": 80197615
},
{
"bgImage": "photoalbum_481041410_user79819261-1-jpg.jpg",
"firstName": "Wille-ICE",
"name": "blowhole",
"uid": 79819261
}
]
}
]
}
Python
def getidbyname(name):
event = response['ApiSearchResult'][0]['searchResults'][0]
for key, value in event.iteritems():
if value == name: continue
elif key == "uid":
return value
But, this won't work, I've never really worked with this many nested elements.

def getidbyname(name):
for i in data['ApiSearchResult'][0]['searchResults']:
if i['name'] == name:
return i['uid']

This might work if your response is already a python dictionary:
def getidbyname(name):
for event in data["ApiSearchResult"][0]["searchResults"]:
if event["name"] == name:
return event["uid"]
If your input is a text value, you need to use json.loads(response) to get a python dictionary out of it.

How can I add if statements to a json dictionary in Python

Is it possible to change a json dictionary payload with if statements?
I have 4 types that say have IDs 1,2,3,4 so I need some sort of select switch code.
The original code is:
payload = {
"added": [
{
"assingedTargets": [
{
"sequencing": 1,
"name": assigned_target_name,
"id": assigned_target_id
}
],
"type": {
"protocol": {
"isDefaultProtocol": True,
"name": "tcp",
"id": 3,
"label": "tcp"
},
"name": target_type,
"id": int(1)
},
"name":"DDITargetGroup5"
}
]
}
What I would like to do is:
payload = {
"added": [
{
"assingedTargets": [
{
"sequencing": 1,
"name": assigned_target_name,
"id": assigned_target_id
}
],
"type": {
"protocol": {
"isDefaultProtocol": True,
"name": "tcp",
"id": 3,
"label": "tcp"
},
if target_type: == "direct dial"
"name": target_type,
"id": int(1)
elif target_type == "private Wire"
"name": target_type
"id": int(2)
},
"name":"DDITargetGroup5"
}
]
}
I get the error:
if target_type: == "direct dial"
^
SyntaxError: invalid syntax
Process finished with exit code 1

If there are just two options, you could use the ternary operator
{
...
"name": target_type,
"id": 1 if target_type == 'direct dial' else 2,
...
}
However, it comes with a cost of readability, so you should consider whether wouldn't be better to separate the conditional statement from the dict.

You can't just plop a conditional statement into a json object. Instead you should create a variable to store the id value (id_val) and use a conditional before you make the object to set its value based on the value of target_type. Then set the dictionary to have the values of those variables (see below).
id_val = 0
if target_type == 'direct dial':
id_val = 1
elif target_type == 'private Wire':
id_val = 2
payload = { "added": [ { "assingedTargets": [ { "sequencing": 1, "name": assigned_target_name, "id": assigned_target_id } ], "type": { "protocol": { "isDefaultProtocol": True, "name": "tcp", "id": 3, "label": "tcp" }, "name": target_type, "id": id_val }, "name":"DDITargetGroup5" } ] }