In the python3 code below I read all the objects in the json then loop through the elements in the json. If I only search for the id is not None that element and the null value are removed. However when I add the or statement to remove the id is not "" the entire file is printed with neither of the id in the or statement removed. Being new to python I am not sure what I am doing wrong. Thank you :).
file
{
"objects": [
{
"version": "1",
"list": [
{
"
"json": null
},
"id": "",
}
],
"h": "na"
},
{
"version": "1",
"list": [
{
"
"json": null
},
"id": "This has text in it and should be printed",
}
],
"h": "na"
},
{
"version": "1",
"list": [
{
"
"json": null
},
"id": null,
}
],
"h": "na"
}
]
}
desired
{
"version": "1",
"list": [
{
"
"json": null
},
"id": "This has text in it and should be printed",
}
],
"h": "na"
},
python3
if v["id"] is not None or v['id'] is not "":
Also tried the below and id were changed:
for obj in data['objects']:
if obj.get('id') is "":
obj['description'] = 'na'
if v["id"] is not None or v['id'] is not "":
No matter what the value is, this statement will always be true.
If the value is None, then the is not "" part will be true.
If the value is an empty string, then the is not None part will be true.
And if the value is anything else, then both parts will be true.
As #Pranav commented, you want to use and here, not or.
Related
I have this sample.json file with me:
{
"details":[
{
"name": "",
"class": "4",
"marks": "72.6"
},
{
"name": "David",
"class": "",
"marks": "78.2"
},
{
"name": "Emily",
"class": "4",
"marks": ""
}
]
}
As you can see for the first one; "name" is string datatype is actually empty.
For the second one; "class" with integer datatype is empty.
And for the third one; "marks" with float datatype is empty.
Now my task is;
to find the fields which are empty, if string is empty replace it with "BLANK", if integer is empty replace it with 0, and if float is empty replace it with 0.0
P.S: I'm doing this with Python like this:
import json
path = open('D:\github repo\python\sample.json')
df = json.load(path)
for i in df["details"]:
print(i["name"])
Also make sure that I don't want to hard-code the values. Coz here if we see there are only 3 fields(name, class, marks) but what if I have more that 3. Then what? How will I find which fields are empty or not?
Like you see here:
{
"code": "AAA",
"lat": "-17.3595",
"lon": "-145.494",
"name": "Anaa Airport",
"city": "Anaa",
"state": "Tuamotu-Gambier",
"country": "French Polynesia",
"woeid": "12512819",
"tz": "Pacific\/Midway",
"phone": "",
"type": "Airports",
"email": "",
"url": "",
"runway_length": "4921",
"elev": "7",
"icao": "NTGA",
"direct_flights": "2",
"carriers": "1"
},
This is just one block, I've N-number of blocks like this. That's why I can't hard_code the values right?
Can anybody help me with it!
Thank You so much!
Since the type info isn't available anywhere programmatically, and there seem to be only three hard-coded fields, I'd just check each of them explicitly.
Short-circuiting with the or operator would even allow you to achieve this fairly elegantly:
for d in df['details']:
d['name'] = d['name'] or 'BLANK'
d['class'] = d['class'] or '0'
d['marks'] = d['marks'] or '0.0'
You could check whether the string is empty with a simple if statement like so.
if not i['name'] == ""
Alternatively, you could also do
if not i['name']
The second if statement makes use of falsy and truthy values in Python. Here's a link to read more about it
You could create a dictionary empty_replacements mapping each key to its corresponding desired empty value:
import json
sample_json = {
"details": [
{
"name": "",
"class": "4",
"marks": "72.6"
},
{
"name": "David",
"class": "",
"marks": "78.2"
},
{
"name": "Emily",
"class": "4",
"marks": ""
}
]
}
empty_replacements = {"name": "BLANK", "class": "0", "marks": "0.0"}
sample_json["details"] = [{
k: v if v else empty_replacements[k]
for k, v in d.items()
} for d in sample_json["details"]]
print('sample_json after replacements: ')
print(json.dumps(
sample_json,
sort_keys=False,
indent=4,
))
Output:
sample_json after replacements:
{
"details": [
{
"name": "BLANK",
"class": "4",
"marks": "72.6"
},
{
"name": "David",
"class": "0",
"marks": "78.2"
},
{
"name": "Emily",
"class": "4",
"marks": "0.0"
}
]
}
I 'm assuming by the dictionary which you provided that marks & class are stored as String.
li=[]
for d in df["details"]:
for k,v in d.items():
if (v==''):
if (k=='name'):
d[k]="BLANK"
elif (k=='class') :
d[k]='0'
elif (k=='marks'):
d[k]='0.0'
li.append(d)
df['details']=li
I would like to find a way to convert a string inside my JSON object into another JSON object (nested) with the use of Python and thus add a key to the values inside my string that are separated by the use of a ";".
My current JSON object:
{
"reference": "#############",
"messageContext": "",
"from": {
"number": "+#############",
"name": ""
},
"to": {
"number": "#############"
},
"message": {
"text": "12302;8;6245;d517666e-41ca-459a-9b35-c49386df537b;2;2;50.8447;-4.3614;2021-04-28T22:24:12.204Z;rec123;PRD",
"media": {
"mediaUri": "",
"contentType": "",
"title": ""
},
"custom": {}
},
"groupings": [
"",
"",
""
],
"time": "2021-05-02 14:03:22",
"timeUtc": "2021-05-02T12:03:22",
"channel": "Sms"
}
The format I would try to have is something like this, where I would change the text to an object while adding key values.
Result, which I try to obtain (Inside of text:" "):
{
"reference": ""#############",",
"messageContext": "",
"from": {
"number": "+"#############",",
"name": ""
},
"to": {
"number": ""#############","
},
"message": {
"text": {
"APSI": "12302",
"idVhl": 8,
"idDrv": 6245,
"GUID": "d517666e-41ca-459a-9b35-c49386df537b",
"idLne": 2,
"idSvc": 2,
"Lat": 50.8447,
"Lon": -4.3614,
"Timestamp": "2021-04-28T22:24:12.204Z",
"Rec": "rec123",
"Env": "PRD"
},
"media": {
"mediaUri": "",
"contentType": "",
"title": ""
},
"custom": {}
},
"groupings": [
"",
"",
""
],
"time": "2021-05-02 14:03:22",
"timeUtc": "2021-05-02T12:03:22",
"channel": "Sms"
}
Given your JSON object as input this function will return your expected output.
def transform(input):
text = input['message']['text']
text_list = text.split(';')
text_dict = {
'APSI': text_list[0],
'idVhl': int(text_list[1]),
'idDrv': int(text_list[2]),
'GUID': text_list[3],
'idLne': int(text_list[4]),
'idSvc': int(text_list[5]),
'Lat': float(text_list[6]),
'Lon': float(text_list[7]),
'Timestamp': text_list[8],
'Rec': text_list[9],
'Env': text_list[10]
}
input['message']['text'] = text_dict
return input
This is done by splitting your text on the ; character which returns a list of all the values you want. You can then parse the list into a dictionary and then assign that dictionary to your original text value.
I want to print a user from a JSON list into Python that I select however I can only print all the users. How do you print a specific user? At the moment I have this which prints all the users out in a ugly format
import json
with open('Admin_sample.json') as f:
admin_json = json.load(f)
print(admin_json['staff'])
The JSON file looks like this
{
"staff": [
{
"id": "DA7153",
"name": [
"Fran\u00c3\u00a7ois",
"Ullman"
],
"department": {
"name": "Admin"
},
"server_admin": "true"
},
{
"id": "DA7356",
"name": [
"Bob",
"Johnson"
],
"department": {
"name": "Admin"
},
"server_admin": "false"
},
],
"assets": [
{
"asset_name": "ENGAGED SLOTH",
"asset_type": "File",
"owner": "DA8333",
"details": {
"security": {
"cia": [
"HIGH",
"INTERMEDIATE",
"LOW"
],
"data_categories": {
"Personal": "true",
"Personal Sensitive": "true",
"Customer Sensitive": "true"
}
},
"retention": 2
},
"file_type": "Document",
"server": {
"server_name": "ISOLATED UGUISU",
"ip": [
10,
234,
148,
52
]
}
},
{
"asset_name": "ISOLATED VIPER",
"asset_type": "File",
"owner": "DA8262",
"details": {
"security": {
"cia": [
"LOW",
"HIGH",
"LOW"
],
"data_categories": {
"Personal": "false",
"Personal Sensitive": "false",
"Customer Sensitive": "true"
}
},
"retention": 2
},
},
]
I just can't work it out. Any help would be appreciated.
Thanks.
You need to index into the staff list, e.g.:
print(admin_json['staff'][0])
I suggest reading up a bit on dictionaries in Python. Dictionary values can be set to any object: in this case, the value of the staff key is set to a list of dicts. Here's an example that will loop through all the staff members and print their names:
staff_list = admin_json['staff']
for person in staff_list:
name_parts = person['name']
full_name = ' '.join(name_parts) # combine name parts into a string
print(full_name)
Try something like this:
import json
def findStaffWithId(allStaff, id):
for staff in allStaff:
if staff["id"] == id:
return staff
return {} # no staff found
with open('Admin_sample.json') as f:
admin_json = json.load(f)
print(findStaffWithId(admin_json['staff'], "DA7356"))
You can list all the users name with
users = [user["name"] for user in admin_json['staff']]
You have two lists in this JSON file. When you try to parse it, you'll be reach a list. For example getting the first staff id:
print(admin_json['staff'][0]['id'])
This will print:
DA7153
When you use "json.loads" this will simply converts JSON file to the Python dictionary. For further info:
https://docs.python.org/3/tutorial/datastructures.html#dictionaries
I have a csv file and trying to compose JSON from it. There are mulitple records in a file but I am just giving one set of sample records here.This structure is driven on the claimID. There is nesting on the claimLineDetail and claimSpecDiag.I guess I have to create some sort of list to handle this then the problem is how am I going to append it in the required structure. I really need some guidance here to achieve the desired result. Is it possible to break out different sections and append it later, I am not sure just assuming, as there are multiple columns.
Code :
import csv,json
data = []
with open('JsonRequestPricingMedical.csv','r') as f:
reader = csv.DictReader(f)
for row in reader:
print row
csv file :
claimId,subscriberId,claimType,claimSubType,providerId,totalChargeAmt,claimLineNo,pos_code,procedureCode,subdiagnosisCode,svcLineFromDt,svcLineToDt,chargedAmt,clmLineUnits,presentOnAdmit,diagnosisCode
18A000730400,101924200,M,M,002664514003,585,1,11,92014,H43393,2017-06-19,2017-06-19,160,1,U,H43393
18A000730400,101924200,M,M,002664514003,585,2,12,92015,H43395,2017-06-19,2017-06-19,160,2,U,H43394
Desired JSON
[
{
"claimsHeader":" {
"claimId": "18A000730400",
"subscriberId": "101924200",
"claimType":{
"code": "M"
},
"claimSubType": {
"code": "M"
},
"providerId" :"002664514003",
"totalChargeAmt": "585",
"claimLineDetail" :[
{
"claimLineNo": "1",
"placeOfService": {
"code": "11"
},
"procedureCode": {
"code": "92014"
},
"subDiagnosisCd": {
"code": "H43393"
},
"svcLineFromDt": "2017-06-19",
"svcLineToDt": "2017-06-19",
"chargedAmt": "160",
"clmLineUnits": "1",
},
{
"claimLineNo": "2",
"placeOfService": {
"code": "12"
},
"procedureCode": {
"code": "92015"
},
"subDiagnosisCd": {
"code": "H433945
},
"svcLineFromDt": "2017-06-19",
"svcLineToDt": "2017-06-19",
"chargedAmt": "160",
"clmLineUnits": "2",
}
],
{
"claimSpecDiag": [
"presentOnAdmit": "",
"diagnosisCode": "H43393",
},
{
"presentOnAdmit": "",
"diagnosisCode": "H43394",
}
]
}
]
When you read a csv, each line represents variables separated by a special char, in your case, comas: ",".
You can get each variable separated by doing line_variables = row.split(',')
Just pass the first line, and for all the other, do something like:
result = {
"claimsHeader":" {
"claimId": line_variables[0],
"subscriberId": line_variables[1],
"claimType":{
"code": line_variables[2]
}
...
Finaly, just add the result to a list (created just before your for loop) with your_list.append(result).
Python newbie:
Default.json
{
"name": {
"provide": ""
},
"test": {
"Fail": {
"centers": None,
"Nearest": 0
},
"far": "",
"Meta": null,
"Only": false,
"Tags": null
},
"Session": "",
"conf": {
"check": "",
"Reg": ""
},
"Token": ""
}
Remote.json
[ {
'name': {
'provide': ''
},
'Name': 'abc',
'test': {
'Service': 'redis',
'Tags': [
'stage'
],
'Fail': {
'centers': None,
'Nearest': 3
},
'Only': false,
'far': '',
'Meta': null
},
'Token': '',
'Session': '',
'conf': {
'Reg': '',
'check': 'name_prefix_match'
},
} ]
I have a default.json and remote .json .Task I am trying to achieve is remove all the json elements from the remote.json for whom the values of remote.json matches with the default.json.For example since the key,value of name:{provider=""} from default.json matches the name:{provider=""} from remote.json.It should get removed from remote.json
with open(remote.json) as f:
with open(default.json) as m:
file=json.load(f)
default=json.load(m)
for i in xrange(len(file)):
for key,value in default.items():
#default[key]=value
#a=filter(lambda x: x[""],file.keys())
1.I am not getting the idea here how to get key, value from default and compare it with file?Any help would be appreciated.
The reason I need to remove element from remote.json is because I need to compare the resultant json with other json file "local.json".If i dont' remove the key ,values with value "" or null or None then the comparison between remote.json and local.json would never be equal.
2.Is there any better way of going for this problem?
local.json
{
"Name": "",
"conf": {
"check": "name_prefix_match",
},
"test": {
"Service": "redis",
"Fail": {
"Near": 3
},
"Tags": ""
}
}
There are some problems with JSON examples because of None & False are not valid JSON objects (and so are single-quoted string literals), so let's pretend that we've already parsed files and got something like
default_json = {
"name": {
"provide": ""
},
"test": {
"Fail": {
"centers": None,
"Nearest": 0
},
"far": "",
"Meta": None,
"Only": False,
"Tags": None
},
"Session": "",
"conf": {
"check": "",
"Reg": ""
},
"Token": ""
}
remote_json = [{
"name": {
"provide": ""
},
"Name": "abc",
"test": {
"Service": "redis",
"Tags": [
"stage"
],
"Fail": {
"centers": None,
"Nearest": 3
},
"Only": False,
"far": "",
"Meta": None
},
"Token": "",
"Session": "",
"conf": {
"Reg": "",
"check": "name_prefix_match"
},
}]
Assuming that remote.json is the list of dictionaries & each one of them should be filtered out using default.json:
filtered_remote_json = [dict(item
for item in dictionary.items()
if item not in default_json.items())
for dictionary in remote_json]
will give us
filtered_remote_json == [{"Name": "abc",
"test": {"Service": "redis", "Tags": ["stage"],
"Fail": {"centers": None,
"Nearest": 3}, "Only": False,
"far": "", "Meta": None},
"conf": {"Reg": "",
"check": "name_prefix_match"}}]
EDIT
if we need to filter sub-dictionaries as well, then next a bit nasty utility function should help
def filter_defaults(json_object, default_json_object):
result = {}
for key, value in json_object.items():
try:
default_value = default_json_object[key]
except KeyError:
# key not in defaults, adding to result
result[key] = value
continue
# we need to process sub-dictionaries as well
if isinstance(value, dict):
value = filter_defaults(value, default_value)
# we are not interested in empty filtered sub-dictionaries
if not value:
continue
# value should differ from default
elif value == default_value:
continue
result[key] = value
return result
then just write
filtered_remote_json = [filter_defaults(dictionary, default_json)
for dictionary in remote_json]
this will give us
filtered_remote_json == [{"Name": "abc",
"test": {"Service": "redis", "Tags": ["stage"],
"Fail": {"Nearest": 3}},
"conf": {"check": "name_prefix_match"}}]