We Keep Coding

Read nested Json File and fetch the required field

I have the following json file, Need to loop over to fetch the company and its file.
data.json
[
{
"count": 0,
"company": "abc",
"state": {
"city": {}
},
"name": "Techno",
"age": 17,
"file": "cubix.so"
},
{
"count": 12,
"company": "def",
"state": {
"city": {}
},
"name": "Uni",
"age": 17,
"file": "cyp.so"
},
{
"count": 22,
"company": "ghi",
"state": {
"city": {}
},
"name": "unicorn",
"age": 17,
"file": "omg.so"
}
]
Need to loops through json file and print the file with its company.
import json
import sys
f=open('data.json')
print(json.dumps(output))
Expected Output:
file with respect to company

Use json.loads() to load from JSON content and context managers while working with files:
import json
with open('data.json') as file:
data = json.loads(file.read())
for obj in data:
print(obj['company'], obj['file'])
Output:
abc cubix.so
def cyp.so
ghi omg.so

After reading the file, you could use json.load() to load the object, which is a list. Then you can iterate over the list and print the respective keys 'company' and 'file'
import json
f = open('data.json')
for e in json.load(f):
print(e['company'], e['file'])
f.close()
output
abc cubix.so
def cyp.so
ghi omg.so

Iterate and update through folder of .json files and update value in python

I've struck out trying to find a suitable script to iterate through a folder of .json files and update a single line.
Below is an example json file located in a path among others. I would like to iterate through the json files in a folder containing several files like this with various information and update the "seller_fee_basis_points" from "0" to say "500" and save.
Would really appreciate the assistance.
{
"name": "Solflare X NFT",
"symbol": "",
"description": "Celebratory Solflare NFT for the Solflare X launch",
"seller_fee_basis_points": 0,
"image": "https://www.arweave.net/abcd5678?ext=png",
"animation_url": "https://www.arweave.net/efgh1234?ext=mp4",
"external_url": "https://solflare.com",
"attributes": [
{
"trait_type": "web",
"value": "yes"
},
{
"trait_type": "mobile",
"value": "yes"
},
{
"trait_type": "extension",
"value": "yes"
}
],
"collection": {
"name": "Solflare X NFT",
"family": "Solflare"
},
"properties": {
"files": [
{
"uri": "https://www.arweave.net/abcd5678?ext=png",
"type": "image/png"
},
{
"uri": "https://watch.videodelivery.net/9876jkl",
"type": "unknown",
"cdn": true
},
{
"uri": "https://www.arweave.net/efgh1234?ext=mp4",
"type": "video/mp4"
}
],
"category": "video",
"creators": [
{
"address": "SOLFLR15asd9d21325bsadythp547912501b",
"share": 100
}
]
}
}
Updated with an answer due to #JCaesar's help
import json
import glob
import os
SOURCE_DIRECTORY = r'my_favourite_directory'
KEY = 'seller_fee_basis_points'
NEW_VALUE = 500
for file in glob.glob(os.path.join(SOURCE_DIRECTORY, '*.json')):
json_data = json.loads(open(file, encoding="utf8").read())
# note that using the update method means
# that if KEY does not exist then it will be created
# which may not be what you want
json_data.update({KEY: NEW_VALUE})
json.dump(json_data, open(file, 'w'), indent=4)

I recommend using glob to find the files you're interested in. Then utilise the json module for reading and writing the JSON content.
This is very concise and has no sanity checking / exception handling but you should get the idea:
import json
import glob
import os
SOURCE_DIRECTORY = 'my_favourite_directory'
KEY = 'seller_fee_basis_points'
NEW_VALUE = 500
for file in glob.glob(os.path.join(SOURCE_DIRECTORY, '*.json')):
json_data = json.loads(open(file).read())
# note that using the update method means
# that if KEY does not exist then it will be created
# which may not be what you want
json_data.update({KEY: NEW_VALUE})
json.dump(json_data, open(file, 'w'), indent=4)

AttributeError 'str' object has no attribute 'keys'

Im getting this error while executing my code TypeError: string indices must be integers. Before that im doing the same thing but its working.
for con in bin3 ['content']['annotations']:
print(con) #This loop is working fine.
This is my python code
import pandas as pd
import json
filename = 'new4.json'
for line in open(filename, 'r'):
print(line)
for item in filename['content']['transform']: #this loop is not working
print(item)
This is my json file
{
"content": {
"transform": [
{
"ID": 5,
"class": 6,
"createTime": "",
"name": "Source",
"advancedProperties": [
{
"ID": 82,
"class": 12,
"name": "Tracing",
"value": "Normal"
}]
}]
}
}

Do this:
Correct way to use a JSON file in Python is to convert it into a Dictionary first - json.load() does it for you
import json
filename = 'new4.json'
with open(filename) as json_file:
data = json.load(json_file)
print(data['content']['transform'])
NOTE: Your Json file is not in Right Format - Closing Braces are missing
Correct JSON Format:
{
"content": {
"transform": [
{
"ID": 5,
"class": 6,
"createTime": "",
"name": "Source",
"advancedProperties": [
{
"ID": 82,
"class": 12,
"name": "Tracing",
"value": "Normal"
}
]
}
]
}
}

Convert complex Json to CSV

The file is from a slack server export file, so the structure varies every time (if people responded to a thread with text or reactions).
I have tried several SO questions, with similar problems. But I guarantee my question is different. This one, This one too,This one as well
Sample JSON file:
"client_msg_id": "f347abdc-9e2a-4cad-a37d-8daaecc5ad51",
"type": "message",
"text": "I came here just to check <#U3QSFG5A4> This is a sample :slightly_smiling_face:",
"user": "U51N464MN",
"ts": "1550511445.321100",
"team": "T1559JB9V",
"user_team": "T1559JB9V",
"source_team": "T1559JB9V",
"user_profile": {
"avatar_hash": "gcc8ae3d55bb",
"image_72": "https:\/\/secure.gravatar.com\/avatar\/fcc8ae3d55bb91cb750438657694f8a0.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0026-72.png",
"first_name": "A",
"real_name": "a name",
"display_name": "user",
"team": "T1559JB9V",
"name": "name",
"is_restricted": false,
"is_ultra_restricted": false
},
"thread_ts": "1550511445.321100",
"reply_count": 3,
"reply_users_count": 3,
"latest_reply": "1550515952.338000",
"reply_users": [
"U51N464MN",
"U8DUH4U2V",
"U3QSFG5A4"
],
"replies": [
{
"user": "U51N464MN",
"ts": "1550511485.321200"
},
{
"user": "U8DUH4U2V",
"ts": "1550515191.337300"
},
{
"user": "U3QSFG5A4",
"ts": "1550515952.338000"
}
],
"subscribed": false,
"reactions": [
{
"name": "trolldance",
"users": [
"U51N464MN",
"U4B30MHQE",
"U68E6A0JF"
],
"count": 3
},
{
"name": "trollface",
"users": [
"U8DUH4U2V"
],
"count": 1
}
]
},
The issue is that there are several keys that vary, so the structure changes within the same json file between messages depending on how other users interact to a given message.

with open("file.json") as file:
d = json.load(file)
df = pd.io.json.json_normalize(d)
df.columns = df.columns.map(lambda x: x.split(".")[-1])

CSV file to JSON for nested array generic template using python (for csv to mongodb insert)

I want to create the JSON file from CSV file using the generic python script.
Found hone package from GitHub but some of the functionalities missing in that code.
csv to json
I want to code like generic template CSV to JSON.
[
{
"birth": {
"day": "7",
"month": "May",
"year": "1985"
},
"name": "Bob",
"reference": "TRUE",
"reference name": "Smith"
}
]
Only handled above type of JSON only.
[
{
"Type": "AwsEc2Instance",
"Id": "i-cafebabe",
"Partition": "aws",
"Region": "us-west-2",
"Tags": {
"billingCode": "Lotus-1-2-3",
"needsPatching": "true"
},
"Details": {
"AwsEc2Instance": {
"Type": "i3.xlarge",
"ImageId": "ami-abcd1234",
"IpV4Addresses": [ "54.194.252.215", "192.168.1.88" ],
"IpV6Addresses": [ "2001:db812341a2b::123" ],
"KeyName": "my_keypair",
"VpcId": "vpc-11112222",
"SubnetId": "subnet-56f5f633",
"LaunchedAt": "2018-05-08T16:46:19.000Z"
}
}
}
]
I want to handle nested array[] ,{}

I have done something like this before and below code can be modified as I have not seen your dataset.
dataframe = pd.read_excel('dataframefilepath', encoding='utf-8', header=0)
'''Adding to list to finally save it as JSON'''
df = []
for (columnName, columnData) in dataframe.iteritems():
if dataframe.columns.get_loc(columnName) > 0:
for indata, rwdata in dataframe.iterrows():
for insav, rwsave in df_to_Save.iterrows():
if rwdata.Selected_Prediction == rwsave.Selected_Prediction:
#print()
df_to_Save.loc[insav, 'Value_to_Save'] = rwdata[dataframe.columns.get_loc(columnName)]
#print(rwdata[dataframe.columns.get_loc(columnName)])
df.append(df_to_Save.set_index('Selected_Prediction').T.to_dict('record'))
df = eval(df)
'''Saving in JSON format'''
path_to_save = '\\your path'
with open(path_to_save, 'w') as json_file:
json.dump(df, json_file)