We Keep Coding

Reading a JSON file in python, reading a value and assigning it to a variable [duplicate]

This question already has answers here:
How can I parse (read) and use JSON?
(5 answers)
Closed last month.
I have a json file which looks like
{
"0": {
"id": 1700388,
"title": "Disconnect: The Wedding Planner",
"year": 2023,
"imdb_id": "tt24640474",
"tmdb_id": 1063242,
"tmdb_type": "movie",
"type": "movie"
},
"1": {
"id": 1631017,
"title": "The Pale Blue Eye",
"year": 2022,
"imdb_id": "tt14138650",
"tmdb_id": 800815,
"tmdb_type": "movie",
"type": "movie"
},
"2": {
"id": 1597915,
"title": "The Man from Toronto",
"year": 2022,
"imdb_id": "tt11671006",
"tmdb_id": 667739,
"tmdb_type": "movie",
"type": "movie"
},
I am trying to read this and extract the "tmbd_id" to store as a variable. I then intend to inject this into a url for an api get request.
The next step is to add the response parameters to the json file. Then adding it all into a loop. There are 1000 entries.
I have been trying to use other answers but I suspect the nature of my json is causing the issue with the integer name causing issues. It has been giving me this error
for i in data['0']:
TypeError: list indices must be integers or slices, not str

You can use json.loads to read the file's content and then iterate to find those id's:
import json
tmdb_ids = []
with open('test.json', 'r') as json_fp:
imdb_info = json.load(json_fp)[0]
for dict_index, inner_dict in imdb_info.items():
tmdb_ids.append(inner_dict['tmdb_id'])
print(tmdb_ids)
Result:
[1063242, 800815, 667739]
You can make it more sophisticated if you need a specific tmdb_id from a specific "movie" by adding some if statements to the for loop, choosing movies you want.
Edit
I've noticed this is an array of dictionaries, thus we can iterate over each "movie chunk" and extract the tmdb_id from each one:
with open('test.json', 'r') as json_fp:
imdb_info = json.load(json_fp)
tmdb_ids = [movie_info['tmdb_id'] for movies_chunk in imdb_info for movie_index, movie_info in movies_chunk.items()]

for reading json:
import json
with open("file.json","r") as file:
text = json.load(file)
for tmbd_id:
for i in text:
print(text[i]["tmdb_id"])

Is there a way to just grab one subset of json data from a large text file?

I'm looking to pull the "name" field from a large json text file and be able to store them in another file for later, but I'm getting every piece of data that was in my previous json file albeit slightly modified. How do I make it so I only grab the data after the "name": field in my json file?
I've tried
names = []
with open('./out.json', 'r') as f:
data = json.load(f)
for name in data:
names.append(data[name])
with open('./names.json','w') as f:
for name in names:
f.write('%s\r\n' % name)
and I'm getting my exact json file back, with no formatting and u' in front of everything, likely from the json.load(f), but I have no idea how to remedy this.
my text file is formatted like this, if it matters:
{
"array":[
{
"name": "Seranul",
"id": 5,
"type": "Paladin",
"itemLevel": 414,
"icon": "Paladin-Holy",
"total": 11107150,
"activeTime": 2205387,
"activeTimeReduced": 2205387
},
{
"name": "Contherious",
"id": 9,
"type": "Hunter",
"itemLevel": 412,
"icon": "Hunter-Marksmanship",
"total": 51102811,
"activeTime": 2637303,
"activeTimeReduced": 2637303
},
{
"name": "Unicorns",
"id": 17,
"type": "Priest",
"itemLevel": null,
"icon": "Priest",
"total": 12252005,
"activeTime": 1768883,
"activeTimeReduced": 1761797
},
...
}
]}
I'm expecting to see the corresponding data for each name field, but I'm getting my entire document back.

It looks like your code is ignoring the structure of the JSON data. Specifically, you are iterating through the keys in the JSON dictionary, which is just array, and then appending the value to you names list. This results in the whole array property being put into your names variable.
Here is what I believe you want: iterate through the entries in array and and them to a list, then export that as JSON to another file.
import json
names = []
with open('./out.json', 'r') as f:
data = json.load(f)
for entry in data["array"]:
names.append(entry["name"])
with open('./names.json', 'w') as f:
f.write(json.dumps(names))
This will result in the following JSON in names.json:
["Seranul", "Contherious", "Unicorns"]

TypeError: string indices must be integers // working with JSON as dict in python

Okay, so I've been banging my head on this for the last 2 days, with no real progress. I am a beginner with python and coding in general, but this is the first issue I haven't been able to solve myself.
So I have this long file with JSON formatting with about 7000 entries from the youtubeapi.
right now I want to have a short script to print certain info ('videoId') for a certain dictionary key (refered to as 'key'):
My script:
import json
f = open ('path file.txt', 'r')
s = f.read()
trailers = json.loads(s)
print(trailers['key']['Items']['id']['videoId'])
# print(trailers['key']['videoId'] gives same response
Error:
print(trailers['key']['Items']['id']['videoId'])
TypeError: string indices must be integers
It does work when I want to print all the information for the dictionary key:
This script works
import json
f = open ('path file.txt', 'r')
s = f.read()
trailers = json.loads(s)
print(trailers['key'])
Also print(type(trailers)) results in class 'dict', as it's supposed to.
My JSON File is formatted like this and is from the youtube API, youtube#searchListResponse.
{
"kind": "youtube#searchListResponse",
"etag": "",
"nextPageToken": "",
"regionCode": "",
"pageInfo": {
"totalResults": 1000000,
"resultsPerPage": 1
},
"items": [
{
"kind": "youtube#searchResult",
"etag": "",
"id": {
"kind": "youtube#video",
"videoId": ""
},
"snippet": {
"publishedAt": "",
"channelId": "",
"title": "",
"description": "",
"thumbnails": {
"default": {
"url": "",
"width": 120,
"height": 90
},
"medium": {
"url": "",
"width": 320,
"height": 180
},
"high": {
"url": "",
"width": 480,
"height": 360
}
},
"channelTitle": "",
"liveBroadcastContent": "none"
}
}
]
}
What other information is needed to be given for you to understand the problem?

The following code gives me all the videoId's from the provided sample data (which is no id's at all in fact):
import json
with open('sampledata', 'r') as datafile:
data = json.loads(datafile.read())
print([item['id']['videoId'] for item in data['items']])
Perhaps you can try this with more data.
Hope this helps.

I didn't really look into the youtube api but looking at the code and the sample you gave it seems you missed out a [0]. Looking at the structure of json there's a list in key items.
import json
f = open ('json1.json', 'r')
s = f.read()
trailers = json.loads(s)
print(trailers['items'][0]['id']['videoId'])

I've not used json before at all. But it's basically imported in the form of dicts with more dicts, lists etc. Where applicable. At least from my understanding.
So when you do type(trailers) you get type dict. Then you do dict with trailers['key']. If you do type of that, it should also be a dict, if things work correctly. Working through the items in each dict should in the end find your error.
Pythons error says you are trying find the index/indices of a string, which only accepts integers, while you are trying to use a dict. So you need to find out why you are getting a string and not dict when using each argument.
Edit to add an example. If your dict contains a string on key 'item', then you get a string in return, not a new dict which you further can get a dict from. item in the json for example, seem to be a list, with dicts in it. Not a dict itself.

Loading JSON file for reading and selecting data

I have a json file that I load into python. I want to take a keyword from the file (which is very big), like country rank or review from info taken from the internet. I tried
json.load('filename.json')
but I am getting an error:
AttributeError: 'str' object has no attribute 'read.'
What am I doing wrong?
Additionally, how do I select part of a json file if it is very big?

I think you need to open the file then pass that to json load like this
import json
from pprint import pprint
with open('filename.json') as data:
output = json.load(data)
pprint(output)

Try the following:
import json
json_data_file = open("json_file_path", 'r').read() # r for reading the file
json_data = json.loads(json_data_file)
Access the data using the keys as follows :
json_data['key']

json.load() expects the file handle after it has been opened:
with open('filename.json') as datafile:
data = json.load(datafile)
For example if your json data looked like this:
{
"maps": [
{
"id": "blabla",
"iscategorical": "0"
},
{
"id": "blabla",
"iscategorical": "0"
}
],
"masks": {
"id": "valore"
},
"om_points": "value",
"parameters": {
"id": "valore"
}
}
To access parts of the data, use:
data["maps"][0]["id"]
data["masks"]["id"]
data["om_points"]
That code can be found in this SO answer:
Parsing values from a JSON file using Python?

Unable to convert returned JSON data into Dict in Scrapy

I am using scrapy to get a certain piece of data from here. As I was suggested here, I used the following code in my script:
pattern = re.compile(r"qubit_product_list = (.*?);", re.M)
script = hxs.select("//script[contains(., 'qubit_product_list')]/text()").extract()[0]
data = pattern.search(script).group(1)
j_data = json.loads(data)
self.log('After calling LOAD Begins')
self.log(j_data) #It is not printing ANYTHING!!!!
self.log('After calling LOAD Ends')
self.log('\n---------------------------------\n')
Which outputs following from variable data:
{
"9102-DBL-sprung slat base": {
"id": "9102",
"name": "Imperial Bedstead",
"url": "/p/Imperial_Bedstead.htm",
"description": "Double - Sprung Slat Base",
"unit_price": 429.99,
"unit_sale_price": 429.99,
"currency": "GBP",
"sku_code": "BENT:1320B-Beech",
"category": "Bed Frames",
"stock": 100
},
"9102-KS-sprung slat base": {
"id": "9102",
"name": "Imperial Bedstead",
"url": "/p/Imperial_Bedstead.htm",
"description": "Kingsize - Sprung Slat Base",
"unit_price": 439.98996,
"unit_sale_price": 439.98996,
"currency": "GBP",
"sku_code": "BENT:1326B-Beech",
"category": "Bed Frames",
"stock": 100
}
}
Now, I want to convert this json like structure to python dict. I tried following but it returns unicode type.
j_data = json.loads(data)
So, how do I get Array/Dict in Python 2.7? Ironically same loads method is returning of type dict when using scrapy shell.

Try this:
#typecasting the JSON to string for json.loads to work
data = str(data)
#returning type dict from json
j_data = json.loads(data)
#typecasting the dict to string before writing to log
self.log(str(j_data))