We Keep Coding

Key 'boot_num' is not recognized when being interpreted from a .JSON file

Currently, I am working on a Boot Sequence in Python for a larger project. For this specific part of the sequence, I need to access a .JSON file (specs.json), establish it as a dictionary in the main program. I then need to take a value from the .JSON file, and add 1 to it, using it's key to find the value. Once that's done, I need to push the changes to the .JSON file. Yet, every time I run the code below, I get the error:
bootNum = spcInfDat['boot_num']
KeyError: 'boot_num'`
Here's the code I currently have:
(Note: I'm using the Python json library, and have imported dumps, dump, and load.)
# Opening of the JSON files
spcInf = open('mki/data/json/specs.json',) # .JSON file that contains the current system's specifications. Not quite needed, but it may make a nice reference?
spcInfDat = load(spcInf)
This code is later followed by this, where I attempt to assign the value to a variable by using it's dictionary key (The for statement was a debug statement, so I could visibly see the Key):
for i in spcInfDat['spec']:
print(CBL + str(i) + CEN)
# Loacting and increasing the value of bootNum.
bootNum = spcInfDat['boot_num']
print(str(bootNum))
bootNum = bootNum + 1
(Another Note: CBL and CEN are just variables I use to colour text I send to the terminal.)
This is the interior of specs.json:
{
"spec": [
{
"os":"name",
"os_type":"getwindowsversion",
"lang":"en",
"cpu_amt":"cpu_count",
"storage_amt":"unk",
"boot_num":1
}
]
}
I'm relatively new with .JSON files, as well as using the Python json library; I only have experience with them through some GeeksforGeeks tutorials I found. There is a rather good chance that I just don't know how .JSON files work in conjunction with the library, but I figure that it would still be worth a shot to check here. The GeeksForGeeks tutorial had no documentation about this, as well as there being minimal I know about how this works, so I'm lost. I've tried searching here, and have found nothing.
Issue Number 2
Now, the prior part works. But, when I attempt to run the code on the following lines:
# Changing the values of specDict.
print(CBL + "Changing values of specDict... 50%" + CEN)
specDict ={
"os":name,
"os_type":ost,
"lang":"en",
"cpu_amt":cr,
"storage_amt":"unk",
"boot_num":bootNum
}
# Writing the product of makeSpec to `specs.json`.
print(CBL + "Writing makeSpec() result to `specs.json`... 75%" + CEN)
jsonobj = dumps(specDict, indent = 4)
with open('mki/data/json/specs.json', "w") as outfile:
dump(jsonobj, outfile)
I get the error:
TypeError: Object of type builtin_function_or_method is not JSON serializable.
Is there a chance that I set up my dictionary incorrectly, or am I using the dump function incorrectly?

You can show the data using:
print(spcInfData)
This shows it to be a dictionary, whose single entry 'spec' has an array, whose zero'th element is a sub-dictionary, whose 'boot_num' entry is an integer.
{'spec': [{'os': 'name', 'os_type': 'getwindowsversion', 'lang': 'en', 'cpu_amt': 'cpu_count', 'storage_amt': 'unk', 'boot_num': 1}]}
So what you are looking for is
boot_num = spcInfData['spec'][0]['boot_num']
and note that the value obtained this way is already an integer. str() is not necessary.
It's also good practice to guard against file format errors so the program handles them gracefully.
try:
boot_num = spcInfData['spec'][0]['boot_num']
except (KeyError, IndexError):
print('Database is corrupt')
Issue Number 2
"Not serializable" means there is something somewhere in your data structure that is not an accepted type and can't be converted to a JSON string.
json.dump() only processes certain types such as strings, dictionaries, and integers. That includes all of the objects that are nested within sub-dictionaries, sub-arrays, etc. See documentation for json.JSONEncoder for a complete list of allowable types.

Converting Python Dict to JSON for MySQL field of JSON type

I am currently getting this error:
Invalid JSON text: "not a JSON text, may need CAST" at position 0 in value for column
This is the value that is trying to be inserted:
{
"ath": 69045,
"ath_date": "2021-11-10T14:24:11.849Z",
"atl": 67.81,
"atl_date": "2013-07-06T00:00:00.000Z"
}
When trying to insert into my database. I believe it is due to malformed JSON however I am using json.dumps() to convert my dictionary. I have tried several things I have found over the last few hours to try and format it correctly but am hitting a wall between two errors.
I tried adding another level as well as wrapping it all in an array as that was recommended in another question, however, that produced the same error.
My Dict:
ticker_market_data[ticker] = {
"all_time": {
"ath": market_data["ath"]["usd"],
"ath_date": market_data["ath_date"]["usd"],
"atl": market_data["atl"]["usd"],
"atl_date": market_data["atl_date"]["usd"],
},
"price_change_percent": {
"1h": market_data["price_change_percentage_1h_in_currency"]["usd"],
"24h": market_data["price_change_percentage_24h"],
"7d": market_data["price_change_percentage_7d"],
"30d": market_data["price_change_percentage_30d"],
"1y": market_data["price_change_percentage_1y"],
},
}
The problem items being all_time and price_change_percent.
This is how I am creating the variables to store in the database:
all_time = json.dumps(ticker_market_data[ticker].get("all_time"))
price_change_percent = json.dumps(ticker_market_data[ticker].get("price_change_percent"))

I wanted to answer my own question though I don't know how much it will benefit the community as it was completely on me. My code posted above was entirely correct, the problem lay in the order of variables being inserted in the SQL. I had the wrong type of variable one position up from where it needed to be. So rather than JSON being inserted into the column it was a float.

Check if JSON var has nullable key (Twitter Streaming API)

I'm downloading tweets from Twitter Streaming API using Tweepy. I manage to check if downloaded data has keys as 'extended_tweet', but I'm struggling with an specific key inside another key.
def on_data(self, data):
savingTweet = {}
if not "retweeted_status" in data:
dataJson = json.loads(data)
if 'extended_tweet' in dataJson:
savingTweet['text'] = dataJson['extended_tweet']['full_text']
else:
savingTweet['text'] = dataJson['text']
if 'coordinates' in dataJson:
if 'coordinates' in dataJson['coordinates']:
savingTweet['coordinates'] = dataJson['coordinates']['coordinates']
else:
savingTweet['coordinates'] = 'null'
I'm checking 'extended_key' propertly, but when I try to do the same with ['coordinates]['coordinates] I get the following error:
TypeError: argument of type 'NoneType' is not iterable
Twitter documentation says that key 'coordinates' has the following structure:
"coordinates":
{
"coordinates":
[
-75.14310264,
40.05701649
],
"type":"Point"
}
I achieved to solve it by just putting the conflictive check in a try, except, but I think this is not the most suitable approach to the problem. Any other idea?

So the twitter API docs are probably lying a bit about what they return (shock horror!) and it looks like you're getting a None in place of the expected data structure. You've already decided against using try, catch, so I won't go over that, but here are a few other suggestions.
Using dict get() default
There are a couple of options that occur to me, the first is to make use of the default ability of the dict get command. You can provide a fall back if the expected key does not exist, which allows you to chain together multiple calls.
For example you can achieve most of what you are trying to do with the following:
return {
'text': data.get('extended_tweet', {}).get('full_text', data['text']),
'coordinates': data.get('coordinates', {}).get('coordinates', 'null')
}
It's not super pretty, but it does work. It's likely to be a little slower that what you are doing too.
Using JSONPath
Another option, which is likely overkill for this situation is to use a JSONPath library which will allow you to search within data structures for items matching a query. Something like:
from jsonpath_rw import parse
matches = parse('extended_tweet.full_text').find(data)
if matches:
print(matches[0].value)
This is going to be a lot slower that what you are doing, and for just a few fields is overkill, but if you are doing a lot of this kind of work it could be a handy tool in the box. JSONPath can also express much more complicated paths, or very deeply nested paths where the get method might not work, or would be unweildy.
Parse the JSON first!
The last thing I would mention is to make sure you parse your JSON before you do your test for "retweeted_status". If the text appears anywhere (say inside the text of a tweet) this test will trigger.
JSON parsing with a competent library is usually extremely fast too, so unless you are having real speed problems it's not necessarily worth worrying about.

Check if a value in a JSON object from 2 different JSON files are similar and create a list of matching results with python

I'm trying to minimize the data i would send to an Application from an API but some values are null but can be found in another API, so i thought about making a python script run on a server to add those null results to the original JSON file.
I'm appending the list that has that information stored to the list that matches it in the original JSON file, this can be done by using a unique ID that corresponds to a video game title in both those files. Here's my code:
import json
games = open('outputgames.json')
releases = open('outputreleases.json')
games_json = json.load(games)
releases_json = json.load(releases)
# This is where all the results are found in the JSON file
# The results are all stored in a list, so to access the first result
# we would access it like this: games_json['results'][0] or games_data[0]
games_data = games_json['results']
releases_data = releases_json['results]
#This is where, i iterate through the data to see IF the id in the object 'game' which is found in releases_data
#is similar to the one in games_data and then storing both matching results in a Dictionary and a list
#then i just dump the results to a json file.
grouped_data = [dict(data_releases = x, data_games= i ) for x in releases_data for i in games_data if i['id'] == x['game']['id']]
with open('final_results.json', mode = 'w') as f:
json.dump(grouped_data, f)
The initial list in games_data['results'] holds about 480 results while the one in releases_data['results'] holds 470. But for some reason, my code seems to be skipping through some results, I'm supposed to be receiving about 480 results but i'm only getting about 260 results. I'm guessing the iteration i am doing with the "IF" statement here is skipping some ids it already passed, but i'm not sure. If someone can help me make the IF statement not resume from where it left but from the top and actually check if ALL ids match.
If someone can please help me with this issue i am having, or if i am doing something wrong. Any help is nice, Thanks in advance.
Here's a sample of what Grouped_data would return, this is only 1 entry. it returns about 260 when run with the json files, but like i said previously i am supposed to get hundreds more returned :
[{"data_games": {"deck": "Tri Force Heroes is a co-op game set in The Legend of Zelda franchise. Three Links must work together to rid the land of Hyrule of evil once more.", "image": {"tiny_url": "http://static.giantbomb.com/uploads/square_mini/8/82063/2778000-tloztfh.jpg", "medium_url": "http://static.giantbomb.com/uploads/scale_medium/8/82063/2778000-tloztfh.jpg", "thumb_url": "http://static.giantbomb.com/uploads/scale_avatar/8/82063/2778000-tloztfh.jpg", "small_url": "http://static.giantbomb.com/uploads/scale_small/8/82063/2778000-tloztfh.jpg", "screen_url": "http://static.giantbomb.com/uploads/screen_medium/8/82063/2778000-tloztfh.jpg", "icon_url": "http://static.giantbomb.com/uploads/square_avatar/8/82063/2778000-tloztfh.jpg", "super_url": "http://static.giantbomb.com/uploads/scale_large/8/82063/2778000-tloztfh.jpg"}, "id": 49994}, "data_releases": {"deck": null, "image": null, "platform": {"api_detail_url": "http://www.giantbomb.com/api/platform/3045-138/", "id": 138, "name": "Nintendo 3DS eShop"}, "expected_release_day": 23, "expected_release_month": 10, "game": {"api_detail_url": "http://www.giantbomb.com/api/game/3030-49994/", "id": 49994, "name": "The Legend of Zelda: Tri Force Heroes"}, "expected_release_year": 2015, "id": 142927, "region": {"api_detail_url": "http://www.giantbomb.com/api/region/3075-1/", "id": 1, "name": "United States"}, "expected_release_quarter": null, "name": "The Legend of Zelda: Tri Force Heroes"}}]<
Here's an example of 'releases_data' and 'games_data' that wasn't returned in the result but does in fact match IDs :
releases_data:
{"deck":null,"game":{"api_detail_url":"http:\/\/www.giantbomb.com\/api\/game\/3030-50627\/","id":50627,"name":"Orion Trail"},"id":144188,"image":null,"name":"Orion Trail","platform":{"api_detail_url":"http:\/\/www.giantbomb.com\/api\/platform\/3045-94\/","id":94,"name":"PC"}}
games_data:
{"deck":"Orion Trail is a single player choose-your-own-space-adventure.","id":50627,"image":{"icon_url":"http:\/\/static.giantbomb.com\/uploads\/square_avatar\/29\/291401\/2775039-6490638002-heade.jpg","medium_url":"http:\/\/static.giantbomb.com\/uploads\/scale_medium\/29\/291401\/2775039-6490638002-heade.jpg","screen_url":"http:\/\/static.giantbomb.com\/uploads\/screen_medium\/29\/291401\/2775039-6490638002-heade.jpg","small_url":"http:\/\/static.giantbomb.com\/uploads\/scale_small\/29\/291401\/2775039-6490638002-heade.jpg","super_url":"http:\/\/static.giantbomb.com\/uploads\/scale_large\/29\/291401\/2775039-6490638002-heade.jpg","thumb_url":"http:\/\/static.giantbomb.com\/uploads\/scale_avatar\/29\/291401\/2775039-6490638002-heade.jpg","tiny_url":"http:\/\/static.giantbomb.com\/uploads\/square_mini\/29\/291401\/2775039-6490638002-heade.jpg"}}

EDIT: THIS IS INCORRECT. I'm leaving it just as context for the comments.
The problem with the example you posted for releases_data is in the very first field: "deck":null If I try to create a JSON object from this string, I get
builtins.NameError: name 'null' is not defined
There must be some try-catch block somewhere that is ignoring this exception. You could just define
null = None
before processing the files, if this is the only problem. Perhaps you ought to test how many JSON objects you can create from each of the two files, to locate any additional problems, before you go back to merging them.
Just as a debugging tip, this took me perhaps five minutes to analyze, once I got the data to work with from you. All I did was call json.loads on both strings and read the error message. It always (well, almost always) pays to start at the bottom and work up. :-)

Parsing JSON with Python from URL

So I'm trying to get json from a URl and the request works and I get the json but I'm not able to print specific things from it.
request_url = 'http://api.tumblr.com/v2/user/following?limit=1'
r = requests.get(request_url, auth=oauth).json()
r["updated"]
I'm very new with python I'm guessing I need to get the json into a array but I have no idea where to even begin.
According to the tumblr api I should be able to get something like this.
{
"meta": {
"status": 200,
"msg": "OK"
},
"response": {
"total_blogs": 4965,
"blogs": [
{
"name": "xxxxx",
"title": "xxxxxx",
"description": "",
"url": "http://xxxxxx.tumblr.com/",
"updated": 1439795949
}
]
}
}
I only need the name, url, and updated just no idea how to seperate that out.

Just access the levels one by one.
for i in r["response"]["blogs"]:
print i["name"],i["url"],i["updated"]
So this code can be used to print all the objects inside the blogs list
To explain how this works:
Json objects are decoded into something called dictionaries in Python. Dictionaries are simple key value pairs. In your example,
r is a dictionary with the following keys:
meta, response
You access the value of a key using r["meta"].
Now meta itself is a dictionary. The keys associated are:
status,msg
So, r["meta"]["status"] gives the status value returned by the request.

You should be able to print values as though it were nested arrays:
r["response"]["blogs"][0]["updated"] should get you the updated bit, don't go straight to it. Just work your way down. Note how blogs is an array, so in a normal case you may actually want to work towards r["response"]["blogs"], then loop through it and for each of those items, grab the ["updated"].
Similarly, r["meta"]["msg"] will get you the meta message.

The JSON data gets converted as dict which is set to r as per your code.
For accessing the value associated with updated key, you need to first access the values before it.
You should first access r["response"] which contains the actual response of the api. From that level, you should next access r["response"]["blogs"] and then loop through that to find the value of the updated key.
If it is a single blog, you can do something like r["response"]["blogs"][0]["updated"]