We Keep Coding

I can't get a value from a JSON API response in python

So I am struggling with getting a value from a JSON response. Looking in other post I have managed to write this code but when I try to search for the key (character_id) that I want in the dictionary python says that the key doesn't exist. My solution consists in getting the JSON object from the response, converting it into a string with json.dumps() and the converting it into a dictionary with json.loads(). Then I try to get 'character_id' from the dictionary but it doesn't exist. I am guessing it is related with the format of the dictionary but I have little to none experience in python. The code that makes the query and tries to get the values is this: (dataRequest is a fuction that makes the request and return the response from the api)
characterName = sys.argv[1];
response = dataRequest('http://census.daybreakgames.com/s:888/get/ps2:v2/character/?name.first_lower=' + characterName + '&c:show=character_id')
jsonString = json.dumps(response.json())
print(jsonString)
dic = json.loads(jsonString)
print(dic)
if 'character_id' in dic:
print(dic['character_id'])
The output of the code is:
{"character_list": [{"character_id": "5428662532301799649"}], "returned": 1}
{'character_list': [{'character_id': '5428662532301799649'}], 'returned': 1}

Welcome #Prieto! From what I can see, you probably don't need to serialize/de-serialize the JSON -- response.json() returns a python dictionary object already.
The issue is that you are looking for the 'character_id' key at the top-level of the dictionary, when it seems to be embedded inside another dictionary, that is inside a list. Try something like this:
#...omitted code
for char_obj in dic["character_list"]:
if "character_id" in char_obj:
print(char_obj["character_id"])

if your dic is like {"character_list": [{"character_id": "5428662532301799649"}], "returned": 1}
you get the value of character_id by
print(dic['character_list'][0][character_id])

The problem here is that you're trying to access a dictionary where the key is actually character_list.
What you need to do is to access the character_list value and iterate over or filter the character_id you want.
Like this:
print(jsonString)
dic = json.loads(jsonString)
print(dic)
character_information = dic['character_list'][0] # we access the character list and assume it is the first value
print(character_information["character_id"]) # this is your character id

The way I see it, the only hiccup with the code is this :
if 'character_id' in dic:
print(dic['character_id'])
The problem is that, the JSON file actually consists of actually 2 dictionaries , first is the main one, which has two keys, character_list and returned. There is a second sub-dictionary inside the array, which is the value for the key character_list.
So, what your code should actually look like is something like this:
for i in dic["character_list"]:
print(i["character_id"])
On a side-note, it will help to look at JSON file in this way :
{
"character_list": [
{
"character_id": "5428662532301799649"
}
],
"returned": 1
}
,where, elements enclosed in curly-brackets'{}' imply they are in a dictionary, whereas elements enclosed in curly-brackets'[]' imply they are in a list

txt with str of dict into dict [duplicate]

I am trying to pass in a JSON file and convert the data into a dictionary.
So far, this is what I have done:
import json
json1_file = open('json1')
json1_str = json1_file.read()
json1_data = json.loads(json1_str)
I'm expecting json1_data to be a dict type but it actually comes out as a list type when I check it with type(json1_data).
What am I missing? I need this to be a dictionary so I can access one of the keys.

Your JSON is an array with a single object inside, so when you read it in you get a list with a dictionary inside. You can access your dictionary by accessing item 0 in the list, as shown below:
json1_data = json.loads(json1_str)[0]
Now you can access the data stored in datapoints just as you were expecting:
datapoints = json1_data['datapoints']
I have one more question if anyone can bite: I am trying to take the average of the first elements in these datapoints(i.e. datapoints[0][0]). Just to list them, I tried doing datapoints[0:5][0] but all I get is the first datapoint with both elements as opposed to wanting to get the first 5 datapoints containing only the first element. Is there a way to do this?
datapoints[0:5][0] doesn't do what you're expecting. datapoints[0:5] returns a new list slice containing just the first 5 elements, and then adding [0] on the end of it will take just the first element from that resulting list slice. What you need to use to get the result you want is a list comprehension:
[p[0] for p in datapoints[0:5]]
Here's a simple way to calculate the mean:
sum(p[0] for p in datapoints[0:5])/5. # Result is 35.8
If you're willing to install NumPy, then it's even easier:
import numpy
json1_file = open('json1')
json1_str = json1_file.read()
json1_data = json.loads(json1_str)[0]
datapoints = numpy.array(json1_data['datapoints'])
avg = datapoints[0:5,0].mean()
# avg is now 35.8
Using the , operator with the slicing syntax for NumPy's arrays has the behavior you were originally expecting with the list slices.

Here is a simple snippet that read's in a json text file from a dictionary. Note that your json file must follow the json standard, so it has to have " double quotes rather then ' single quotes.
Your JSON dump.txt File:
{"test":"1", "test2":123}
Python Script:
import json
with open('/your/path/to/a/dict/dump.txt') as handle:
dictdump = json.loads(handle.read())

You can use the following:
import json
with open('<yourFile>.json', 'r') as JSON:
json_dict = json.load(JSON)
# Now you can use it like dictionary
# For example:
print(json_dict["username"])

The best way to Load JSON Data into Dictionary is You can user the inbuilt json loader.
Below is the sample snippet that can be used.
import json
f = open("data.json")
data = json.load(f))
f.close()
type(data)
print(data[<keyFromTheJsonFile>])

I am working with a Python code for a REST API, so this is for those who are working on similar projects.
I extract data from an URL using a POST request and the raw output is JSON. For some reason the output is already a dictionary, not a list, and I'm able to refer to the nested dictionary keys right away, like this:
datapoint_1 = json1_data['datapoints']['datapoint_1']
where datapoint_1 is inside the datapoints dictionary.

pass the data using javascript ajax from get methods
**//javascript function
function addnewcustomer(){
//This function run when button click
//get the value from input box using getElementById
var new_cust_name = document.getElementById("new_customer").value;
var new_cust_cont = document.getElementById("new_contact_number").value;
var new_cust_email = document.getElementById("new_email").value;
var new_cust_gender = document.getElementById("new_gender").value;
var new_cust_cityname = document.getElementById("new_cityname").value;
var new_cust_pincode = document.getElementById("new_pincode").value;
var new_cust_state = document.getElementById("new_state").value;
var new_cust_contry = document.getElementById("new_contry").value;
//create json or if we know python that is call dictionary.
var data = {"cust_name":new_cust_name, "cust_cont":new_cust_cont, "cust_email":new_cust_email, "cust_gender":new_cust_gender, "cust_cityname":new_cust_cityname, "cust_pincode":new_cust_pincode, "cust_state":new_cust_state, "cust_contry":new_cust_contry};
//apply stringfy method on json
data = JSON.stringify(data);
//insert data into database using javascript ajax
var send_data = new XMLHttpRequest();
send_data.open("GET", "http://localhost:8000/invoice_system/addnewcustomer/?customerinfo="+data,true);
send_data.send();
send_data.onreadystatechange = function(){
if(send_data.readyState==4 && send_data.status==200){
alert(send_data.responseText);
}
}
}
django views
def addNewCustomer(request):
#if method is get then condition is true and controller check the further line
if request.method == "GET":
#this line catch the json from the javascript ajax.
cust_info = request.GET.get("customerinfo")
#fill the value in variable which is coming from ajax.
#it is a json so first we will get the value from using json.loads method.
#cust_name is a key which is pass by javascript json.
#as we know json is a key value pair. the cust_name is a key which pass by javascript json
cust_name = json.loads(cust_info)['cust_name']
cust_cont = json.loads(cust_info)['cust_cont']
cust_email = json.loads(cust_info)['cust_email']
cust_gender = json.loads(cust_info)['cust_gender']
cust_cityname = json.loads(cust_info)['cust_cityname']
cust_pincode = json.loads(cust_info)['cust_pincode']
cust_state = json.loads(cust_info)['cust_state']
cust_contry = json.loads(cust_info)['cust_contry']
#it print the value of cust_name variable on server
print(cust_name)
print(cust_cont)
print(cust_email)
print(cust_gender)
print(cust_cityname)
print(cust_pincode)
print(cust_state)
print(cust_contry)
return HttpResponse("Yes I am reach here.")**

Process malformed JSON string in Python

I'm trying to process a log from Symphony using Pandas, but have some trouble with a malformed JSON which I can't parse.
An example of the log :
'{id:46025,
work_assignment:43313=>43313,
declaration:<p><strong>Bijkomende interventie.</strong></p>\r\n\r\n<p>H </p>\r\n\r\n<p><strong><em>Vaststellingen.</em></strong></p>\r\n\r\n<p><strong><em>CV. </em></strong>De.</p>=><p><strong>Bijkomende interventie.</strong></p>\r\n\r\n<p>He </p>\r\n\r\n<p><strong><em>Vaststellingen.</em></strong></p>\r\n\r\n<p><strong><em>CV. </em></strong>De.</p>,conclusions:<p>H </p>=><p>H </p>}'
What is the best way to process this?
For each part (id/work_assignment/declaration/etc) I would like to retrieve the old and new value (which are separated by "=>").

Use the following code:
def clean(my_log):
my_log.replace("{", "").replace("}", "") # Removes the unneeded { }
my_items = list(my_log.split(",")) # Split at the comma to get the pairs
my_dict = {}
for i in my_items:
key, value = i.split(":") # Split at the colon to separate the key and value
my_dict[key] = value # Add to the dictionary
return my_dict
Function returns a Python dictionary, which can then be converted to JSON using a serializer if needed, or directly used.
Hope I helped :D

Parsing Python JSON with multiple same strings with different values

I am stuck on an issue where I am trying to parse for the id string in JSON that exists more than 1 time. I am using the requests library to pull json from an API. I am trying to retrieve all of the values of "id" but have only been able to successfully pull the one that I define. Example json:
{
"apps": [{
"id": "app1",
"id": "app2",
"id": "new-app"
}]
}
So what I have done so far is turn the json response into dictionary so that I am actually parse the first iteration of "id". I have tried to create for loops but have been getting KeyError when trying to find string id or TypeError: list indices must be integers or slices, not str. The only thing that I have been able to do successfully is define which id locations to output.
(data['apps'][N]['id']) -> where N = 0, 1 or 2
This would work if there was only going to be 1 string of id at a time but will always be multiple and the location will change from time to time.
So how do return the values of all strings for "id" from this single json output? Full code below:
import requests
url = "http://x.x.x.x:8080/v2/apps/"
response = requests.get(url)
#Error if not 200 and exit
ifresponse.status_code!=200:
print("Status:", response.status_code, "CheckURL.Exiting")
exit()
#Turn response into a dict and parse for ids
data = response.json()
for n in data:
print(data['apps'][0]['id'])
OUTPUT:
app1
UPDATE:
Was able to get resolution thanks to Robᵩ. Here is what I ended up using:
def list_hook(pairs):
result = {}
for name, value in pairs:
if name == 'id':
result.setdefault(name, []).append(value)
print(value)
data = response.json(object_pairs_hook = list_hook)
Also The API that I posted as example is not a real API. It was just supposed to be a visual representation of what I was trying to achieve. I am actually using Mesosphere's Marathon API . Trying to build a python listener for port mapping containers.

Your best choice is to contact the author of the API and let him know that his data format is silly.
Your next-best choice is to modify the behavior of the the JSON parser by passing in a hook function. Something like this should work:
def list_hook(pairs):
result = {}
for name, value in pairs:
if name == 'id':
result.setdefault(name, []).append(value)
else:
result[name] = value
return result
data = response.json(object_pairs_hook = list_hook)
for i in range(3):
print(i, data['apps'][0]['id'][i])

python: argument generated by a prior function error

so this is probably very simple but it's confusing me.
I have a function that takes in a txt file in JSON form and sorts it in descending order of bandwidth. The function is:
def sort_guards_with_input(guards):
json_source = json.dumps(guards)
data = json.loads(str(json_source))
data['relays'].sort(key=lambda item: item['bandwidth'], reverse=True)
data = json.dumps(data)
return data
segundo = sort_guards_with_input("the original txt file")
..and this returns the sorted file of the form (lets call it TEXT):
{"relays": [{"nickname": "Snowden4ever pd7wih1gdUU8bLhWsvH6QHDWfs8",
"bandwidth": 201000, "type": ["Fast", "Guard", "HSDir", "Named", "Running",
"Stable", "V2Dir", "Valid"]},{"nickname": "rmblue jMdIu0VZYE+S2oeHShQBAHsdj80",
"bandwidth": 8, "type": ["Fast", "Guard", "HSDir", "Running", "Stable", "Unnamed",
"V2Dir", "Valid"]}]}
Now I have a function that pulls out the banwidth and nickname and creates a list. The function is:
def get_sorted_names_bw(guards):
sorted_guards_bw = list(entry['bandwidth'] for entry in guards["relays"])
sorted_guards_names = list(d['nickname'] for d in guards["relays"])
temps = [None]*(len(sorted_guards_bw)+len(sorted_guards_names))
temps[::2] = sorted_guards_bw
temps[1::2] = sorted_guards_names
sorted_grds_w_names = [temps[i:i+2] for i in range(0, len(temps), 2)]
return sorted_grds_w_names
The problem is when I try and print the result of get_sorted_names_bw by doing:
print get_sorted_names_bw(segundo)
.. I get the error:
sorted_guards_bw = list(entry['bandwidth'] for entry in guards["relays"])
TypeError: string indices must be integers, not str
But if i try and print the the result of get_sorted_names_bw with copy and pasting TEXT as the argument it returns a result (the wrong one because nicknames and bandwidths are mixed up, that's another problem I'll deal with myself, unless the reader is feeling very kind and wants to help with that too :) ). Namely:
[[201000, 'rmblue jMdIu0VZYE+S2oeHShQBAHsdj80'], [8, 'Snowden4ever pd7wih1gdUU8bLhWsvH6QHDWfs8']]
Why do I get an error when I try use an argument generated by a prior function but don't get an error when I just copy and paste the argument?
Thanks and sorry for the long post.

Your function sort_guards_with_input dumps the data to a JSON string and returns that string. But get_sorted_names_bw assumes it is receiving the actual data (as a dict), not a string representation of it. The easiest thing is probably to just have sort_guards_with_input return data without dumping it to JSON. That is:
def sort_guards_with_input(guards):
json_source = json.dumps(guards)
data = json.loads(str(json_source))
data['relays'].sort(key=lambda item: item['bandwidth'], reverse=True)
return data