We Keep Coding

txt with str of dict into dict [duplicate]

I am trying to pass in a JSON file and convert the data into a dictionary.
So far, this is what I have done:
import json
json1_file = open('json1')
json1_str = json1_file.read()
json1_data = json.loads(json1_str)
I'm expecting json1_data to be a dict type but it actually comes out as a list type when I check it with type(json1_data).
What am I missing? I need this to be a dictionary so I can access one of the keys.

Your JSON is an array with a single object inside, so when you read it in you get a list with a dictionary inside. You can access your dictionary by accessing item 0 in the list, as shown below:
json1_data = json.loads(json1_str)[0]
Now you can access the data stored in datapoints just as you were expecting:
datapoints = json1_data['datapoints']
I have one more question if anyone can bite: I am trying to take the average of the first elements in these datapoints(i.e. datapoints[0][0]). Just to list them, I tried doing datapoints[0:5][0] but all I get is the first datapoint with both elements as opposed to wanting to get the first 5 datapoints containing only the first element. Is there a way to do this?
datapoints[0:5][0] doesn't do what you're expecting. datapoints[0:5] returns a new list slice containing just the first 5 elements, and then adding [0] on the end of it will take just the first element from that resulting list slice. What you need to use to get the result you want is a list comprehension:
[p[0] for p in datapoints[0:5]]
Here's a simple way to calculate the mean:
sum(p[0] for p in datapoints[0:5])/5. # Result is 35.8
If you're willing to install NumPy, then it's even easier:
import numpy
json1_file = open('json1')
json1_str = json1_file.read()
json1_data = json.loads(json1_str)[0]
datapoints = numpy.array(json1_data['datapoints'])
avg = datapoints[0:5,0].mean()
# avg is now 35.8
Using the , operator with the slicing syntax for NumPy's arrays has the behavior you were originally expecting with the list slices.

Here is a simple snippet that read's in a json text file from a dictionary. Note that your json file must follow the json standard, so it has to have " double quotes rather then ' single quotes.
Your JSON dump.txt File:
{"test":"1", "test2":123}
Python Script:
import json
with open('/your/path/to/a/dict/dump.txt') as handle:
dictdump = json.loads(handle.read())

You can use the following:
import json
with open('<yourFile>.json', 'r') as JSON:
json_dict = json.load(JSON)
# Now you can use it like dictionary
# For example:
print(json_dict["username"])

The best way to Load JSON Data into Dictionary is You can user the inbuilt json loader.
Below is the sample snippet that can be used.
import json
f = open("data.json")
data = json.load(f))
f.close()
type(data)
print(data[<keyFromTheJsonFile>])

I am working with a Python code for a REST API, so this is for those who are working on similar projects.
I extract data from an URL using a POST request and the raw output is JSON. For some reason the output is already a dictionary, not a list, and I'm able to refer to the nested dictionary keys right away, like this:
datapoint_1 = json1_data['datapoints']['datapoint_1']
where datapoint_1 is inside the datapoints dictionary.

pass the data using javascript ajax from get methods
**//javascript function
function addnewcustomer(){
//This function run when button click
//get the value from input box using getElementById
var new_cust_name = document.getElementById("new_customer").value;
var new_cust_cont = document.getElementById("new_contact_number").value;
var new_cust_email = document.getElementById("new_email").value;
var new_cust_gender = document.getElementById("new_gender").value;
var new_cust_cityname = document.getElementById("new_cityname").value;
var new_cust_pincode = document.getElementById("new_pincode").value;
var new_cust_state = document.getElementById("new_state").value;
var new_cust_contry = document.getElementById("new_contry").value;
//create json or if we know python that is call dictionary.
var data = {"cust_name":new_cust_name, "cust_cont":new_cust_cont, "cust_email":new_cust_email, "cust_gender":new_cust_gender, "cust_cityname":new_cust_cityname, "cust_pincode":new_cust_pincode, "cust_state":new_cust_state, "cust_contry":new_cust_contry};
//apply stringfy method on json
data = JSON.stringify(data);
//insert data into database using javascript ajax
var send_data = new XMLHttpRequest();
send_data.open("GET", "http://localhost:8000/invoice_system/addnewcustomer/?customerinfo="+data,true);
send_data.send();
send_data.onreadystatechange = function(){
if(send_data.readyState==4 && send_data.status==200){
alert(send_data.responseText);
}
}
}
django views
def addNewCustomer(request):
#if method is get then condition is true and controller check the further line
if request.method == "GET":
#this line catch the json from the javascript ajax.
cust_info = request.GET.get("customerinfo")
#fill the value in variable which is coming from ajax.
#it is a json so first we will get the value from using json.loads method.
#cust_name is a key which is pass by javascript json.
#as we know json is a key value pair. the cust_name is a key which pass by javascript json
cust_name = json.loads(cust_info)['cust_name']
cust_cont = json.loads(cust_info)['cust_cont']
cust_email = json.loads(cust_info)['cust_email']
cust_gender = json.loads(cust_info)['cust_gender']
cust_cityname = json.loads(cust_info)['cust_cityname']
cust_pincode = json.loads(cust_info)['cust_pincode']
cust_state = json.loads(cust_info)['cust_state']
cust_contry = json.loads(cust_info)['cust_contry']
#it print the value of cust_name variable on server
print(cust_name)
print(cust_cont)
print(cust_email)
print(cust_gender)
print(cust_cityname)
print(cust_pincode)
print(cust_state)
print(cust_contry)
return HttpResponse("Yes I am reach here.")**

Remove 'u' in JSON dictionary

I hope to use the dictionary load by json file. However, each item contains the character 'u'. I need to remove the 'u's.
I tried dumps, but it does not work.
import ast
import json
data= {u'dot',
u'dog',
u'fog',
u'eeee'}
res = eval(json.dumps(data))
print res
I hope to get: {
'dot',
'dog',
'fog,
'eeee'
}
But the error is:
TypeError: set([u'eeee', u'fog', u'dog', u'dot']) is not JSON serializable

The strings that start with u are unicode strings.
In your case, this has nothing to do with the problem:
data= {u'dot',
u'dog',
u'fog',
u'eeee'}
This creates a set and stores the results in the data variable. The json serializer can't handle sets since the json spec makes no mention of them. If you change this to be a list, the serializer can handle the data:
res = set(eval(json.dumps(list(data))))
Here I'm converting the data variable to a list to serialize it, then converting it back to a set to store the result in the res variable.
Alternatively, you can directly ask Python to convert the unicode strings to strings, using something like this:
res = {x.encode("utf-8") for x in data}
print(res)

How do I replace a JSON list to print to a CSV?

I'm using an API to gather some data that comes to me in JSON format. I'm using json.loads to import the data and can successfully write it to a CSV. Unfortunately, the data comes in in a format that I don't want so I'd like to reformat the json list.
I've tried creating a new list and assigning the JSON list to the desired list. I get the following error: TypeError: list indices must be integers or slices, not str
import requests
import json
import csv
response = requests.get(url).text //json source
data = json.loads(response)
newsdata = (data["response"]["docs"])
// These two lines reformat the date to what I want it to look like
newsdate = [y["pub_date"] for y in newsdata]
newsdate = [y.split('T')[0] for y in newsdate]
newsdata["pub_date"] = newsdate // This line is what I've tried to replace the json
newssnip = [y["snippet"] for y in newsdata]
newshead = [y["headline"]["main"] for y in newsdata]
for z in newsdata:
csvwriter.writerow([z["pub_date"], //This is the JSON data i want to reformat
z["headline"]["main"],
z["snippet"],
z["web_url"]])
I expected the newsdata["pub_date"] to be overwritten when I assigned newsdate to it but I get the following error instead: TypeError: list indices must be integers or slices, not str
Thank you for your help! :)
EDIT:
I've uploaded an example json response here on github called "exmaple.json": https://github.com/theChef613/nytnewsscrapper

That error is saying that newsdata is list and is therefore not subscriptable with a string. If you post the raw JSON data returned or also print(type(newsdata)) to figure out what class newsdata is and how to work with it. It's also possible that newsdata is a 2D (or N-d) array where the first element is the key and the second element is the value.

Process malformed JSON string in Python

I'm trying to process a log from Symphony using Pandas, but have some trouble with a malformed JSON which I can't parse.
An example of the log :
'{id:46025,
work_assignment:43313=>43313,
declaration:<p><strong>Bijkomende interventie.</strong></p>\r\n\r\n<p>H </p>\r\n\r\n<p><strong><em>Vaststellingen.</em></strong></p>\r\n\r\n<p><strong><em>CV. </em></strong>De.</p>=><p><strong>Bijkomende interventie.</strong></p>\r\n\r\n<p>He </p>\r\n\r\n<p><strong><em>Vaststellingen.</em></strong></p>\r\n\r\n<p><strong><em>CV. </em></strong>De.</p>,conclusions:<p>H </p>=><p>H </p>}'
What is the best way to process this?
For each part (id/work_assignment/declaration/etc) I would like to retrieve the old and new value (which are separated by "=>").

Use the following code:
def clean(my_log):
my_log.replace("{", "").replace("}", "") # Removes the unneeded { }
my_items = list(my_log.split(",")) # Split at the comma to get the pairs
my_dict = {}
for i in my_items:
key, value = i.split(":") # Split at the colon to separate the key and value
my_dict[key] = value # Add to the dictionary
return my_dict
Function returns a Python dictionary, which can then be converted to JSON using a serializer if needed, or directly used.
Hope I helped :D

grabbing an ID from string

I have a string where I'd like to grab the "id" number 12079500908. I am trying to use ast.literal_eval but received a ValueError: malformed string. Is there any other way to get the id number from the string below?
doc_request = urllib2.Request("https://api.box.com/2.0/search?query=SEARCHTERMS", headers=doc_headers)
doc_response = urllib2.urlopen(doc_request)
view_doc_response = doc_response.read()
doc_dict=ast.literal_eval(view_doc_response)
Edit
Output:
view_doc_response = '{"total_count":1,"entries":[{"type":"file","id":"12079500908","sequence_id":"1","etag":"1","sha1":"6887169228cab0cfb341059194bc980e1be8ad90","name":"file.pdf","description":"","size":897838,"path_collection":{"total_count":2,"entries":[{"type":"folder","id":"0","sequence_id":null,"etag":null,"name":"All Files"},{"type":"folder","id":"1352745576","sequence_id":"0","etag":"0","name":"Patient Files"}]},"created_at":"2013-12-03T10:23:30-08:00","modified_at":"2013-12-03T11:17:52-08:00","trashed_at":null,"purged_at":null,"content_created_at":"2013-12-03T10:23:30-08:00","content_modified_at":"2013-12-03T11:17:52-08:00","created_by":{"type":"user","id":"20672372","name":"name","login":"email"},"modified_by":{"type":"user","id":"206732372","name":"name","login":"email"},"owned_by":{"type":"user","id":"206737772","name":"name","login":"email"},"shared_link":{"url":"https:\\/\\/www.box.net\\/s\\/ymfslf1phfqiw65bunjg","download_url":"https:\\/\\/www.box.net\\/shared\\/static\\/ymfslf1phfqiw65bunjg.pdf","vanity_url":null,"is_password_enabled":false,"unshared_at":null,"download_count":0,"preview_count":0,"access":"open","permissions":{"can_download":true,"can_preview":true}},"parent":{"type":"folder","id":"1352745576","sequence_id":"0","etag":"0","name":"Patient Files"},"item_status":"active"}],"limit":30,"offset":0}'
calling doc_dict gives:
ValueError: malformed string

ast.literal_eval is for parsing valid Python syntax, what you have is JSON. Valid JSON looks a lot like Python syntax except that JSON can contain null, true, and false which are mapped to None, True, and False in Python when passed through a JSON decoder. You can use json.loads for this. The code might look something like this:
import json
doc_dict = json.loads(view_doc_response)
first_id = doc_dict['entries'][0]['id'] # with your data, should be 12079500908
Note that this assumes that you manually added the ... at the end of the string, presumably after shortening it. If that ... is actually in your code as well then you have invalid JSON and you will need to do some processing before it will work.