I'm trying to work with dictionaries inside a list in a JSON file. The data imports fine and reads fine. For the life of me I can't figure out how to printout the "member_id" keys. I just want to print the list of "member_id" numbers. I was initially using json.loads, then switched to json.dumps. Any help would really be appreciated.
import urllib2
import json
nyt_api_key = '72c9a68bbc504e91a3919efda17ae621%3A7%3A70586819'
url= 'http://api.nytimes.com/svc/politics/v3/us/legislative/congress/113'
json_obj = urllib2.urlopen(url)
data = json.load(json_obj)
data2 = json.dumps(data, sort_keys=True, indent=True, skipkeys = True)
print data2
Output from print data2: (The list goes on and on so it is truncated. There is a closing bracket at the bottom of the list. So it's dictionaries within a list.)
"positions": [
{
"dw_nominate": "0.466",
"member_id": "A000055",
"vote_position": "Yes"
},
{
"dw_nominate": "0.995",
"member_id": "A000367",
"vote_position": "Yes"
},
{
"dw_nominate": "0.666",
"member_id": "A000369",
"vote_position": "Yes"
},
Output from print data2['member_id'], output is the same if using 'positions', 'vote_position', etc.:
Traceback (most recent call last):
File "/Users/Owner/PycharmProjects/untitled2/1", line 9, in <module>
print data2["positions"]
TypeError: string indices must be integers, not str
Output from print data:
u'positions': [{u'dw_nominate': u'0.466', u'vote_position': u'Yes', u'member_id': u'A000055'}, {u'dw_nominate': u'0.995', u'vote_position': u'Yes', u'member_id': u'A000367'}, {u'dw_nominate': u'0.666', u'vote_position': u'Yes', u'member_id': u'A000369'}
Output from print data['positions']:
print data["positions"]
KeyError: 'positions'
Output from print.data(keys):
[u'status', u'results', u'copyright']
Process finished with exit code 0
I just want to print the list of "member_id" numbers.
So you need to loop over positions and access the member_id in each dict:
data ={"positions": [
{
"dw_nominate": "0.466",
"member_id": "A000055",
"vote_position": "Yes"
},
{
"dw_nominate": "0.995",
"member_id": "A000367",
"vote_position": "Yes"
},
{
"dw_nominate": "0.666",
"member_id": "A000369",
"vote_position": "Yes"
}]}
print([d["member_id"] for d in data["results"]["positions"]])
['A000055', 'A000367', 'A000369']
If you look at the API documentation there are examples of each json response.
data2 is a string value, it doesn't have keys. I think what you want to print is data["positions"]
That's a weird output from data, you don't even have the braces. Try printing the type(data), it should be dict
So I should change the heading of this to Scrapping JSON for XML in Python. I'm sure not everyone else would have the same issues I did with JSON but after many frustrating hours I decided to go down path #2... the xml version. The xml version was much easier to work with right out of the gate. In about 1/10 the time I got what I was looking for.
from urllib2 import urlopen
from xml.dom import minidom
feed = urlopen("http://api.nytimes.com/svc/politics/v3/us/legislative.xml?
doc = minidom.parse(feed)
id_element = doc.getElementsByTagName("member_id")
id_number0 = id_element[0].childNodes[0].nodeValue #just a sample
id_number1 = id_element[1].childNodes[0].nodeValue #just a sample
id_number2 = id_element[2].childNodes[0].nodeValue #just a sample
print len(id_element) #to see how many items were in the variable
count = 0
for item in id_element:
print id_element[count].childNodes[0].nodeValue
count = count + 1
if count == 434:
break
This is definitely not the cleanest loop. I'm still working on that. But the code solves the problem that I had originally posted. The API key is not the actual one, formatting in the answer window was throwing it off so I just erased a bunch of it. You can find the API at the NYT developer website.
Thanks to everyone who posted.
Related
I tried to get weather details using Open Weather Map APIs which worked till a point. I want to find the description of the clouds (scattered, heavy etc) and I am getting a keyword error which I am unable to correct
This is the code I have written-
import requests , json
apiKey = "d5f6e96071109af97ee3b206fe8cb0cb"
baseURL = "https://api.openweathermap.org/data/2.5/weather?q={city name}&appid={API key}"
cityName = "ranchi"
completeURL = f"https://api.openweathermap.org/data/2.5/weather?q={cityName}&appid={apiKey}"
response = requests.get(completeURL)
data = response.json()
print(data)
print("Minimum Temperature ",data["main"]["temp_min"])
print("Maximum Temperature ",data["main"]["temp_max"])
print("Temperature ",data["main"]["pressure"])
print("Visibility ",data["visibility"])
print("Humidity ",data["main"]["humidity"])
print("Clouds ",data["weather"]["description"])
This is the error (of the last print statement) I am getting-
line 23, in <module>
print("Clouds ",data["weather"]["description"])
TypeError: list indices must be integers or slices, not str
What is the keyword I am missing ?
For reference, this is an excerpt of the output of the API call:
"coord": {
"lon": 85.3333,
"lat": 23.35
},
"weather": [{
"id": 802,
"main": "Clouds",
"description": "scattered clouds"
}],
"main": {
"temp": 312.21,
"feels_like": 310.96
....
If you look closely, you will notice that, for example, main directly references a json object. So you can refer to data["main"]["humidity"] and expect it to work. However, weather references a list, not a json object. So referring to it as data["weather"]["description"] will assume that description is the index of the weather list.
To solve the issue, you only need to provide the index first (0 in this case) like so:
data["weather"][0]["description"]
And everything should work as expected.
Keeping all same except data["weather"][0]["description"] because the "weather" key has a list of values from which you have to get a "description".
I Hope, it solves your problem!
import requests , json
apiKey = "d5f6e96071109af97ee3b206fe8cb0cb"
baseURL = "https://api.openweathermap.org/data/2.5/weather?q={city name}&appid={API key}"
cityName = "ranchi"
completeURL = f"https://api.openweathermap.org/data/2.5/weather?q={cityName}&appid={apiKey}"
response = requests.get(completeURL)
data = response.json()
print(data)
print("Minimum Temperature ",data["main"]["temp_min"])
print("Maximum Temperature ",data["main"]["temp_max"])
print("Temperature ",data["main"]["pressure"])
print("Visibility ",data["visibility"])
print("Humidity ",data["main"]["humidity"])
print("Clouds ",data["weather"][0]["description"])
I have a big tree in a JSON file and I'm searching the python syntax for loading nested in nested keys from this JSON.
Assume I have this :
{
"FireWall": {
"eth0": {
"INPUT": {
"PING": 1,
}
}
}
}
According to the man page and some questions in Stackoverflow i tried this (and some variations) :
import json
config = open('config.json', 'r')
data = json.load('config')
config.close()
if data['{"FireWall", {"eth0", {"INPUT", {"Ping"}}}}'] == 1:
print('This is working')
With no result. What is the right way to do this (as simple as possible) ? Thank you !
You are trying data = json.load('config') to load string not file object and data['{"FireWall", {"eth0", {"INPUT", {"Ping"}}}}'] it's not right way to access nested dictionary key value.
import json
with open('config.json', 'r') as f:
data = json.load(f)
if data["FireWall"]["eth0"]["INPUT"]["Ping"] == 1:
print('This is working')
data is a nested dictionary, so:
data["FireWall"]["eth0"]["INPUT"]["Ping"]
will be equal to 1; or at least it will when you fix your call to json.load.
Try this:
data["FireWall"]["eth0"]["INPUT"]["PING"]
This will give you the value in PING
I am trying to build an IOT setup. I am thinking of using a json file to store states of the sensors and lights of the setup.
I have created a function to test out my concept. Here is what I wrote so far for the data side of things.
{
"sensor_data": [
{
"sensor_id": "302CEM/lion/light1",
"sensor_state": "on"
},
{
"sensor_id": "302CEM/lion/light2",
"sensor_state": "off"
}
]
}
def read_from_db():
with open('datajson.json') as f:
data = json.load(f)
for sensors in data['sensor_data']:
name = sensors['sensor_id']
read_from_db()
What I want to do is to parse the sensor_id into an array so that I can access them by saying for example sensor_name[0]. I am not sure how to go about it. I tried array.array but it doesn't save any values, have also tried .append but not the result I expected. Any suggestions?
If I understood correctly, all you have to do is assign all those sensors to names using a for loop and then return the result:
import json
def read_from_db():
with open('sensor_data.json') as f:
data = json.load(f)
names = [sensors['sensor_id'] for sensors in data['sensor_data']]
return names
sensor_names = read_from_db()
for i in range(len(sensor_names)):
print(sensor_names[i])
This will print:
302CEM/lion/light1
302CEM/lion/light2
I have this method that writes json data to a file. The title is based on books and data is the book publisher,date,author, etc. The method works fine if I wanted to add one book.
Code
import json
def createJson(title,firstName,lastName,date,pageCount,publisher):
print "\n*** Inside createJson method for " + title + "***\n";
data = {}
data[title] = []
data[title].append({
'firstName:', firstName,
'lastName:', lastName,
'date:', date,
'pageCount:', pageCount,
'publisher:', publisher
})
with open('data.json','a') as outfile:
json.dump(data,outfile , default = set_default)
def set_default(obj):
if isinstance(obj,set):
return list(obj)
if __name__ == '__main__':
createJson("stephen-king-it","stephen","king","1971","233","Viking Press")
JSON File with one book/one method call
{
"stephen-king-it": [
["pageCount:233", "publisher:Viking Press", "firstName:stephen", "date:1971", "lastName:king"]
]
}
However if I call the method multiple times , thus adding more book data to the json file. The format is all wrong. For instance if I simply call the method twice with a main method of
if __name__ == '__main__':
createJson("stephen-king-it","stephen","king","1971","233","Viking Press")
createJson("william-golding-lord of the flies","william","golding","1944","134","Penguin Books")
My JSON file looks like
{
"stephen-king-it": [
["pageCount:233", "publisher:Viking Press", "firstName:stephen", "date:1971", "lastName:king"]
]
} {
"william-golding-lord of the flies": [
["pageCount:134", "publisher:Penguin Books", "firstName:william","lastName:golding", "date:1944"]
]
}
Which is obviously wrong. Is there a simple fix to edit my method to produce a correct JSON format? I look at many simple examples online on putting json data in python. But all of them gave me format errors when I checked on JSONLint.com . I have been racking my brain to fix this problem and editing the file to make it correct. However all my efforts were to no avail. Any help is appreciated. Thank you very much.
Simply appending new objects to your file doesn't create valid JSON. You need to add your new data inside the top-level object, then rewrite the entire file.
This should work:
def createJson(title,firstName,lastName,date,pageCount,publisher):
print "\n*** Inside createJson method for " + title + "***\n";
# Load any existing json data,
# or create an empty object if the file is not found,
# or is empty
try:
with open('data.json') as infile:
data = json.load(infile)
except FileNotFoundError:
data = {}
if not data:
data = {}
data[title] = []
data[title].append({
'firstName:', firstName,
'lastName:', lastName,
'date:', date,
'pageCount:', pageCount,
'publisher:', publisher
})
with open('data.json','w') as outfile:
json.dump(data,outfile , default = set_default)
A JSON can either be an array or a dictionary. In your case the JSON has two objects, one with the key stephen-king-it and another with william-golding-lord of the flies. Either of these on their own would be okay, but the way you combine them is invalid.
Using an array you could do this:
[
{ "stephen-king-it": [] },
{ "william-golding-lord of the flies": [] }
]
Or a dictionary style format (I would recommend this):
{
"stephen-king-it": [],
"william-golding-lord of the flies": []
}
Also the data you are appending looks like it should be formatted as key value pairs in a dictionary (which would be ideal). You need to change it to this:
data[title].append({
'firstName': firstName,
'lastName': lastName,
'date': date,
'pageCount': pageCount,
'publisher': publisher
})
dct_data = json_tour_data.__dict__
tour_data = json.dumps(dct_data)
How to remove these backslashes from json? Here is my output:
"{\"strFileOpenDateAjxKey\": \"2018-01-16 12:40:22.526417\",
\"strFilePassengerAjxKey\": \"Zahra Walji\", \"strFileOpenMobileAjxKey\":
\"46464664\", \"strFileOpenDepartmentAjxKey\": \"Finance department\",
\"strFileOpenAccountCodeAjxKey\": \"CARTZS\",
\"strFileOpenProfileCodeAjxKey\": \"CARTZS\",
\"strFileopenOriginalCountryIdAjxKey\": 61, \"blnBoundAjxKey\": 1,
\"strTransactionCurrencyJsKey\": \"Shillings\",
\"intCurrencyPrecisionJsKey\": 3, \"strPackageTypeJsKey\": \"PKG\",
\"strUserNameAjxKey\": \"admin\", \"strPasswordAjxKey\": \"1234\"}"
The answer is you have to just play with json.dumps() and json.loads().
The following is my code which worked:-
import json
json_data = {'key1': 'first'}
json_data = json.dumps(json_data)
return {
'statusCode': 200,
'schools': json.loads(json_data)
}
The output of above code is as follows:
Response:
{
"schools": {
"key1": "first"
},
"statusCode": 200
}
I would recommend checking your Response object where your parse your tour_data variable/dictionary in your views. I originally had the same issue as you but here's what I changed.
Original implementation: Response(json.dumps(a_dictionary), status=status.HTTP_200_OK)
to
New implementation: Response(a_dictionary, status=status.HTTP_200_OK, content_type='json')
The key things here are:
1. Getting rid of the json.dumps conversion method and just pass through a plain python dictionary e.g. see a_dictionary.
2. Setting content_type='json' on the Response object.
you can use replace("\'", '"') for that.
json = '''{\"strFileOpenDateAjxKey\": \"2018-01-16 12:40:22.526417\",
\"strFilePassengerAjxKey\": \"Zahra Walji\", \"strFileOpenMobileAjxKey\":
\"46464664\", \"strFileOpenDepartmentAjxKey\": \"Finance department\",
\"strFileOpenAccountCodeAjxKey\": \"CARTZS\",
\"strFileOpenProfileCodeAjxKey\": \"CARTZS\",
\"strFileopenOriginalCountryIdAjxKey\": 61, \"blnBoundAjxKey\": 1,
\"strTransactionCurrencyJsKey\": \"Shillings\",
\"intCurrencyPrecisionJsKey\": 3, \"strPackageTypeJsKey\": \"PKG\",
\"strUserNameAjxKey\": \"admin\", \"strPasswordAjxKey\": \"1234\"}'''
newString = json.replace("\'", '"')
print(newString)
check from here
that is the output when pressed the run in my side.
To anyone else who might be facing this issue.
simply put:
the original question seems to be taking JSON data (I emphasize, it's already JSON) and rendering it into JSON again.
dct_data = json_tour_data.dict
tour_data = json.dumps(dct_data)