I am trying to iterate over a JSON object, using simplejson.
def main(arg1):
response = urllib2.urlopen("http://search.twitter.com/search.json?q=" + arg1) #+ "&rpp=100&page=15")
twitsearch = simplejson.load(response)
twitsearch = twitsearch['results']
twitsearch = twitsearch['text']
print twitsearch
I am passing a list of values to search for in Twitter, like "I'm", "Think", etc.
The problem is that there are multiple text fields, one each for every Tweet. I want to iterate over the entire JSON object, pulling out the "text" field.
How would I do this? I'm reading the documentation and can't see exactly where it talks about this.
EDIT: It appears to be stored as a list of JSON objects.
Trying to do this:
for x in twitsearch:
x['text']
How would I store x['text'] in a list? Append?
Note that
twitsearch['results']
is a Python list. You can iterate over that list, storing the text component of each of those objects in your own list. A list comprehension would be a good thing to use here.
text_list = [x['text'] for x in twitsearch['results']]
Easy. Figured it out.
tweets = []
for x in twitsearch:
tweets.append(x['text'])
Related
So I am struggling with getting a value from a JSON response. Looking in other post I have managed to write this code but when I try to search for the key (character_id) that I want in the dictionary python says that the key doesn't exist. My solution consists in getting the JSON object from the response, converting it into a string with json.dumps() and the converting it into a dictionary with json.loads(). Then I try to get 'character_id' from the dictionary but it doesn't exist. I am guessing it is related with the format of the dictionary but I have little to none experience in python. The code that makes the query and tries to get the values is this: (dataRequest is a fuction that makes the request and return the response from the api)
characterName = sys.argv[1];
response = dataRequest('http://census.daybreakgames.com/s:888/get/ps2:v2/character/?name.first_lower=' + characterName + '&c:show=character_id')
jsonString = json.dumps(response.json())
print(jsonString)
dic = json.loads(jsonString)
print(dic)
if 'character_id' in dic:
print(dic['character_id'])
The output of the code is:
{"character_list": [{"character_id": "5428662532301799649"}], "returned": 1}
{'character_list': [{'character_id': '5428662532301799649'}], 'returned': 1}
Welcome #Prieto! From what I can see, you probably don't need to serialize/de-serialize the JSON -- response.json() returns a python dictionary object already.
The issue is that you are looking for the 'character_id' key at the top-level of the dictionary, when it seems to be embedded inside another dictionary, that is inside a list. Try something like this:
#...omitted code
for char_obj in dic["character_list"]:
if "character_id" in char_obj:
print(char_obj["character_id"])
if your dic is like {"character_list": [{"character_id": "5428662532301799649"}], "returned": 1}
you get the value of character_id by
print(dic['character_list'][0][character_id])
The problem here is that you're trying to access a dictionary where the key is actually character_list.
What you need to do is to access the character_list value and iterate over or filter the character_id you want.
Like this:
print(jsonString)
dic = json.loads(jsonString)
print(dic)
character_information = dic['character_list'][0] # we access the character list and assume it is the first value
print(character_information["character_id"]) # this is your character id
The way I see it, the only hiccup with the code is this :
if 'character_id' in dic:
print(dic['character_id'])
The problem is that, the JSON file actually consists of actually 2 dictionaries , first is the main one, which has two keys, character_list and returned. There is a second sub-dictionary inside the array, which is the value for the key character_list.
So, what your code should actually look like is something like this:
for i in dic["character_list"]:
print(i["character_id"])
On a side-note, it will help to look at JSON file in this way :
{
"character_list": [
{
"character_id": "5428662532301799649"
}
],
"returned": 1
}
,where, elements enclosed in curly-brackets'{}' imply they are in a dictionary, whereas elements enclosed in curly-brackets'[]' imply they are in a list
I have pulled JSON data from a url. The result is a dictionary. How can I transform this dictionary so metric is a column, and the time is the index for each value
Thanks in advance
time------------------------AdrActCnt-----BlkCnt------BlkSizeByte
2021-01-28T00:00:00.000Z----1097896.0-----145.0-------190568423.0
2021-01-29T00:00:00.000Z----1208741.0-----152.0-------199725189.0
2021-01-29T00:00:00.000Z----1087755.0-----136.0-------177349536.0
Output:
{"metricData":{"metrics":["AdrActCnt","BlkCnt","BlkSizeByte"],"series":
[{"time":"2021-01-28T00:00:00Z","values"["1097896.0","145.0","190568423.0"]},
{"time":"2021-01-29T00:00:00Z","values":["1208741.0","152.0","199725189.0"]},
{"time":"2021-01-30T00:00:00Z","values":["1087755.0","136.0","177349536.0"]}
You may be looking for a dict comprehension, which is similar to a list comprehension, just creates a dictionary at the end:
liststuff = [{"time":"2021-01-28T00:00:00.000Z","values":["1097896.0","145.0","190568423.0"]},{"time":"2021-01-29T00:00:00.000Z","values":["1208741.0","152.0","199725189.0"]},{"time":"2021-01-30T00:00:00.000Z","values":["1087755.0","136.0","177349536.0"]}]
dictstuff = {item['time']:item['values'] for item in liststuff}
print(dictstuff)
{'2021-01-28T00:00:00.000Z': ['1097896.0', '145.0', '190568423.0'], '2021-01-29T00:00:00.000Z': ['1208741.0', '152.0', '199725189.0'], '2021-01-30T00:00:00.000Z': ['1087755.0', '136.0', '177349536.0']}
liststuff is your data, just needed [] wrapping (I assume that's a typo in the question, it's not valid JSON without the brackets). If you need help with parsing the string, use json.loads() (from the json module) to make it actual Python data:
import json
jsonstuff = '[{"time":"2021-01-28T00:00:00.000Z","values":["1097896.0","145.0","190568423.0"]},{"time":"2021-01-29T00:00:00.000Z","values":["1208741.0","152.0","199725189.0"]},{"time":"2021-01-30T00:00:00.000Z","values":["1087755.0","136.0","177349536.0"]}]'
liststuff = json.loads(jsonstuff)
(here jsonstuff is the string you've downloaded)
What this code does it gets the values of a json.loads file. It gives me a list of dictionary that are organized by dates. This code works, my understanding is, i am taking the the first value in the list of dictionaries, so the first dictionary, but shouldn't. self.get_jsonparsed_data(self.ticker_text.get().upper())[0] work as well? In my case it doesn't, I was hoping if someone can explain why I does not work.
def get_jsonparsed_data(self, ticker):
#quote
url = (f"https://financialmodelingprep.com/api/v3/quote/{ticker}?)
response = urlopen(url)
data = response.read().decode("utf-8")
return json.loads(data)
def search_info(self):
#self.info.delete(0, END)
recent_filing = []
for header in self.get_jsonparsed_data(self.ticker_text.get().upper())[:1]:
recent_filing.append(header)
ticker = self.ticker_text.get()
#output dictionary values with proper format
try:
recent_filing_dict = recent_filing[0]
This works.
I get the first dictionary which is what i want but when i do self.get_jsonparsed_data(self.ticker_text.get().upper())[0] instead of self.get_jsonparsed_data(self.ticker_text.get().upper())[:1] it gives me an error
which pretty much is saying there isnt any values appended to recent_filing_dict. I was just hoping if someone can explain why?
"for" loops through iterable and self.get_jsonparsed_data(self.ticker_text.get().upper())[0] seemingly returns an item rather than iterable (list) while self.get_jsonparsed_data(self.ticker_text.get().upper())[:1] returns an iterable (single item list) which is iterated over by for loop
I have a json code from API and I want to get new chat members with the code below but I only get the first two results and not the last (Tester). Why? It should itereate through the whole json file, shouldn't it?
r = requests.get("https://api.../getUpdates").json()
chat_members = []
a = 0
for i in r:
chat_members.append(r['result'][a]['message']['new_chat_members'][0]['last_name'])
a = a + 1
Json here:
{"ok":true,"result":[{"update_id":213849278,
"message":{"message_id":37731,"from":{"id":593029363,"is_bot":false,"first_name": "#tutu"},"chat":{"id":-1001272017174,"title":"tester account","username":"v_glob","type":"supergroup"},"date":1537470595,"new_chat_participant":{"id":593029363,"is_bot":false,"first_name":"tutu "},"new_chat_member":{"id":593029363,"is_bot":false,"first_name":"\u7535\u62a5\u589e\u7c89\uff0c\u4e2d\u82f1\u6587\u5ba2\u670d\uff0c\u62c9\u4eba\u6e05\u5783\u573e\u8f6f\u4ef6\uff0c\u5e7f\u544a\u63a8\u5e7f\uff0cKYC\u6750\u6599\u8ba4\u8bc1\uff0c","last_name":"#tutupeng"},"new_chat_members":[{"id":593029363,"is_bot":false,"first_name":"\u7535\u62a5\u589e\u7c89\uff0c\u4e2d\u82f1\u6587\u5ba2\u670d\uff0c\u62c9\u4eba\u6e05\u5783\u573e\u8f6f\u4ef6\uff0c\u5e7f\u544a\u63a8\u5e7f\uff0cKYC\u6750\u6599\u8ba4\u8bc1\uff0c","last_name":"#tutu"}]}},{"update_id":213849279,
"message":{"message_id":37732,"from":{"id":658150956,"is_bot":false,"first_name":"Rebecca","last_name":"Lawson"},"chat":{"id":-10012720,"title":"v glob OFFICIAL","username":"v_glob","type":"supergroup"},"date":1537484441,"new_chat_participant":{"id":65815,"is_bot":false,"first_name":"Rebecca","last_name":"Lawson"},"new_chat_member":{"id":65815,"is_bot":false,"first_name":"Rebecca","last_name":"Lawson"},"new_chat_members":[{"id":65815,"is_bot":false,"first_name":"Rebecca","last_name":"Lawson"}]}},{"update_id":213849280,
"message":{"message_id":12,"from":{"id":696749142,"is_bot":false,"first_name":"daniel","language_code":"cs-cz"},"chat":{"id":696749142,"first_name":"daniel","type":"private"},"date":1537537013,"text":"/stat","entities":[{"offset":0,"length":5,"type":"bot_command"}]}},{"update_id":213849281,
"message":{"message_id":37740,"from":{"id":669620,"is_bot":false,"first_name":"Ivan","last_name":"Tester"},"chat":{"id":-100127201,"title":"test account","username":"v_glob","type":"supergroup"},"date":1537537597,"new_chat_participant":{"id":669620191,"is_bot":false,"first_name":"Ivan","last_name":"Tester"},"new_chat_member":{"id":669620191,"is_bot":false,"first_name":"Ivan","last_name":"Tester"},"new_chat_members":[{"id":669620191,"is_bot":false,"first_name":"Ivan","last_name":"Tester"}]}}]}
Because you iterate over the entire response dict. The top level only has two items, so that's what you iterate over. Note that you don't actually use the iterator variable, and you have a completely unnecessary separate counter.
Instead, you should be iterating over the result dict:
for result in r['result']:
if "new_chat_members" in result['message']:
chat_members.append(result['message']['new_chat_members'][0]['last_name'])
A colleague of mine has come up with a solution:
for i in l['result']:
chat_members.append(i['message']['new_chat_member']['first_name'])
To sum up: Iterate through 'result' with no positional arguments
I have got this json file
[{"url": ["instrumentos-musicales-126030594.htm", "liquidacion-muebles-y-electrodomesticos-127660457.htm"], "title": ["INSTRUMENTOS MUSICALES", "LIQUIDACION, MUEBLES Y ELECTRODOMESTICOS"]}]
mydata = json.load(json_data)
And then I'd like to manipulate mydata to append url-title pair to a list paginas
What would be a good solution in Python? I want to be able to have something like
paginas[0].url
instrumentos-musicales-126030594.htm
paginas[0].title
"INSTRUMENTOS MUSICALES"
paginas[1].url
"liquidacion-muebles-y-electrodomesticos-127660457.htm"
paginas[1].title
"LIQUIDACION, MUEBLES Y ELECTRODOMESTICOS"
paginas = mydata[0]
paginas["url"].append(new_url)
paginas["title"].append(new_title)
You still need to use json.dump to save it, of course.
If you want to extract the values to a new list:
new_list =[mydata[0]["url"],mydata[0]["title"]]
will give you:
[['instrumentos-musicales-126030594.htm', 'liquidacion-muebles-y-electrodomesticos-127660457.htm'], ['INSTRUMENTOS MUSICALES', 'LIQUIDACION, MUEBLES Y ELECTRODOMESTICOS']]
To access elements you can using indexing:
`mydata[0]["url"][0]` will give you "instrumentos-musicales-126030594.htm"
mydata[0]["title"][0] will give you "INSTRUMENTOS MUSICALES" etc..
You don't need to do anything to your data structure.