We Keep Coding

Accessing Values In Nested Dict In Python - String Indices Must Be Integers

I've gone through other similar problems here, but still can't identify my problem.
I have this JSON data returned from an API call:
{
"Open": {
"1638316800000": 120.5400009155,
"1640995200000": 106.1399993896,
"1643673600000": 67.2799987793,
"1646092800000": 65.4300003052,
"1648771200000": 50.1800003052,
"1651104000000": 31.5699996948
},
"High": {
"1638316800000": 126.75,
"1640995200000": 106.8000030518,
"1643673600000": 71.5,
"1646092800000": 66.5400009155,
"1648771200000": 50.2599983215,
"1651104000000": 31.6900005341
},
"Low": {
"1638316800000": 88.4000015259,
"1640995200000": 50.0,
"1643673600000": 53.5,
"1646092800000": 33.4599990845,
"1648771200000": 30.5799999237,
"1651104000000": 30.5209999084
},
"Close": {
"1638316800000": 103.6900024414,
"1640995200000": 65.7399978638,
"1643673600000": 67.5599975586,
"1646092800000": 50.2400016785,
"1648771200000": 31.2199993134,
"1651104000000": 30.6100006104
}
}
All I'm trying to do is assign the "Close" data to a new variable close and return close instead of the entire dictionary response.
Here is what I'm currently trying and I've tried different variations and all keep returning "string indices must be integers"
#app.route("/history")
def display_history():
symbol = request.args.get('symbol', default="AAPL")
period = request.args.get('period', default="1y")
interval = request.args.get('interval', default="1mo")
quote = yf.Ticker(symbol)
hist = quote.history(period=period, interval=interval)
data = hist.to_json()
close = data["Close"]
return close

You are trying to use data, which is a string with json format, as your interpreter told you.
In order to read it as a dictionary with the key "Close", you can use the function loads from json package. It deserializes the string to a Python dictionary :
data = hist.to_json()
data = json.loads(data)
close = data["Close"]
Additionally, it appears that Ticker.history(), from yfinance module, returns a pandas Dataframe. If it is the case, you can use this instead :
data = hist.to_dict()
close = data['Close']
This way, the data is not converted to Json then back to Python dictionary again but straight to a dictionary.

Exctract a value from a Json file(python)

Hi i'm not an expert and this problem kept me stuck for such a long time I hope that someone here can help me
i would like to exctract the value "interestExpense" from the following json file:
{'incomeBeforeTax': 17780000000,
'minorityInterest': 103000000,
'netIncome': 17937000000,
'sellingGeneralAdministrative': 5918000000,
'grossProfit': 16507000000,
'ebit': 10589000000,
'endDate': 1640908800,
'operatingIncome': 10589000000,
'interestExpense': -1803000000,
'incomeTaxExpense': -130000000,
'totalRevenue': 136341000000,
'totalOperatingExpenses': 125752000000,
'costOfRevenue': 119834000000,
'totalOtherIncomeExpenseNet': 7191000000,
'netIncomeFromContinuingOps': 17910000000,
'netIncomeApplicableToCommonShares': 17937000000}
In this case the result should be -130000000 as a string but i m trying to find a way to create an list(or an array) with all those floats so that i can decide which one to pick, i have no idea how to manipulate this kind of data(json)
For example
print(list[0])
should return 17780000000(the value associated with incomeBeforeTax)
is this actually possible?
The output is generated from this code:
annual_is_stms=[]
url_financials ='https://finance.yahoo.com/quote/{}/financials?p{}'
stock= 'F'
response = requests.get(url_financials.format(stock,stock),headers=headers)
soup = BeautifulSoup(response.text,'html.parser')
pattern = re.compile(r'\s--\sData\s--\s')
script_data = soup.find('script',text=pattern).contents[0]
script_data[:500]
script_data[-500:]
start = script_data.find("context")-2
json_data =json.loads(script_data[start:-12])
json_data['context']['dispatcher']['stores']['QuoteSummaryStore'].keys()
#all data relative financials
annual_is=json_data['context']['dispatcher']['stores']['QuoteSummaryStore']['incomeStatementHistory']['incomeStatementHistory']
for s in annual_is:
statement = {}
for key, val in s.items():
try:
statement[key] = val['raw']
except TypeError:
continue
except KeyError:
continue
annual_is_stms.append(statement)
print(annual_is_stms[0])

If you are using python, you need to include the json module and parse it as an object:
import json
# some JSON:
x = '{ "name":"John", "age":30, "city":"New York"}'
# parse x:
y = json.loads(x)
# the result is a Python dictionary:
print(y["age"])
Regards
L.

Ok, so the output snippet you posted comes from this line:
print(annual_is_stms[0])
If you now want the: -1803000000 you should do:
print(annual_is_stms[0]['interestExpense'])
If you want the: -130000000 you should do:
print(annual_is_stms[0]['incomeTaxExpense'])
and if you want the: 17780000000 you should do:
print(annual_is_stms[0]['incomeBeforeTax'])

Copy and paste this into Python.
data = {'incomeBeforeTax': 17780000000,
'minorityInterest': 103000000,
'netIncome': 17937000000,
'sellingGeneralAdministrative': 5918000000,
'grossProfit': 16507000000,
'ebit': 10589000000,
'endDate': 1640908800,
'operatingIncome': 10589000000,
'interestExpense': -1803000000,
'incomeTaxExpense': -130000000,
'totalRevenue': 136341000000,
'totalOperatingExpenses': 125752000000,
'costOfRevenue': 119834000000,
'totalOtherIncomeExpenseNet': 7191000000,
'netIncomeFromContinuingOps': 17910000000,
'netIncomeApplicableToCommonShares': 17937000000}
print(data['interestExpense'])

combining two pandas dataframes as single json output

I'm trying to combine two pandas dataframes into a single JSON output.
The json output below is the result from this code - df.to_json(orient = "split")
{
columns: [],
index: [],
data: [
[
"COMPANY ONE",
"123 HAPPY PLACE",
"GOTHAM CITY",
"NJ",
12345,
"US",
8675309,
"",
"",
"",
"",
""
],
[.....]
]
}
A little background, I get the data from a csv file, and usually I have to separate the file in two parts, one good and the other bad. I've been using pandas for this process, which is great. So df contains the good data and say dfbad contains the bad data.
I used df.to_json(orient = "split") to output the good data, which I really like the structure of it. Now I want to do the same thing for the bad data, same structure, so something like this:
[{good}, {bad}]
I apologize in advance if the example above is not clear.
I tried to this:
jsonify(good = df.to_json(orient = "split"), bad = dfbad.to_json(orient = "split"))
But i know this is not going to work because the result for good and bad are turned into a string; which I don't want, I want to be able to have access to it.
data_dict = {}
data_dict['bad'] = dfbad.to_dict()
data_dict['good'] = df.to_dict()
return pd.json.dumps(data_dict)
This returns fine as a json, but not the structure I want, the way .to_json(orient = "split") does, unless I have to customize it.
Can anybody help with this issue? or can pinpoint in another direction how to solve this issue.
Thanks in advance!
UPDATE:
I found the solution, here is what I did:
good_json = df.to_json(orient="split")
bad_json = dfbad.to_json(orient="split")
return jsonify(bad = json.loads(bad_json), good = json.loads(good_json))
I added json.loads, you have to import it - import json - and it's now returning as a JSON output. If you have other suggestions, please let me know. I'm open to learn more about Pandas.

Python - Count JSON elements before extracting data

I use an API which gives me a JSON file structured like this:
{
offset: 0,
results: [
{
source_link: "http://www.example.com/1",
source_link/_title: "Title example 1",
source_link/_source: "/1",
source_link/_text: "Title example 1"
},
{
source_link: "http://www.example.com/2",
source_link/_title: "Title example 2",
source_link/_source: "/2",
source_link/_text: "Title example 2"
},
...
And I use this code in Python to extract the data I need:
import json
import urllib2
u = urllib2.urlopen('myapiurl')
z = json.load(u)
u.close
link = z['results'][1]['source_link']
title = z['results'][1]['source_link/_title']
The problem is that to use it I have to know the number of the element from which I'm extracting the data. My results can have different length every time, so what I want to do is to count the number of elements in results at first, so I would be able to set up a loop to extract data from each element.

To check the length of the results key:
len(z["results"])
But if you're just looping around them, a for loop is perfect:
for result in x["results"]:
print(result["source_link"])

You didn't need to know the length of the result, you are fine with a for loop:
for result in z['results']:
# process the results here
Anyway, if you want to know the length of 'results': len(z.results)

If you want to get the length, you can try:
len(z['result'])
But in python, what we usually do is:
for i in z['result']:
# do whatever you like with `i`
Hope this helps.

You don't need, or likely want, to count them in order to loop over them, you could do:
import json
import urllib2
u = urllib2.urlopen('myapiurl')
z = json.load(u)
u.close
for result in z['results']:
link = result['source_link']
title = result['source_link/_title']
# do something with link/title
Or you could do:
u = urllib2.urlopen('myapiurl')
z = json.load(u)
u.close
link = [result['source_link'] for result in z['results']]
title = [result['source_link/_title'] for result in z['results']]
# do something with links/titles lists

Few pointers:
No need to know results's length to iterate it. You can use for result in z['results'].
lists start from 0.
If you do need the index take a look at enumerate.

use this command to print the result on the terminal and then can check the number of results
print(len(z['results'][0]))

Raster: How to get elevation at lat/long using python?

I also posted this question in the GIS section of SO. As I'm not sure if this rather a 'pure' python question I also ask it here again.
I was wondering if anyone has some experience in getting elevation data from a raster without using ArcGIS, but rather get the information as a python list or dict?
I get my XY data as a list of tuples.
I'd like to loop through the list or pass it to a function or class-method to get the corresponding elevation for the xy-pairs.
I did some research on the topic and the gdal API sounds promising. Can anyone advice me how to go about things, pitfalls, sample code? Other options?
Thanks for your efforts, LarsVegas

I recommend checking out the Google Elevation API
It's very straightforward to use:
http://maps.googleapis.com/maps/api/elevation/json?locations=39.7391536,-104.9847034&sensor=true_or_false
{
"results" : [
{
"elevation" : 1608.637939453125,
"location" : {
"lat" : 39.73915360,
"lng" : -104.98470340
},
"resolution" : 4.771975994110107
}
],
"status" : "OK"
}
note that the free version is limited to 2500 requests per day.

We used this code to get elevation for a given latitude/longitude (NOTE: we only asked to print the elevation, and the rounded lat and long values).
import urllib.request
import json
lati = input("Enter the latitude:")
lngi = input("Enter the longitude:")
# url_params completes the base url with the given latitude and longitude values
ELEVATION_BASE_URL = 'http://maps.googleapis.com/maps/api/elevation/json?'
URL_PARAMS = "locations=%s,%s&sensor=%s" % (lati, lngi, "false")
url=ELEVATION_BASE_URL + URL_PARAMS
with urllib.request.urlopen(url) as f:
response = json.loads(f.read().decode())
status = response["status"]
result = response["results"][0]
print(float(result["elevation"]))
print(float(result["location"]["lat"]))
print(float(result["location"]["lng"]))

Have a look at altimeter a wrapper for the Google Elevation API

Here is the another one nice API that I`v built: https://algorithmia.com/algorithms/Gaploid/Elevation
import Algorithmia
input = {
"lat": "50.2111",
"lon": "18.1233"
}
client = Algorithmia.client('YOUR_API_KEY')
algo = client.algo('Gaploid/Elevation/0.3.0')
print algo.pipe(input)