I create a dictionary, ss, which has a total number of shares for a specific stock. Then I need to do some calculations and create a new dictionary, temp, which I will feed into my .html page for presentation. The update function does not work as I intend because all that is in temp is the last stock, not a listing of all the stocks I have purchased. When I print ss, there is [{key:value, etc}], but when I print temp there is no [ ] around the {key:value, etc}. I think I am missing something basic.
Also my .html page is not reading the temp dictionary as the page is empty. Here is the code:
#app.route("/")
#login_required
def index():
"""Show portfolio of stocks"""
#dictionary to feed data into index
temp={}
#Select from trades all stocks held by this user
ss = db.execute("SELECT SUM(shares), symbol FROM trades WHERE id=? GROUP BY symbol", session["user_id"])
print(ss)
#lookup current price for each stock and create index of all stocks held
for row in ss:
data=lookup(row["symbol"])
totval=row["SUM(shares)"]*data["price"]
temp1={"symbol":data["symbol"], "name":data["name"], "shares":row["SUM(shares)"], "price":data["price"], "total value":totval}
temp.update(temp1)
print(temp)
return render_template("index.html", temp=temp)
Any direction would be great. Thanks.
TL;DR
# Note use of the data["symbol"]
temp.update({
data["symbol"]: {
"symbol": data["symbol"],
"name": data["name"],
"shares": row["SUM(shares)"],
"price": data["price"],
"total value": totval
}
})
There are two generic manners of referencing related data, lists and dictionaries. In the simplest manner, consider:
apple
orange
pear
Using basic grammatical syntax, we can understand one is a list that "enumerates" it's part; the adjacency is the meaningful relationship between each individual part. The context and use of the list relates to it's external (variable) context.
A dictionary, on the other hand, specifies something specific is contextual to the list of definitions specifically. Consider:
fruit: apple, orange, pear
Here, fruit is an enumeration of different types of fruit; to define "fruit" is to give a list of qualifying "fruit" names contextual to an external (variable) context. Or:
fruit: An edible, usually sweet and fleshy form of such a structure.
Maybe a "real" definition.
So if we consider how we would refer to a list versus a dictionary, to add definitions to a list, we create a new list by (generally) appending a new item:
apple
orange
pear
+ kiwi
Before we had three, now we have four (contextually).
Whereas we append a new definition by specifying it's definition and naming it:
fruit: An edible, usually sweet and fleshy form of such a structure.
+ vegetable: A plant cultivated for its edible parts.
We could, if we want, update fruit by redefining it:
fruit: An edible, usually sweet and fleshy form of such a structure.
vegetable: A plant cultivated for its edible parts.
+ fruit: I like fruit.
Which gives us a dictionary of only it's constituent parts:
vegetable: A plant cultivated for its edible parts.
fruit: I like fruit.
Because you can only define (and update) the internal reference (fruit).
In psuedocode, a list:
fruits = []
fruits.add('apple')
fruits.add('orange')
fruits.add('pear')
// ['apple','orange','pear']
Likewise, a definition list works on the "key" relationship, and thus you may add or redefine a key relation:
foods = {}
foods['fruit'] = 'I like fruit.'
foods['vegetables'] = 'Gotta eat your veggies.'
// {
// fruit: 'I like fruit.',
// vegetables: 'Gotta eat your veggies!',
// }
In this sense, "updating" a dictionary means redefining and/or providing a new "key" relationship (internally).
Consider:
fruits = []
fruits.append('apple')
fruits.append('orange')
fruits.append('pear')
print(', '.join(fruits))
# apple, orange, pear
foods = {
'fruits': 'Fruits are good.'
}
# Adding a definition
foods['vegetables'] = 'Gotta eat your veggies!'
# Updating a definition
foods.update({
'fruits': 'I like fruit!',
'meats': 'Can haz meat?'
})
for food in foods.values():
print(food)
# I like fruit!
# Gotta eat your veggies!
# Can haz meat?
https://onlinegdb.com/SkneEJsw_
What you really need, then, are unique keys for your dictionary. Unique in the sense that within the dictionary's context, one key equals one definition. Which I think will look this:
# Note use of the data["symbol"]
temp.update({
data["symbol"]: {
"symbol": data["symbol"],
"name": data["name"],
"shares": row["SUM(shares)"],
"price": data["price"],
"total value": totval
}
})
Or directly:
temp[data["symbol"]] = {
"symbol": data["symbol"],
"name": data["name"],
"shares": row["SUM(shares)"],
"price": data["price"],
"total value": totval
}
Now you're updating your dictionary with meaningfully defined terms that resolve to a specific definition based on a key term.
There already is an individual row for each stock in ss. Remember, key/value pairs can be added to dictionaries quite simply by "declaring" them, eg row["totval"] = {value}. Hint, SELECT as much as possible in the SQL, eg symbol, name in the sql.
When I print ss, there is [{key:value, etc}], but when I print temp there is no [] around the {key:value, etc}. I think I am missing something basic.
I think you're mismatching types, which is a common mistake. I'm not sure what API/package you're using for db.execute, but that method seems to assign a list ([]) to ss. On the other hand, your temp value is a dict, ({}). I suggest one of two solutions.
If render_template expects temp to be a dict instead of a list, try this, as DinoCoderSaurus suggests:
def index():
# other code here
for row in ss:
data = lookup(row["symbol"])
totval = row["SUM(shares)"] * data["price"]
# notice omission of data["symbol"] in temp1 assignment
temp1 = { "name": data["name"], "shares": row["SUM(shares)"], "price": data["price"], "total value":totval }
# assign temp1 to data["symbol"] key in new dict
temp[data["symbol"]] = temp1
On the other hand, if render_template expects temp to be a list like ss seems to be, try:
def index():
# list to feed data into index (note change in data structure)
temp = []
# other code
for row in ss:
data = lookup(row["symbol"])
totval = row["SUM(shares)"] * data["price"]
temp1 = { "symbol": data["symbol"], "name": data["name"], "shares": row["SUM(shares)"], "price": data["price"], "total value": totval }
temp.append(temp1)
Related
I need some advice on the following code. I need to be able to extract two pieces of information but I need them in the same variable, for example,
item_out_list = [{
"fvRsNodeAtt": {
"attributes": {
"annotation": "",
"childAction": "",
"descr": "apple",
"value": "mango"
}
}
},
{"fvRsNodeAtt": {
"attributes": {
"annotation": "",
"childAction": "",
"descr": "peach",
"value": "banana"
}
}
}
]
static = [item['fvRsNodeAtt']['attributes']['descr'] for item in item_out_list]
print(static)
So the above gives me the value Apple, which is great but what I want to do is also grab "value" at the same time. I know could run another list and grab the value separate but I need them on same line, so when I print it out, I see apple mango.
I could join them together easily but I thought they might be an easier or efficient way of accessing the extra keys because it might be that I need 3 or 4 keys from the json above as it grows, for example, "annotation", "childAction", "descr", etc.
So the item_out_list is coming from a REST API, so there is multiple entries every time.
So when I lookup the values I want, it might be
apple mango
peach apple
banana orange.
so every time I query the API, I will get a different response, I am then using List comp to pull that out and save to CSV. The CSV isnt the problem, I just need to access more then one value and save to a variable
any help would be great
static = [item_all[item]['attributes']['descr'] + ', ' + item_all[item]['attributes']['value'] for item_all in item_out_list for item in item_all]
print (static)
output:
['apple, mango', 'peach, banana']
here is what you need
static = [list(item_out_list['fvRsNodeAtt']['attributes'].values())[2: 4]]
heres another way
static = [item_out_list['fvRsNodeAtt']['attributes']['descr'] + " " + item_out_list['fvRsNodeAtt']['attributes']['value']]
here how to get everything without looping
static = [item_out_list['fvRsNodeAtt']['attributes']]
I have been using rapid api to get some data on certain food products and below is an example of the json data i got back. I have been able to get some data such as the ingredients and but where i am struggling is getting the data that are nested inside each other. My question is how would i be able to get for example the data of "amount" which is inside nutrients in python.
"ingredients": "Whole Grain Corn, Sugar, Corn Syrup, Corn Meal"
"nutrition": {
"nutrients": [
{
"name": "Calcium",
"amount": 100.0,
"unit": "mg",
"percentOfDailyNeeds": 10.0
},
{
"name": "Carbohydrates",
"amount": 23.0,
"unit": "g",
"percentOfDailyNeeds": 7.67
},
The way which i was able to get the ingredients was by doing this which worked and printed out the ingredients
ingredients = response.json().get("ingredients")
But how would i do the same thing to get specific data inside nutrients such as the "name" carbohydrates?
It's a list (https://docs.python.org/3/tutorial/datastructures.html).
You can access it by the index (starting at 0). So to get Carbohydrates you would do dictName["nutrition"]["nutrients"][1]["name"]. To get Calcium you would do dictName["nutrition"]["nutrients"][0]["name"].
It's probably easiest to just assign and then loop through with
nutrients = dictName["nutrition"]["nutrients"]
for nutrient in nutrients:
print(nutrient["name"])
You can do this with a filter too, which would look something like this
carbs = list(filter(lambda x: x['name'] == 'Carbohydrates', response.json()['nutrition']['nutrients']))
Which is a bit more compact. The output you get is
[{'name': 'Carbohydrates', 'amount': 23.0, 'unit': 'g', 'percentOfDailyNeeds': 7.67}]
Simplest solution would be to unpack your values one level at a time:
data = response.json()
nutrition = data.get("nutrition", [])
nutrients = [
item
for item in nutrition.get("nutrients", [])
if item.get("name") == "Carbohydrates"
]
if nutrients:
print(nutrients[0])
The trickiest part is working with the array. I've used list comprehension to build a new filtered list (some would prefer filter function), and then printed an item if the list is non-empty
I've seen something like this construction somewhere:
list.append({
'title': scName,
'link': scLink,
})
print('Names:', list['title'])
print('Links:', list['link'])
Can you please show a working example?
Here, the dict is being appended to a list. And to use inner data, we've to put list[<index_of_inner_dict>][key_of_that_data]. In short, we've to go to the whole inner data index, then the index of required value. Here we have only one dict, it is simply list[0]. Also, scName and scLink are not defined, I'm assuming that they are simple strings. Your code:
l=[]
l.append({
'title': "scName",
'link': "scLink"
})
print (l)
print('Names:', l[0]['title'])
print('Links:', l[0]['link'])
What I believe you are asking is for someone to give you an example of how to use a dictionary. To give some background on dictionaries, they store information in key-value pairs. The key, which is the "index name" you mentioned in your title is mapped to a value stored. You can read here if you are still confused.
For the code example you gave, what it looks like you are attempting to do is add multiple dictionaries to a list and then access those values. Here is an example.
lst_of_employees = []
lst_of_employees.append({"name": "John", "salary": "10000"})
lst_of_employees.append({"name": "Jane", "salary": "20000"})
for emp in lst_of_employees:
print(f"{emp['name']} makes ${emp['salary']} a year.")
You can make the value of the key-value pair whatever you would like. Here is an example with the value stored at the key "salary" as another dictionary.
lst_of_employees = []
lst_of_employees.append({"name": "John", "salary": {"base": 8000, "bonus": 2000}})
lst_of_employees.append({"name": "Jane", "salary": {"base": 15000, "bonus": 5000}})
for emp in lst_of_employees:
employee = emp["name"]
base = emp["salary"]["base"]
bonus = emp["salary"]["bonus"]
print(f"{emp['name']} makes ${base+bonus} a year.")
I have a list like this:
data.append(
{
"type": type,
"description": description,
"amount": 1,
}
)
Every time there is a new object I want to check if there already is an entry in the list with the same description. If there is, I need to add 1 to the amount.
How can I do this the most efficient? Is the only way going through all the entries?
I suggest making data a dict and using the description as a key.
If you are concerned about the efficiency of using the string as a key, read this: efficiency of long (str) keys in python dictionary.
Example:
data = {}
while loop(): # your code here
existing = data.get(description)
if existing is None:
data[description] = {
"type": type,
"description": description,
"amount": 1,
}
else:
existing["amount"] += 1
In either case you should first benchmark the two solutions (the other one being the iterative approach) before reaching any conclusions about efficiency.
I am using a public API at www.gpcontract.co.uk to populate a large variably nested dictionary representing a hierarchy of UK health organisations.
Some background information
The top level of the hierarchy is the four UK countries (England, Scotland, Wales and Northern Ireland), then regional organisations all the way down to individual clinics. The depth of the hierarchy is different for each of the countries and can change depending on the year. Each organisation has a name, orgcode and dictionary listing its child organisations.
Unfortunately, the full nested hierarchy is not available from the API, but calls to http://www.gpcontract.co.uk/api/children/[organisation code]/[year] will return the immediate child organisations of any other.
So that the hierarchy can be easily navigated in my app, I want to generate an offline dictionary of this full hierarchy (on a per year basis) which will be saved using pickle and bundled with the app.
Getting this means a lot of API calls, and I am having trouble converting the returned JSON into the dictionary object I require.
Here is an example of one tiny part of the hierarchy (I have only shown a single child organisation as an example).
JSON hierarchy example
{
"eng": {
"name": "England",
"orgcode": "eng",
"children": {}
},
"sco": {
"name": "Scotland",
"orgcode": "sco",
"children": {}
},
"wal": {
"name": "Wales",
"orgcode": "wal",
"children": {}
},
"nir": {
"name": "Northern Ireland",
"orgcode": "nir",
"children": {
"blcg": {
"name": "Belfast Local Commissioning Group",
"orgcode": "blcg",
"children": {
"abc": {
"name": "Random Clinic",
"orgcode": "abc",
"children": {}
}
}
}
}
}
}
Here’s the script I’m using to make the API calls and populate the dictionary:
My script
import json, pickle, urllib.request, urllib.error, urllib.parse
# Organisation hierarchy may vary between years. Set the year here.
year = 2017
# This function returns a list containing a dictionary for each child organisation with keys for name and orgcode
def get_child_orgs(orgcode, year):
orgcode = str(orgcode)
year = str(year)
# Correct 4-digit year to 2-digit
if len(year) > 2:
year = year[2:]
try:
child_data = json.loads(urllib.request.urlopen('http://www.gpcontract.co.uk/api/children/' + str(orgcode) + '/' + year).read())
output = []
if child_data != []:
for item in child_data['children']:
output.append({'name' : item['name'], 'orgcode' : str(item['orgcode']).lower(), 'children' : {}})
return output
except urllib.error.HTTPError:
print('HTTP error!')
except:
print('Other error!')
# I start with a template of the top level of the hierarchy and then populate it
hierarchy = {'eng' : {'name' : 'England', 'orgcode' : 'eng', 'children' : {}}, 'nir' : {'name' : 'Northern Ireland', 'orgcode' : 'nir', 'children' : {}}, 'sco' : {'name' : 'Scotland', 'orgcode' : 'sco', 'children' : {}}, 'wal' : {'name' : 'Wales', 'orgcode' : 'wal', 'children' : {}}}
print('Loading data...\n')
# Here I use nested for loops to make API calls and populate the dictionary down the levels of the hierarchy. The bottom level contains the most items.
for country in ('eng', 'nir', 'sco', 'wal'):
for item1 in get_child_orgs(country, year):
hierarchy[country]['children'][item1['orgcode']] = item1
for item2 in get_child_orgs(item1['orgcode'], year):
hierarchy[country]['children'][item1['orgcode']]['children'][item2['orgcode']] = item2
# Only England and Wales hierarchies go deeper than this
if country in ('eng', 'wal'):
level3 = get_child_orgs(item2['orgcode'], year)
# Check not empty array
if level3 != []:
for item3 in level3:
hierarchy[country]['children'][item1['orgcode']]['children'][item2['orgcode']]['children'][item3['orgcode']] = item3
level4 = get_child_orgs(item3['orgcode'], year)
# Check not empty array
if level4 != []:
for item4 in level4:
hierarchy[country]['children'][item1['orgcode']]['children'][item2['orgcode']]['children'][item3['orgcode']]['children'][item4['orgcode']] = item4
# Save the completed hierarchy with pickle
file_name = 'hierarchy_' + str(year) + '.dat'
with open(file_name, 'wb') as out_file:
pickle.dump(hierarchy, out_file)
print('Success!')
The problem
This seems to work most of the time, but it feels hacky and sometimes crashes when a nested for loop returns a "NoneType is not iterable error". I realise this is making a lot of API calls and takes several minutes to run, but I cannot see a way around this, as I want the completed hierarchy available offline for the user to make the data searchable quickly. I will then use the API in a slightly different way to get the actual healthcare data for the chosen organisation.
My question
Is there a cleaner and more flexible way to do this that would accommodate the variable nesting of the organisation hierarchy?
Is there a way to do this significantly more quickly?
I am relatively inexperienced with JSON so any help would be appreciated.
I think this question may be better suited over on the Code Review Stack Exchange, but as you mention that your code sometimes crashes and returns NoneType errors I'll give it the benefit of the doubt.
Looking at your description, this is what stands out to me
Each organisation has a name, orgcode and dictionary listing its child organisations. [API calls] will return the immediate child organisations of any other.
So, what this suggests to me (and how it looks in your sample data) is that all your data is exactly equivalent; the hierarchy only exists due to the nesting of the data and is not enforced by the format of any particular node.
This, consequently, means that you should be able to have a single piece of code which handles the nesting of an infinitely (or arbitrarily, if you prefer) deep tree. Obviously, you do this for the API call itself (get_child_orgs()), so just replicate that for constructing the tree.
def populate_hierarchy(organization,year):
""" Recursively Populate the Organization Hierarchy
organization should be a dict with an "orgcode" key with a string value
and "children" key with a dict value.
year should be a 2-4 character string representing a year.
"""
orgcode = organization['orgcode']
## get_child_orgs returns a list of organizations
children = get_child_orgs(orgcode,year)
## get_child_orgs returns None on Errors
if children:
for child in children:
## Add child to the current organization's children, using
## orgcode as its key
organization['children'][child['orgcode']] = child
## Recursively populate the child's sub-hierarchy
populate_hierarchy(child,year)
## Technically, the way this is written, returning organization is
## pointless because we're modifying organization in place, but I'm
## doing it anyway to explicitly denote the end of the function
return organization
for country in hierarchy.values():
populate_hierarchy(country,year)
It's worth noting (since you were checking for empty lists prior to iterating in your original code) that for x in y still functions correctly if y is an empty list, so you don't need to check.
The NoneType Error likely arises because you catch the Error in get_child_orgs and then implicitly return None. Therefore, for example level3 = get_child_orgs[etc...] results in level3 = None; this leads to if None != []: in the next line being True, and then you try to iterate over None with for item3 in None: which would raise the error. As noted in the code above, this is why I check the truthiness of children.
As for whether this can be done more quickly, you can try working with the threading/multiprocessing modules. I just don't know how profitable either of those will be for three reasons:
I haven't tried out the API, so I don't know how much time you have to gain from implementing multiple threads/processes
I have seen API's which timeout requests from IP Addresses when you query too quickly/too often (which would make the implementation pointless)
You say you're only running this process once per year, so runtime in perspective of a full year seems pretty insignificant (obviously, unless the current API calls are taking literal days to complete)
Finally, I would simply question whether pickle is the appropriate method of storing the information, or if you wouldn't just be better off using json.dump/load (for the record, the json module doesn't care if you change the extension to .dat if you're partial to that extension name).