Related
I have a dictionary filled with 3 keys: "Ticker", "Title" and "Value". The ticker key contains 100 stock tickers that corresponds to the value of the purchase and title that are in the same position. So here is an example of the dictionary:
{'Ticker': {0: 'AKUS', 1: 'HHC', 2: 'IOVA', 3: 'DDOG'},
'Title': {0: '10%', 1: 'Dir, 10%', 2: 'Dir', 3: 'Dir'},
'Value': {0: '+$374,908,350', 1: '+$109,214,243', 2: '+$65,000,000', 3: '+$49,999,940'}}
So "AKUS" corresponds with the 10% and +$374,908,350.
I am only showing 4 items in the dictionary but my actual dictionary has 100.
My question is regarding a new dictionary that only contains tickers and values but everything in that dictionary has the same title.
For example, I want to create a 10% dictionary that contains all the tickers and values of stocks where the title contained 10%.
I know some stocks have multiple titles but I don't mind the same stocks being in more than one dictionary. Would some one be able to let me know how I should go about doing this? Thank you in advance, I have been stuck on this for a while.
Simple to do using pandas if you are OK using that; so assuming your dictionary is named d:
df = pd.DataFrame.from_dict(d)
df10 = df[df['Title'].str.contains('10%')]
print(df10)
produces
Ticker Title Value
0 AKUS 10% +$374,908,350
1 HHC Dir, 10% +$109,214,243
As you only want to get the "Ticker" and "Value" values that have a "Title" that contains "10%" in its value, you need to filter both "Ticker" and "Value" against that. You can do it verbose or using dictionary comprehension:
stocks = {'Ticker': {0: 'AKUS', 1: 'HHC', 2: 'IOVA', 3: 'DDOG'},
'Title': {0: '10%', 1: 'Dir, 10%', 2: 'Dir', 3: 'Dir'},
'Value': {0: '+$374,908,350', 1: '+$109,214,243', 2: '+$65,000,000', 3: '+$49,999,940'}
}
ten_percent_stocks = {"Ticker":{}, "Value": {}}
for k, v in stocks["Title"].items():
if "10%" in v:
ten_percent_stocks["Ticker"][k] = stocks["Ticker"][k]
ten_percent_stocks["Value"][k] = stocks["Value"][k]
With dictionary comprehension you could get the same result by doing this:
ten_percent_stocks = {"Ticker": {k: v for k, v in stocks["Ticker"].items()
if "10%" in stocks["Title"][k]},
"Value": {k: v for k, v in stocks["Value"].items()
if "10%" in stocks["Title"][k]}
}
But I'll find writing the actual for loop a bit cleaner.
The result in both cases is:
{'Ticker': {0: 'AKUS', 1: 'HHC'}, 'Value': {0: '+$374,908,350', 1: '+$109,214,243'}}
An additional change in your origin dictionary could be, as you use always the same indices, instead of storing informations in three separate dictionaries, make use of tuples, that stores stocks in the order ticker, title and value, i.e.:
stocks = {0: ('AKUS', '10%', '+$374,908,350'),
1: ('HHC', 'Dir, 10%', '+$109,214,243'),
2: ('IOVA', 'Dir', '+$65,000,000'),
3: ('DDOG', 'Dir', '+$49,999,940')
}
# Filtering stocks where title contains "10%":
ten_percent_stocks = {k: (v[0], v[2]) for k, v in stocks.items() if "10%" in v[1]}
Giving you the result as follows:
{0: ('AKUS', '+$374,908,350'), 1: ('HHC', '+$109,214,243')}
I have a dictionary filled with 3 keys: "Ticker", "Title" and "Value". The ticker key contains 100 stock tickers that corresponds to the value of the purchase and title that are in the same position.
This organization of the data makes no sense. Your data represents 100 stock tickers; a stock ticker is a coherent thing that can be represented with a dict; therefore, the data should be a list of dicts, where each dict has the 'Ticker', 'Title' and 'Value' keys and gives that data for that stock.
So, the first step is to organize the data properly; then any further manipulation is trivial.
We know how to get the necessary data for a stock with a given ID number: we access each key, and then for each of those keys we index again with the ID. So, getting the ticker for stock 0 is just all_stocks['Ticker'][0] (supposing our original dict is named all_stocks), etc. We can easily use that sort of logic to create a dictionary for stock 0. Let's wrap that logic in a function:
def make_stock_dict(all_stocks, stock_id):
return {
'Ticker': all_stocks['Ticker'][stock_id],
'Title': all_stocks['Title'][stock_id],
'Value': all_stocks['Value'][stock_id]
}
Now it's trivial to use a list comprehension to apply the function:
converted_stocks = [make_stock_dict(all_stocks, i) for i in range(100)]
For example, I want to create a 10% dictionary that contains all the tickers and values of stocks where the title contained 10%.
Same idea, now that we have properly organized data: we figure out the code that tells us whether a stock qualifies, then we apply it to each stock.
[s for s in converted_stocks if '10%' in s['Title']]
I have a list of dictionaries like shown below and i would like to extract the partID and the corresponding quantity for a specific orderID using python, but i don't know how to do it.
dataList = [{'orderID': 'D00001', 'customerID': 'C00001', 'partID': 'P00001', 'quantity': 2},
{'orderID': 'D00002', 'customerID': 'C00002', 'partID': 'P00002', 'quantity': 1},
{'orderID': 'D00003', 'customerID': 'C00003', 'partID': 'P00001', 'quantity': 1},
{'orderID': 'D00004', 'customerID': 'C00004', 'partID': 'P00003', 'quantity': 3}]
So for example, when i search my dataList for a specific orderID == 'D00003', i would like to receive both the partID ('P00001'), as well as the corresponding quantity (1) of the specified order. How would you go about this? Any help is much appreciated.
It depends.
You are not going to do that a lot of time, you can just iterate over the list of dictionaries until you find the "correct" one:
search_for_order_id = 'D00001'
for d in dataList:
if d['orderID'] == search_for_order_id:
print(d['partID'], d['quantity'])
break # assuming orderID is unique
Outputs
P00001 2
Since this solution is O(n), if you are going to do this search a lot of times it will add up.
In that case it will be better to transform the data to a dictionary of dictionaries, with orderID being the outer key (again, assuming orderID is unique):
better = {d['orderID']: d for d in dataList}
This is also O(n) but you pay it only once. Any subsequent lookup is an O(1) dictionary lookup:
search_for_order_id = 'D00001'
print(better[search_for_order_id]['partID'], better[search_for_order_id]['quantity'])
Also outputs
P00001 2
I believe you would like to familiarize yourself with the pandas package, which is very useful for data analysis. If these are the kind of problems you're up against, I advise you to take the time and take a tutorial in pandas. It can do a lot, and is very popular.
Your dataList is very similar to a DataFrame structure, so what you're looking for would be as simple as:
import pandas as pd
df = pd.DataFrame(dataList)
df[df['orderID']=='D00003']
You can use this:
results = [[x['orderID'], x['partID'], x['quantity']] for x in dataList]
for i in results:
print(i)
Also,
results = [['Order ID: ' + x['orderID'], 'Part ID: ' + x['partID'],'Quantity:
' + str(x['quantity'])] for x in dataList]
To get the partID you can make use of the filter function.
myData = [{"x": 1, "y": 1}, {"x": 2, "y": 5}]
filtered = filter(lambda item: item["x"] == 1) # Search for an object with x equal to 1
# Get the next item from the filter (the matching item) and get the y property.
print(next(filtered)["y"])
You should be able to apply this to your situation.
I need help retrieving a value from a JSON response object in python. Specifically, how do I access the prices-asks-price value? I'm having trouble:
JSON object:
{'prices': [{'asks': [{'liquidity': 10000000, 'price': '1.16049'}],
'bids': [{'liquidity': 10000000, 'price': '1.15989'}],
'closeoutAsk': '1.16064',
'closeoutBid': '1.15974',
'instrument': 'EUR_USD',
'quoteHomeConversionFactors': {'negativeUnits': '1.00000000',
'positiveUnits': '1.00000000'},
'status': 'non-tradeable',
'time': '2018-08-31T20:59:57.748335979Z',
'tradeable': False,
'type': 'PRICE',
'unitsAvailable': {'default': {'long': '4063619', 'short': '4063619'},
'openOnly': {'long': '4063619', 'short': '4063619'},
'reduceFirst': {'long': '4063619', 'short': '4063619'},
'reduceOnly': {'long': '0', 'short': '0'}}}],
'time': '2018-09-02T18:56:45.022341038Z'}
Code:
data = pd.io.json.json_normalize(response['prices'])
asks = data['asks']
asks[0]
Out: [{'liquidity': 10000000, 'price': '1.16049'}]
I want to get the value 1.16049 - but having trouble after trying different things.
Thanks
asks[0] returns a list so you might do something like
asks[0][0]['price']
or
data = pd.io.json.json_normalize(response['prices'])
price = data['asks'][0][0]['price']
The data that you have contains jsons and lists inside one another. Hence you need to navigate through this accordingly. Lists are accessed through indexes (starting from 0) and jsons are accessed through keys.
price_value=data['prices'][0]['asks'][0]['price']
liquidity_value=data['prices'][0]['asks'][0]['liquidity']
Explaining this logic in this case : I assume that your big json object is stored in a object called data. First accessing prices key in this object. Then I have index 0 because the next key is present inside a list. Then after you go inside the list, you have a key called asks. Now again you have a list here so you need to access it using index 0. Then finally the key liquidity and price is here.
I try to write a World of Warcraft Auctionhouse analyzing tool.
For each auction i have data that looks like this:
{
'timeLeftHash': 4,
'bid': 3345887,
'timestamp': 1415339912,
'auc': 1438188059,
'quantity': 1,
'id': 309774,
'ownerHash': 751,
'buy': 3717652,
'ownerRealmHash': 1,
'item': 35965
}
I'd like to combine all dicts that have the same value of "item" so i can get a minBuy, avgBuy, maxBuy, minQuantity, avgQuantity, maxQantity and the sum of combined auctions for the specific item.
How can i archieve that?
I already tried to write it in a Second list of dicts, but then the min and max is missing...
You could try to make a dictionary where the key is the item ID and the Value is a list of tuples of price and quantity.
If you would like to keep all the information, you could also make a dictionary where the key is the item ID and the value is a list of dictionaries corresponding to that ID and from there extract the info that you want through a generator.
data = [
{'item': 35964, 'buy': 3717650, ...},
{'item': 35965, 'buy': 3717652, ...},
...
]
by_item = {}
for d in data:
by_item.setdefault(d['item'], []).append(d['buy'])
stats = dict((k, {'minBuy': min(v), 'maxBuy': max(v)})
for k, v in by_item.iteritems())
I am working with a set of data that I have converted to a list of dictionaries
For example one item in my list is
{'reportDate': u'R20070501', 'idnum': u'1078099', 'columnLabel': u'2005',
'actionDate': u'C20070627', 'data': u'76,000', 'rowLabel': u'Sales of Bananas'}
Per request
The second item in my list could be:
{'reportDate': u'R20070501', 'idnum': u'1078099', 'columnLabel': u'2006',
'actionDate': u'C20070627', 'data': u'86,000', 'rowLabel': u'Sales of Bananas'}
The third item could be:
{'reportDate': u'R20070501', 'idnum': u'1078100', 'columnLabel': u'Full Year 2005',
'actionDate': u'C20070627', 'data': u'116,000', 'rowLabel': u'Sales of Cherries'}
The fourth item could be:
{'reportDate': u'R20070501', 'idnum': u'1078100', 'columnLabel': u'Full Year 2006',
'actionDate': u'C20070627', 'data': u'76,000', 'rowLabel': u'Sales of Sales of Cherries'}
The reason I need to pickle this is because I need to find out all of the ways the columns were labeled before I consolidate the results and put them into a database. The first and second items will be one row in the results, the third and fourth would be the next line in the results (after someone decides what the uniform column header label should be)
I tested pickle and was able to save and retrieve my data. However, I need to be able to preserve the order in the output. One idea I have is to add another key that would be a counter so I could retrieve my data and then sort by the counter. Is there a better way?
I don't want to put this into a database because it is not permanent.
I marked an answer down below. It is not what I am getting, so I need to figure out if the problem is somewhere else in my code.
So what's wrong with pickle? If you structure your data as a list of dicts, then everything should work as you want it to (if I understand your problem).
>>> import pickle
>>> d1 = {1:'one', 2:'two', 3:'three'}
>>> d2 = {1:'eleven', 2:'twelve', 3:'thirteen'}
>>> d3 = {1:'twenty-one', 2:'twenty-two', 3:'twenty-three'}
>>> data = [d1, d2, d3]
>>> out = open('data.pickle', 'wb')
>>> pickle.dump(data, out)
>>> out.close()
>>> input = open('data.pickle')
>>> data2 = pickle.load(input)
>>> data == data2
True
The Python dict is an unordered container. If you need to preserve the order of the entries, you should consider using a list of 2-tuples.
Another option would be to keep an extra, ordered list of the keys. This way you can benefit from the quick, keyed access offered by the dictionary, while still being able to iterate through its values in an ordered fashion:
data = {'reportDate': u'R20070501', 'idnum': u'1078099',
'columnLabel': u'2005', 'actionDate': u'C20070627',
'data': u'76,000', 'rowLabel': u'Sales of Bananas'}
dataOrder = ['reportDate', 'idnum', 'columnLabel',
'actionDate', 'data', 'rowLabel']
for key in dataOrder:
print key, data[key]
Python does not retain order in dictionaries.
However, there is the OrderedDict class in the collections module.
Another option would be to use a list of tuples:
[('reportDate', u'R20080501'), ('idnum', u'1078099'), ...etc]
You can use the built in dict() if you need to convert this to a dictionary later.