I have a array that looks like this
comp_data = [
{
"product_id": 432263,
"price_zone_id": 1,
"oreilly_price": 0,
"amazon_price": 0,
"az_price": 0,
"rockauto_price": 0,
"napa_price": 0,
"oreilly_index": 0,
"amazon_index": 0,
"az_index": 0,
"rockauto_index": 0,
"napa_index": 0,
"weighted_comp_price": 0,
"weighted_comp_index": 0,
"week": None
}
]
Skipping the fields product_id and price_zone_id, I want to create a list of dictionaries by mapping the names of each key. For Example
You can see amazon_price, amazon_index, ultimately I want a list that looks like this
[
{
'amazon_price': 0,
'amazon_index': 0,
'name': 'Amazon' --> This would be simply adding .title() to the name.
},
{
'az_price': 0,
'az_index': 0,
'name': 'Az' --> This would be simply adding .title() to the name.
},
{
'orielly_price': 0,
'orielly_index': 0,
'name': 'ORielly' --> This would be simply adding .title() to the name.
}
]
My current code looks like this and is generating wrong output
stores_data = {}
for data in comp_data:
dict_keys = data.keys()
for keys in dict_keys:
if keys != 'product_id' and keys != 'price_zone_id':
store_name = keys.split('_')[0]
value_type = keys.split('_')[-1]
stores[store_name][value_type] = {}
Stores are eassentially the string infront of _index or _price. ex amazon_index, store would be amazon
For this key "weighted_comp_price", the store would be Weighted Comp
For each dict
Find the _price keys
Verify you have the corresponding _index key
Save it
stores_data = []
for data in comp_data:
for price_key in (k for k in data if k.endswith("_price")):
name = price_key.rsplit("_", maxsplit=1)[0]
index_key = f'{name}_index'
if index_key in data:
stores_data.append({price_key: data[price_key], index_key: data[index_key], 'name': name.title()})
Giving
[{'oreilly_price': 0, 'oreilly_index': 0, 'name': 'Oreilly'},
{'az_price': 0, 'az_index': 0, 'name': 'Az'},
{'napa_price': 0, 'napa_index': 0, 'name': 'Napa'},
{'amazon_price': 0, 'amazon_index': 0, 'name': 'Amazon'},
{'rockauto_price': 0, 'rockauto_index': 0, 'name': 'Rockauto'},
{'weighted_comp_price': 0, 'weighted_comp_index': 0, 'name': 'Weighted_Comp'}]
Your code is creating "price" and "index" instead of the whole name. Which did you want?
comp_data = [
{
"product_id": 432263,
"price_zone_id": 1,
"oreilly_price": 0,
"amazon_price": 0,
"az_price": 0,
"rockauto_price": 0,
"napa_price": 0,
"oreilly_index": 0,
"amazon_index": 0,
"az_index": 0,
"rockauto_index": 0,
"napa_index": 0,
"weighted_comp_price": 0,
"weighted_comp_index": 0,
"week": None
}
]
stores = {}
for data in comp_data:
for key,value in data.items():
if key not in ('product_id','price_zone_id','week'):
words = key.split('_')
store_name ='_'.join(words[:-1])
value_type = words[-1]
if store_name not in stores:
stores[store_name] = {'name': store_name.title()}
stores[store_name][key] = value
from pprint import pprint
pprint(stores)
Output:
{'amazon': {'amazon_index': 0, 'amazon_price': 0, 'name': 'Amazon'},
'az': {'az_index': 0, 'az_price': 0, 'name': 'Az'},
'napa': {'name': 'Napa', 'napa_index': 0, 'napa_price': 0},
'oreilly': {'name': 'Oreilly', 'oreilly_index': 0, 'oreilly_price': 0},
'rockauto': {'name': 'Rockauto', 'rockauto_index': 0, 'rockauto_price': 0},
'weighted_comp': {'name': 'Weighted_Comp',
'weighted_comp_index': 0,
'weighted_comp_price': 0}}
Related
What is wrong with this code? When I try to run it I get OperationFailure: unknown top level operator: $ne full error: {'ok': 0.0, 'errmsg': 'unknown top level operator: $ne', 'code': 2, 'codeName': 'BadValue'}.
Any ideas what this means? Thank you in advance :)
import pandas as pd
def length_vs_references(articles):
res = {"1-5" : 0, "6-10" : 0, "11-15" : 0, "16-20" : 0, "21-25" : 0, "25-30" : 0, ">30" :0}
n = {"1-5" : 0, "6-10" : 0, "11-15" : 0, "16-20" : 0, "21-25" : 0, "25-30" : 0, ">30" :0}
cursor = articles.aggregate([
{'$match': {'$and' : [{'references': {'$exists': False}
}, {'$ne':['$page_end', '']}, {'$ne':['$page_start', '']} ]}},
{'$project': {'len_refernces': {"$size": '$references'},
'pages': {'$subtract': [{"$toInt": 'page_end'},
{"$toInt" : 'page_start'}]}}},
{'$bucket' :{
'$groupBy': '$pages',
'boundaries': [ 0, 6, 11, 16, 21, 26, 31, 1000000],
'default': 'Other',
'key': {
'output': {"average": {"$avg" : '$len_references'}},
}
}
}
])
return cursor
print(length_vs_references(articles))
Reading between the lines I suspect you want:
cursor = articles.aggregate([
{'$match': {'references': {'$exists': False}, 'page_end': {'$ne': ''}, 'page_start': {'$ne': ''}}},
{'$project': {'len_refernces': {"$size": '$references'},
'pages': {'$subtract': [{"$toInt": '$page_end'},
{"$toInt": '$page_start'}]}}},
{'$bucket': {
'groupBy': '$pages',
'boundaries': [0, 6, 11, 16, 21, 26, 31, 1000000],
'default': 'Other'
}
}
])
You don't need to AND your match filters as they are ANDed by default. I'm guessing you are trying to filter out blank page_end and page_start items. If not, please describe what you are trying to do.
I have a list of dictionaries (API response) and I use the following function to search for certain nations:
def nation_search(self):
result = next((item for item in nations_v2 if (item["nation"]).lower() == (f"{self}").lower()), False)
if result:
return result
else:
return next((item for item in nations_v2 if (item["leader"]).lower() == (f"{self}").lower()), False)
2 examples :
nations_v2 = [{'nation_id': 5270, 'nation': 'Indo-Froschtia', 'leader': 'Saxplayer', 'continent': 2, 'war_policy': 4, 'domestic_policy': 2, 'color': 15, 'alliance_id': 790, 'alliance': 'Rose', 'alliance_position': 3, 'cities': 28, 'offensive_wars': 0, 'defensive_wars': 0, 'score': 4945, 'v_mode': False, 'v_mode_turns': 0, 'beige_turns': 0, 'last_active': '2020-08-10 04:04:48', 'founded': '2014-08-05 00:09:31', 'soldiers': 0, 'tanks': 0, 'aircraft': 2100, 'ships': 0, 'missiles': 0, 'nukes': 0},
{'nation_id': 582, 'nation': 'Nightsilver Woods', 'leader': 'Luna', 'continent': 4, 'war_policy': 4, 'domestic_policy': 2, 'color': 10, 'alliance_id': 615, 'alliance': 'Seven Kingdoms', 'alliance_position': 2, 'cities': 23, 'offensive_wars': 0, 'defensive_wars': 0, 'score': 3971.25, 'v_mode': False, 'v_mode_turns': 0, 'beige_turns': 0, 'last_active': '2020-08-10 00:22:16', 'founded': '2014-08-05 00:09:35', 'soldiers': 0, 'tanks': 0, 'aircraft': 1725, 'ships': 115, 'missiles': 0, 'nukes': 0}]
I want to add a fuzzy-search using fuzzywuzzy to get 5 possible matches in case there's a spelling error in the argument passed into the function but I can't seem to figure it out.
I only want to search in values for nation and leader.
If you need 5 possible matches, use process.
from fuzzywuzzy import process
def nation_search(self):
nations_only = [ v2['nation'].lower() for v2 in nations_v2 ]
leaders_only = [ v2['leader'].lower() for v2 in nations_v2 ]
matched_nations = process.extract((f"{self}").lower(), nations_only, limit=5)
matched_leaders = process.extract((f"{self}").lower(), leaders_only, limit=5)
return matched_nations, matched_leaders
when i try the following line of code
print(self.exchange3.privateGetPosition([{'currentQty'}]))
i get the the following error
File "c:/Users/User2/sample_market_maker/SAMPLE/botv2.py", line 130, in place_orders
print(self.exchange3.privateGetPosition([{"currentQty"}]))
File "C:\Users\User2\.virtualenvs\sample_market_maker-9Hdi4SL0\lib\site-packages\ccxt\base\exchange.py", line 364, in request
return self.fetch2(path, api, method, params, headers, body)
File "C:\Users\User2\.virtualenvs\sample_market_maker-9Hdi4SL0\lib\site-packages\ccxt\base\exchange.py", line 360, in fetch2
request = self.sign(path, api, method, params, headers, body)
File "C:\Users\User2\.virtualenvs\sample_market_maker-9Hdi4SL0\lib\site-packages\ccxt\bitmex.py", line 577, in sign
query += '?' + self.urlencode(params)
TypeError: can only concatenate str (not "list") to str
when I print to see the available nested dict, i get the following
print(self.exchange3.privateGetPosition())
[{'account': 108879, 'symbol': 'XBTUSD', 'currency': 'XBt', 'underlying':
'XBT', 'quoteCurrency': 'USD', 'commission': 0.00075, 'initMarginReq': 0.04,
'maintMarginReq': 0.005, 'riskLimit': 20000000000, 'leverage': 25,
'crossMargin': False, 'deleveragePercentile': None, 'rebalancedPnl': 0,
'prevRealisedPnl': -18679, 'prevUnrealisedPnl': 0, 'prevClosePrice':
4348.84, 'openingTimestamp': '2018-11-29T20:00:00.000Z', 'openingQty': 0,
'openingCost': 0, 'openingComm': 0, 'openOrderBuyQty': 0,
'openOrderBuyCost': 0, 'openOrderBuyPremium': 0, 'openOrderSellQty': 0,
'openOrderSellCost': 0, 'openOrderSellPremium': 0, 'execBuyQty':
0, 'execBuyCost': 0, 'execSellQty': 0, 'execSellCost': 0, 'execQty': 0,
'execCost': 0, 'execComm': 0, 'currentTimestamp': '2018-11-
29T20:00:00.369Z', 'currentQty': 0, 'currentCost': 0, 'currentComm': 0,
'realisedCost': 0, 'unrealisedCost': 0, 'grossOpenCost': 0,
'grossOpenPremium': 0, 'grossExecCost': 0, 'isOpen': False, 'markPrice':
None, 'markValue': 0, 'riskValue': 0, 'homeNotional': 0, 'foreignNotional':
0, 'posState': '', 'posCost': 0, 'posCost2': 0, 'posCross': 0, 'posInit': 0,
'posComm': 0, 'posLoss': 0, 'posMargin': 0, 'posMaint': 0, 'posAllowance':
0, 'taxableMargin': 0, 'initMargin': 0, 'maintMargin': 0, 'sessionMargin':
0, 'targetExcessMargin': 0, 'varMargin': 0, 'realisedGrossPnl': 0,
'realisedTax': 0, 'realisedPnl': 0, 'unrealisedGrossPnl': 0, 'longBankrupt':
0, 'shortBankrupt': 0, 'taxBase': 0, 'indicativeTaxRate': 0,
'indicativeTax': 0, 'unrealisedTax': 0, 'unrealisedPnl': 0,
'unrealisedPnlPcnt': 0, 'unrealisedRoePcnt': 0, 'simpleQty': None,
'simpleCost': None, 'simpleValue': None, 'simplePnl': None, 'simplePnlPcnt':
None, 'avgCostPrice': None, 'avgEntryPrice': None,
'breakEvenPrice': None, 'marginCallPrice': None, 'liquidationPrice': None,
'bankruptPrice': None, 'timestamp': '2018-11-29T20:00:00.369Z', 'lastPrice':
None, 'lastValue': 0}]
Anybody know what I am doing wrong, I am trying the get the currentQty value?
This is python37
The proper way to do it:
positions = self.exchange3.privateGetPosition()
print(positions[0]['currentQty'])
More about it:
https://github.com/ccxt/ccxt/wiki/Manual#implicit-api-methods
https://www.bitmex.com/api/explorer/#!/Position/Position_get
This question already has answers here:
Writing a dict to txt file and reading it back?
(6 answers)
Closed 5 years ago.
everyone.
I was tinkering with ways to store dictionaries and retrieve them later, loading them into and replacing the existing dictionary values.
I was intending to use this as a way to store game data in a simple text-based game.
# Player dictionaries
p1 = {'name' : "placeholder", 'hp': 0, 'str': 0, 'stm': 0, 'mana': 0,
'atk': 0, 'def': 0, 'cyra' : 0, 'monero' : 0, 'bits': 0, 'dollars': 0,
'nordas': 0,}
p2 = {'name' : "placeholder", 'hp': 0, 'str': 0, 'stm': 0, 'mana': 0,
'atk': 0, 'def': 0, 'cyra' : 0, 'monero' : 0, 'bits': 0, 'dollars': 0,
'nordas': 0,}
p3 = {'name' : "placeholder", 'hp': 0, 'str': 0, 'stm': 0, 'mana': 0,
'atk': 0, 'def': 0, 'cyra' : 0, 'monero' : 0, 'bits': 0, 'dollars': 0,
'nordas': 0,}
p4 = {'name' : "placeholder", 'hp': 0, 'str': 0, 'stm': 0, 'mana': 0,
'atk': 0, 'def': 0, 'cyra' : 0, 'monero' : 0, 'bits': 0, 'dollars': 0,
'nordas': 0,}
p5 = {'name' : "placeholder", 'hp': 0, 'str': 0, 'stm': 0, 'mana': 0,
'atk': 0, 'def': 0, 'cyra' : 0, 'monero' : 0, 'bits': 0, 'dollars': 0,
'nordas': 0,}
p6 = {'name' : "placeholder", 'hp': 0, 'str': 0, 'stm': 0, 'mana': 0,
'atk': 0, 'def': 0, 'cyra' : 0, 'monero' : 0, 'bits': 0, 'dollars': 0,
'nordas': 0,}
#save function
import pickle
def save() :
f = open("p1.txt","w")
f.write( str(p1) )
f.close()
f = open("p2.txt","w")
f.write( str(p2) )
f.close()
f = open("p3.txt","w")
f.write( str(p3) )
f.close()
f = open("p4.txt","w")
f.write( str(p4) )
f.close()
f = open("p5.txt","w")
f.write( str(p5) )
f.close()
f = open("p6.txt","w")
f.write( str(p6) )
f.close()
print ("The game was saved.")
To my delight, I discovered that this does work, and it creates six files names "p1.txt", "p2.txt", and so forth in the folder that the program is saved. It looks exactly like the original dictionary:
{'name': 'placeholder', 'hp': 0, 'str': 0, 'stm': 0, 'mana': 0, 'atk': 0, 'def': 0, 'cyra': 0, 'monero': 0, 'bits': 0, 'dollars': 0, 'nordas': 0}
However, upon attempting to try and create an import_save() function, I ran into a problem. I couldn't use unpickle(), because that's not how I exported the dictionary in the first place. Any tips on how this could be solved?
I was also looking to find a way that the imported dictionary could over-write the existing dictionary? I couldn't find anything about it... I'm not against re-writing the code if that is needed.
Try this instead:
import json
# save to file
with open("p1.txt", "w") as f:
json.dump(p1, f)
# read from file
with open("p1.txt", "r") as f:
p1 = json.load(f)
Although as Patrick Haugh mentions, and the duplicate answer also indicates, json will convert all dictionary keys to unicode strings, so, for example, if you had integers like {1: 'a'} decoded would be {u'1': u'a'} which is not the same as the original dictionary - note the key is now a string, not an integer. Using the pickle module would avoid this particular issue, but it is not human readable, nor is it readable by other computer languages - pickle is a Python object serialization.
As #Mark suggested, json is the best way:
import json
p_list = [p1, p2, p3, p4, p5, p6]
# Write
for i in range(1, 7):
with open('p'+str(i)+'.json', 'w') as f:
json.dump(p_list[i-1], f)
#Read
p_read_list = list()
for i in range(1, 7):
with open('p'+str(i)+'.json', 'r') as f:
p_read_list.append(json.load(f))
print(p_read_list[0])
So i have this dictionary "runs":
[{
'id': 12,
'suite_id': 2,
'name': 'name',
'description': "desc.",
'nice_id': 3,
'joku_id': None,
'onko': False,
'eikai': False,
'tehty': None,
'config': None,
'config_ids': [],
'passed_count': 1,
'blocked_count': 2,
'untested_count': 3,
'retest_count': 4,
'failed_count': 5,
'custom_status1_count': 0,
'custom_status2_count': 0,
'custom_status3_count': 0,
'custom_status4_count': 0,
'custom_status5_count': 0,
'custom_status6_count': 0,
'custom_status7_count': 0,
'projekti_id': 1,
'plan_id': None,
'created_on': 12343214,
'created_by': 11,
'url': 'google.com'
}, {
'id': 16,
'suite_id': 2,
'name': 'namae)',
'description': "desc1",
'nice_id': 5,
'joku_id': None,
'onko': False,
'eikai': False,
'tehty': None,
'config': None,
'config_ids': [],
'passed_count': 100,
'blocked_count': 1,
'untested_count': 3,
'retest_count': 2,
'failed_count': 5,
'custom_status1_count': 0,
'custom_status2_count': 0,
'custom_status3_count': 0,
'custom_status4_count': 0,
'custom_status5_count': 0,
'custom_status6_count': 0,
'custom_status7_count': 0,
'prokti_id': 7,
'plan_id': None,
'created_on': 4321341644,
'created_by': 11,
'url': 'google.com/2' }
there is "id" for about 50 times. that is just a part of it.
i need to find all "id":s (Not joku_ids, ncie_ids etc. Only "id") and make a string/dict of them
and same for name, and description
i have tried:
j = json.load(run)
ids = (j["id"])
j = json.load(run)
names = (j["name"])
j = json.load(run)
descriptions = (j["description"])
but it returns:
AttributeError: 'list' object has no attribute 'read'
I also need to send a request with specific id and in this case the specific id is marked by o. so id[o]
the request code is below:
test = client.send_get('get_tests/1/ ')
so i need to have the id[o] instead of the 1.
i have tried
test = client.send_get('get_tests/' + id[o] + '/ ')
but it returns:
TypeError: 'int' object is not subscriptable
May be this can help you.
id = []
for i in runs :
id.append(i.get('id'))
[12, 16]
You are trying to pass a list to a json.load function. Please read the docs. Load() does not accep lists, it accepts
a .read()-supporting file-like object containing a JSON document
If you want your result in list of dictionary then:
result = [{x:y} for i in range(len(data)) for x,y in data[i].items() if x=='id' or x=='name' or x=='description']
output:
[{'name': 'name'}, {'id': 12}, {'description': 'desc.'}, {'name': 'namae)'}, {'id': 16}, {'description': 'desc1'}]
the data is your list of dictionary data.
hope this answer helpful for you.