Related
I have two lists in Python and I'm trying to map the values of one to the other.
List 1 (coordinates):
['7,16', '71,84', '72,48', '36,52', '75,36', '52,28', '76,44', '11,69', '56,35',
'15,21', '32,74', '88,32', '10,74', '61,34', '51,85', '10,75', '55,96',
'94,12', '34,64', '71,59', '76,75', '25,16', '54,100', '62,1', '60,85',
'16,32', '14,77', '40,78', '2,60', '71,4', '78,91', '100,98', '42,32', '37,49',
'49,34', '3,5', '42,77', '39,60', '38,77', '49,40', '40,53', '57,48', '14,99',
'66,67', '10,9', '97,3', '66,76', '86,68', '10,60', '8,87']
List 2 (index):
[3, 2, 3, 3, 3, 3, 3, 1, 3, 3, 2, 3, 1, 3, 2, 1, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3,
1, 2, 1, 3, 2, 2, 3, 3, 3, 3, 2, 2, 2, 3, 3, 3, 1, 2, 3, 3, 2, 2, 1, 1]
For the output, I need to have something like:
cluster_1: [x, y], [a,b]...
cluster_2: [c, d], [e, f]...
cluster_3: [g, h], [o, j]...
I tried doing this in a dictionary, but I can only get it to put in the last coordinate in the for loop for each value. It also always outputs keys starting from 0, and I'm looking to label them starting from 1.
for i in range(len(patients)):
# other stuff
k = 3
for b in range(k):
if cluster == (k - b):
dct['cluster_%s' % b] = patients[i]
which outputs:
{'cluster_0': '97,3', 'cluster_1': '86,68', 'cluster_2': '8,87'}
I've tried using dct['cluster_%s' % b].append(patients[i]) but I get a key error on cluster_0. Any help would be much appreciated!
You can zip your indices and coordinates, then loop over them element-wise and populate a dictionary based on the index.
clusters = {}
for idx, coord in zip(index, coords):
if idx in clusters:
clusters[idx].append(coord.split(','))
else:
clusters[idx] = [coord.split(',')]
result, where clusters[i] refers the the i-th cluster.
>>> clusters
{
3: [['7', '16'], ['72', '48'], ['36', '52'], ['75', '36'], ['52', '28'], ['76', '44'], ['56', '35'], ['15', '21'], ['88', '32'], ['61', '34'], ['94', '12'], ['71', '59'], ['25', '16'], ['62', '1'], ['16', '32'], ['71', '4'], ['42', '32'], ['37', '49'], ['49', '34'], ['3', '5'], ['49', '40'], ['40', '53'], ['57', '48'], ['10', '9'], ['97', '3']],
2: [['71', '84'], ['32', '74'], ['51', '85'], ['55', '96'], ['34', '64'], ['76', '75'], ['54', '100'], ['60', '85'], ['40', '78'], ['78', '91'], ['100', '98'], ['42', '77'], ['39', '60'], ['38', '77'], ['66', '67'], ['66', '76'], ['86', '68']],
1: [['11', '69'], ['10', '74'], ['10', '75'], ['14', '77'], ['2', '60'], ['14', '99'], ['10', '60'], ['8', '87']]
}
You could use defaultdict along with zip:
from collections import defaultdict
clusters = defaultdict(list)
for id, value in zip(cluster_indices, values):
clusters[id].append(value.split(","))
print(dict(clusters)) # {3: [['7', '16'], ['72', '48'], ...
A defaultdict can be converted to a dict with dict(clusters). However, this may not be necessary since defaultdict basically extends dict.
Note: If you need int values, then you may replace value.split(",") with [int(v) for v in value.split(",")] or list(map(int, value.split(","))). Casting them already at this point will save you an iteration later.
from collections import defaultdict
clusters = defaultdict(list)
for id, value in zip(cluster_indices, values):
clusters[id].append([int(v) for v in value.split(",")])
print(dict(clusters)) # {3: [[7, 16], [72, 48], ...
The group-by behaviour extracted to a function groupby (using a lambda function to allow any kind of transformation) so it can be reused:
from collections import defaultdict
def groupby(indices, values, map_fn):
grouped = defaultdict(list)
for id, value in zip(indices, values):
grouped[id].append(map_fn(id, value))
return dict(grouped)
clusters = groupby(cluster_indices, values, lambda _, value: value.split(","))
print(clusters) # {3: [['7', '16'], ['72', '48'], ...
Here just another way by using itertools.groupby:
from itertools import groupby
from operator import itemgetter
data = sorted(zip(cluster_indices, values), key=itemgetter(0))
grouped = groupby(data, key=itemgetter(0))
clusters = {
cluster: [value[1].split(",") for value in list(values)]
for cluster, values in grouped
}
print(clusters) # {3: [['7', '16'], ['72', '48'], ...
However, I would use the defaultdict approach above or Cory Kramer's answer as it is more simple and easier to read (and therefore preferable)!
I have the following 2d list and dictionary:
List2d = [['1', '55', '32', '667' ],
['43', '76', '55', '100'],
['23', '70', '15', '300']]
dictionary = {'New York':0, "London": 0, "Tokyo": 0, "Toronto": 0 }
How do I replace all the values of the dictionary with sums of the columns in List2d? So dictionary will look like this:
dictionary= {'New York' : 67, 'London': 201, 'Tokyo': 102, 'Toronto': 1067}
#67 comes from adding up first column (1+43+23) in 'List2d'
#201 comes from adding up second column (55+76+70) in 'List2d'
#102 comes from adding up third column (32+55+15) in 'List2d'
#1067 comes from adding up fourth column (667+100+300) in 'List2d'
Since Python 3.7, keys in dict are ordered.
You can use enumerate in order to keep track of the position of the element in the dict while iterating over it. Then, you use the i as an index on each row of the 2d list, convert each value to int and do a sum of the result.
List2d = [['1', '55', '32', '667' ],
['43', '76', '55', '100'],
['23', '70', '15', '300']]
dictionary = {'New York':0, "London": 0, "Tokyo": 0, "Toronto": 0 }
for i, city in enumerate(dictionary.keys()):
dictionary[city] = sum(int(row[i]) for row in List2d)
print(dictionary)
# {'New York': 67, 'London': 201, 'Tokyo': 102, 'Toronto': 1067}
Use pandas
#!pip install pandas
import pandas as pd
pd.DataFrame(List2d, columns=dictionary.keys()).astype(int).sum(axis=0).to_dict()
output:
{'New York': 67, 'London': 201, 'Tokyo': 102, 'Toronto': 1067}
# File 1
Column = ['1', '2', '3']
# File 2
Column = ['-2', '-6', '-7', '-6', '-7']
# File 3
Column=['0', '3', '4', '6', '5']
# File 4
Column = ['-1', '-2', '-3', '-3', '-3']
# Combined files
Column = ['1', '2', '3', '-2', '-6', '-7', '-6', '-7', '0', '3', '4', '6', '5', '-1', '-2', '-3', '-3', '-3']
Guys, I want to select either max or min value from each file in the combined files.
Expected output:
Column = ['3', '-7', '6', '-3']
Any help will be appreciated!
I think you are asking for the abs maximum value for each column. Try the code below
Column1 = [1, 2, 3]
Column2 = [-2, -6, -7, -6, -7]
Column3 = [0, 3, 4, 6, 5]
Column4 = [-1, -2, -3, -3, -3]
print(max(Column1, key=abs))
print(max(Column2, key=abs))
print(max(Column3, key=abs))
print(max(Column4, key=abs))
Within your lists are strings and not integers so you should first convert them into integers:
--> https://www.geeksforgeeks.org/python-converting-all-strings-in-list-to-integers/
It's the same as asking a person "What's the biggest value of apples, oranges, pears".
After that what you simply do is use the max and min function within python.
Column = [1, 2, 3]
print(max(Column))
--> 3
print(min(Column))
--> 1
I hope I could help a little bit. :)
Use this method
column=[sorted(column1)[random.randint(-1,0)]]
Use one of these.
This method first sort the lists
column=[]
column.append(sorted(column1)[random.randint(-1,0)])
column.append(sorted(column2)[random.randint(-1,0)])
column.append(sorted(column3)[random.randint(-1,0)])
column.appemd(sorted(column4)[random.randint(-1,0)])
column.append(sorted(column5)[random.randint(-1,0)])
Thus use random.choice function
column=[]
column.append(random.choice(max(column1),min(column1)))
column.append(random.choice(max(column2),min(column2)))
column.append(random.choice(max(column3),min(column3)))
column.append(random.choice(max(column4),min(column4)))
column.append(random.choice(max(column5),min(column5)))
I am attempting to create a contract bridge match point scoring system. In the list below the 1st, 3rd, etc. numbers are the pair numbers (players) and the 2nd, 4th etc. numbers are the scores achieved by each pair. So pair 2 scored 430, pair 3 scored 420 and so on.
I want to loop through the list and score as follows:
for each pair score that pair 2 beats they receive 2 points, for each they tie 1 point and where they don't beat they get 0 points. The loop then continues and compares each pair's score in the same way. In the example below, pair 2 gets 7 points (beating 3 other pairs and a tie with 1), pair 7 gets 0 points, pair 6 gets 12 points beating every other pair.
My list (generated from an elasticsearch json object) is:
['2', '430', '3', '420', '4', '460', '5', '400', '7', '0', '1', '430', '6', '480']
The python code I have tried (after multiple variations) is:
nsp_mp = 0
ewp_mp = 0
ns_list = []
for row in arr["hits"]["hits"]:
nsp = row["_source"]["nsp"]
nsscore = row["_source"]["nsscore"]
ns_list.append(nsp)
ns_list.append(nsscore)
print(ns_list)
x = ns_list[1]
for i in range(6): #number of competing pairs
if x > ns_list[1::2][i]:
nsp_mp = nsp_mp + 2
elif x == ns_list[1::2][i]:
nsp_mp = nsp_mp
else:
nsp_mp = nsp_mp + 1
print(nsp_mp)
which produces:
['2', '430', '3', '420', '4', '460', '5', '400', '7', '0', '1', '430', '6', '480']
7
which as per calculation above is correct. But when I try to execute a loop it does not return the correct results.
Maybe the approach is wrong. What is the correct way to do this?
The elasticsearch json object is:
arr = {'took': 0, 'timed_out': False, '_shards': {'total': 5, 'successful': 5, 'skipped': 0, 'failed': 0}, 'hits': {'total': 7, 'max_score': 1.0, 'hits': [{'_index': 'match', '_type': 'score', '_id': 'L_L122cBjpp4O0gQG0qd', '_score': 1.0, '_source': {'tournament_id': 1, 'board_number': '1', 'nsp': '2', 'ewp': '9', 'contract': '3NT', 'by': 'S', 'tricks': '10', 'nsscore': '430', 'ewscore': '0', 'timestamp': '2018-12-23T16:45:32.896151'}}, {'_index': 'match', '_type': 'score', '_id': 'MPL122cBjpp4O0gQHEog', '_score': 1.0, '_source': {'tournament_id': 1, 'board_number': '1', 'nsp': '3', 'ewp': '10', 'contract': '4S', 'by': 'N', 'tricks': '10', 'nsscore': '420', 'ewscore': '0', 'timestamp': '2018-12-23T16:45:33.027631'}}, {'_index': 'match', '_type': 'score', '_id': 'MfL122cBjpp4O0gQHEqk', '_score': 1.0, '_source': {'tournament_id': 1, 'board_number': '1', 'nsp': '4', 'ewp': '11', 'contract': '3NT', 'by': 'N', 'tricks': '11', 'nsscore': '460', 'ewscore': '0', 'timestamp': '2018-12-23T16:45:33.158060'}}, {'_index': 'match', '_type': 'score', '_id': 'MvL122cBjpp4O0gQHUoj', '_score': 1.0, '_source': {'tournament_id': 1, 'board_number': '1', 'nsp': '5', 'ewp': '12', 'contract': '3NT', 'by': 'S', 'tricks': '10', 'nsscore': '400', 'ewscore': '0', 'timestamp': '2018-12-23T16:45:33.285460'}}, {'_index': 'match', '_type': 'score', '_id': 'NPL122cBjpp4O0gQHkof', '_score': 1.0, '_source': {'tournament_id': 1, 'board_number': '1', 'nsp': '7', 'ewp': '14', 'contract': '3NT', 'by': 'S', 'tricks': '8', 'nsscore': '0', 'ewscore': '50', 'timestamp': '2018-12-23T16:45:33.538710'}}, {'_index': 'match', '_type': 'score', '_id': 'LvL122cBjpp4O0gQGkqt', '_score': 1.0, '_source': {'tournament_id': 1, 'board_number': '1', 'nsp': '1', 'ewp': '8', 'contract': '3NT', 'by': 'N', 'tricks': '10', 'nsscore': '430', 'ewscore': '0', 'timestamp': '2018-12-23T16:45:32.405998'}}, {'_index': 'match', '_type': 'score', '_id': 'M_L122cBjpp4O0gQHUqg', '_score': 1.0, '_source': {'tournament_id': 1, 'board_number': '1', 'nsp': '6', 'ewp': '13', 'contract': '4S', 'by': 'S', 'tricks': '11', 'nsscore': '480', 'ewscore': '0', 'timestamp': '2018-12-23T16:45:33.411104'}}]}}
List appears to be a poor data structure for this, I think you are making everything worse by flattening your elasticsearch object.
Note there are a few minor mistakes in listings below - to make sure
I'm not solving someone's homework for free. I also realize this is
not the most efficient way of doing so.
Try with dicts:
1) convert elasticsearch json you have to a dict with a better structure:
scores = {}
for row in arr["hits"]["hits"]:
nsp = row["_source"]["nsp"]
nsscore = row["_source"]["nsscore"]
scores[nsp] = nsscore
This will give you something like this:
{'1': '430',
'2': '430',
'3': '420',
'4': '460',
'5': '400',
'6': '480',
'7': '0'}
2) write a function to calculate pair score:
def calculate_score(pair, scores):
score = 0
for p in scores:
if p == pair:
continue
if scores[p] < scores[pair]:
score += 2 # win
elif scores[p] == scores[pair]:
score += 1
return score
This should give you something like this:
In [13]: calculate_score('1', scores)
Out[13]: 7
In [14]: calculate_score('7', scores)
Out[14]: 0
3) loop over all pairs, calculating scores. I'll leave this as exercise.
The main problem with your code is, that the loop is one short, you have 7 entries. Then you should convert the numbers to int, so that the comparison is correct. In your code, you get for ties 0 points.
Instead of having a list, with flattend pairs, you should use tuple pairs.
ns_list = []
for row in arr["hits"]["hits"]:
nsp = int(row["_source"]["nsp"])
nsscore = int(row["_source"]["nsscore"])
ns_list.append((nsp, nsscore))
print(ns_list)
x = ns_list[0][1]
nsp_mp = 0
for nsp, nsscore in ns_list:
if x > nsscore:
nsp_mp += 2
elif x == nsscore:
nsp_mp += 1
print(nsp_mp)
So we can do it like so:
import itertools
d = [(i['_source']['nsp'], i['_source']['nsscore']) for i in arr['hits']['hits']]
d
[('2', '430'),
('3', '420'),
('4', '460'),
('5', '400'),
('7', '0'),
('1', '430'),
('6', '480')]
c = itertools.combinations(d, 2)
counts = {}
for tup in c:
p1, p2 = tup
if not counts.get(p1[0]):
counts[p1[0]] = 0
if int(p1[1]) > int(p2[1]):
counts[p1[0]] += 1
counts
{'2': 3, '3': 2, '4': 3, '5': 1, '7': 0, '1': 0}
I first convert the list of your score to a dictionary object using itertools, then iterating through each key, and for each key, compare the values available in the list
and add accordingly the score you provided and since in this approach you will always add the value 1 because you will always compare it with itself so at end i decrease 1 from the final score there may be a better approach for this
ls = ['2', '430', '3', '420', '4', '460', '5', '400', '7', '0', '1', '430', '6', '480']
d = dict(itertools.zip_longest(*[iter(ls)] * 2, fillvalue=""))
values= d.values()
for item in d.keys():
score=0
for i in values:
if d[item]>i:
score+=2
elif d[item]==i:
score+=1
else:
pass
print(item,":",score-1)
Output:
2 : 7
3 : 4
4 : 10
5 : 2
7 : 0
1 : 7
6 : 12
I have a pandas dataframe df.
One column is a string of numbers (as characters) divided by blank space
I need to convert it to multidim numpy array.
I thought that :
df.A.apply(lambda x: np.array(x.split(" "))).values
would make the trick
Actually it returns an array of array....
array([array(['70', '80', '82', ..., '106', '109', '82'], dtype='<U3'),
array(['151', '150', '147', ..., '193', '183', '184'], dtype='<U3'),
Which does not seem to be what I look what i am looking for whcih should rather look like
array([[[['70', '80', '82', ..., '106', '109', '82'],['151', '150', '147', ..., '193', '183', '184']....
First: what shoudl I do to have my daya in the second format?
Second: I am actually a bit confused about the difference between the 2 data structures. In the end of the day a multidimensional array is an array of arrays. From this perspective it would seem that the 2 are the same structure. But I am sure I am missing somthing
EXAMPLE:
df=pd.DataFrame({"A":[0,1,2,3],"B":["1 2 3 4","5 6 7 8","9 10 11 12","13 14 15 16"]})
A B
0 0 "1 2 3 4"
1 1 "5 6 7 8"
2 2 "9 10 11 12"
3 3 "13 14 15 16"
This command
df.B.apply(lambda x: np.array(x.split(" "))).values
gives:
array([array(['1', '2', '3', '4'], dtype='<U1'),
array(['5', '6', '7', '8'], dtype='<U1'),
array(['9', '10', '11', '12'], dtype='<U2'),
array(['13', '14', '15', '16'], dtype='<U2')], dtype=object)
instead of
array([['1', '2', '3', '4'],
['5', '6', '7', '8'],
['9', '10', '11', '12'],
['13', '14', '15', '16']], dtype='<U2')
Question1: How do I get this last structure?
Question2: what is the difference between the 2? Technically are both array of arrays...
you can do it using str.split on df.A directly, with the parameter expand=True and then use values such as:
df = pd.DataFrame({'A':['70 80 82','151 150 147']})
print (df.A.str.split(' ',expand=True).values)
array([['70', '80', '82'],
['151', '150', '147']], dtype=object)
with your method, if all the strings contain the same amount of numbers, you can still use np.stack to get the same result:
print (np.stack(df.A.apply(lambda x: np.array(x.split(" "))).values))
EDIT: for the difference, not sure I can explain it good enough but I try. let's define
arr1 = df.A.str.split(' ',expand=True).values
arr2 = df.A.apply(lambda x: np.array(x.split(" "))).values
First you can notice that the shape is not the same:
print(arr1.shape)
(2, 3)
print(arr2.shape)
(2,)
so I would say one difference is that arr2 is a 1D array of elements that happens to be also 1D array. When you construct arr2 with values, it constructs a 1D array from the serie df.A.apply(lambda x: np.array(x.split(" "))) without looking at the type in this serie. For arr1, the difference is that df.A.str.split(' ',expand=True) is not a serie but a dataframe, so using values will construct an 2D array with a shape being (number of rows,nb of columns). In both case you use values, but actually having an array in a cell of a serie (as created in your method) will not create a 2D array.
Then, if you want to access any element (such as the first row second element) you can do it by arr1[0,1] while arr2[0,1] will throw an error because this structure is not a 2D array, but arr2[0][1] gives the good answer because you access the second element [1] of the first 1D array [0] in arr2.
I hope it gives some explanation.