We Keep Coding

Rearrange numpy array by indices on multiple axes

I was wondering how to go about reordering numpy arrays on different axes and with two shared dimensions. For instance, say I have the following (truncated) array of shape (3, 57, 2):
A = array([[[0.93191024, 0.15612787],
[0.71642773, 0.56644261],
[0.56412285, 0.53022111],
[0.08851486, 0.16253565],
[0.34093212, 0.41074817],
[0.02038229, 0.03693136],
[0.09371394, 0.06565686],
[0.93267282, 0.21915236],
[0.96060752, 0.07309468],
[0.82970486, 0.59032961],
[0.3997659 , 0.51113411],
[0.9933917 , 0.1476475 ],
[0.45059107, 0.98992075],
[0.64420259, 0.99047411],
[0.11300111, 0.15220392],
[0.28461117, 0.63130115],
[0.31121419, 0.62935332],
[0.05057692, 0.22276605],
[0.44223485, 0.47280298],
[0.45543663, 0.53295429],
[0.41775544, 0.90341879],
[0.80630092, 0.42294245],
[0.82215799, 0.78450402],
[0.53719004, 0.56694119],
[0.47927382, 0.55033868],
[0.51007433, 0.27610559],
[0.33946435, 0.97437761],
[0.97263574, 0.07449039],
[0.840479 , 0.84473952],
[0.65787646, 0.91151828],
[0.93803356, 0.35804058],
[0.93322294, 0.38717009],
[0.4193876 , 0.54307245],
[0.9016539 , 0.31755714],
[0.66398876, 0.40193374],
[0.65885493, 0.83359592],
[0.32875318, 0.23673417],
[0.87984555, 0.39703699],
[0.69733635, 0.59550783],
[0.81226642, 0.05097729],
[0.12092097, 0.45202225],
[0.17937865, 0.50205434],
[0.32294824, 0.7195445 ],
[0.78753686, 0.69291673],
[0.24804844, 0.30187789],
[0.0446775 , 0.81477633],
[0.505237 , 0.29917803],
[0.67580571, 0.95910518],
[0.08254148, 0.74803897],
[0.43566526, 0.31926868],
[0.4359975 , 0.2017858 ],
[0.19281317, 0.31834615],
[0.89746816, 0.05457466],
[0.3012639 , 0.24771521],
[0.60127988, 0.2070745 ],
[0.01068053, 0.15385391],
[0.36193366, 0.80346892]],
[[0.84691843, 0.64659873],
[0.4138493 , 0.65155583],
[0.1952568 , 0.79107071],
[0.44328237, 0.96547454],
[0.25811086, 0.63944066],
[0.85353904, 0.24630087],
[0.02200961, 0.43702606],
[0.51187364, 0.21604095],
[0.80455184, 0.72895394],
[0.56534909, 0.9832036 ],
[0.56615037, 0.41387091],
[0.15117167, 0.96842998],
[0.45100418, 0.33455142],
[0.76995385, 0.80203815],
[0.09164816, 0.08733553],
[0.0885783 , 0.03341749],
[0.56167238, 0.18389973],
[0.66844602, 0.84802942],
[0.52667165, 0.82921114],
[0.3668573 , 0.79127918],
[0.83358175, 0.2357053 ],
[0.99023907, 0.72416636],
[0.04969955, 0.16445317],
[0.83618043, 0.61106043],
[0.44079159, 0.53960843],
[0.28535309, 0.57542243],
[0.02355444, 0.96444916],
[0.99996202, 0.22699034],
[0.35725371, 0.3284021 ],
[0.24784074, 0.10957504],
[0.68893807, 0.33428212],
[0.19549847, 0.53242997],
[0.37411674, 0.10671197],
[0.38613786, 0.71247659],
[0.58098534, 0.46313774],
[0.94364596, 0.74239641],
[0.94446665, 0.80260609],
[0.15405246, 0.20687026],
[0.06607917, 0.39675446],
[0.49393799, 0.98976068],
[0.90401917, 0.21254029],
[0.39854018, 0.51468104],
[0.57275152, 0.23703318],
[0.28854573, 0.61574602],
[0.33767913, 0.71972114],
[0.29835304, 0.21042103],
[0.53764989, 0.10600488],
[0.93471516, 0.22275683],
[0.46161675, 0.71566021],
[0.60435755, 0.29129393],
[0.93588236, 0.45631744],
[0.68130624, 0.14505727],
[0.26229578, 0.16338432],
[0.61311318, 0.07904725],
[0.42231007, 0.40815435],
[0.18240378, 0.18156717],
[0.32826971, 0.51764318]],
[[0.61310538, 0.70781868],
[0.81069552, 0.40163557],
[0.41225382, 0.55698844],
[0.79904472, 0.57460263],
[0.15534107, 0.14216233],
[0.07721588, 0.00293088],
[0.75088213, 0.15521204],
[0.0505958 , 0.70318609],
[0.40682637, 0.77522382],
[0.65678125, 0.42817843],
[0.98454818, 0.0959132 ],
[0.04875253, 0.33109192],
[0.96330185, 0.01330943],
[0.06014761, 0.90105887],
[0.88935171, 0.82924489],
[0.68668051, 0.92581709],
[0.64966411, 0.33036089],
[0.92391759, 0.13665613],
[0.78596527, 0.01177382],
[0.36857492, 0.97854483],
[0.11493051, 0.63332842],
[0.89560528, 0.24904867],
[0.19925101, 0.78383848],
[0.16796212, 0.86980891],
[0.64345971, 0.87019182],
[0.98074058, 0.66858333],
[0.89239524, 0.68869392],
[0.05101222, 0.06246318],
[0.19412597, 0.58740619],
[0.47544267, 0.66208743],
[0.30888323, 0.03953938],
[0.39305549, 0.24437666],
[0.34944342, 0.94852464],
[0.93839378, 0.30905846],
[0.28667766, 0.18239694],
[0.93015124, 0.23186617],
[0.41991688, 0.6963337 ],
[0.73953573, 0.99268293],
[0.13625265, 0.40126139],
[0.5658131 , 0.81199939],
[0.34594279, 0.90360269],
[0.95083541, 0.97844242],
[0.13066022, 0.16431073],
[0.31765483, 0.41965885],
[0.99760275, 0.77221808],
[0.64802972, 0.71946862],
[0.35160349, 0.64195416],
[0.03926753, 0.27586446],
[0.98373212, 0.55210807],
[0.76304081, 0.62046984],
[0.75019444, 0.4965763 ],
[0.99821063, 0.92457013],
[0.8526248 , 0.0722389 ],
[0.01161104, 0.36656463],
[0.54781289, 0.32044447],
[0.68337198, 0.20499322],
[0.41070141, 0.72194802]]])
And I want to rearrange the elements at dimensions 0 and 1 with an ndarray representing indices of shape (3, 57), i.e. for each row I want to rearrange the 57 entries according to each index list of 57:
B = array([[12, 1, 9, 10, 5, 2, 7, 8, 6, 16, 4, 0, 11, 14, 13, 22,
21, 29, 3, 15, 28, 27, 18, 23, 34, 20, 33, 31, 25, 35, 26, 24,
32, 30, 17, 19, 40, 36, 39, 41, 47, 43, 37, 49, 44, 46, 45, 38,
42, 48, 50, 53, 52, 54, 51, 55, 56],
[46, 42, 55, 52, 31, 24, 43, 48, 44, 49, 35, 51, 33, 53, 16, 41,
10, 4, 7, 13, 22, 8, 40, 15, 5, 12, 1, 9, 25, 3, 30, 26,
6, 0, 34, 14, 32, 2, 28, 27, 39, 11, 17, 21, 20, 19, 23, 18,
45, 36, 50, 38, 37, 29, 47, 54, 56],
[ 4, 1, 14, 12, 6, 8, 0, 11, 13, 7, 2, 3, 9, 5, 16, 10,
27, 28, 20, 22, 32, 25, 15, 37, 17, 34, 24, 18, 19, 43, 39, 26,
23, 29, 30, 47, 45, 21, 31, 33, 35, 38, 44, 46, 40, 50, 36, 48,
49, 51, 41, 42, 52, 53, 54, 55, 56]])
How would I go about transforming array A by indices array B?

If the first dimension is 3 you can use:
np.array([A[i, B[i]] for i in range(len(A))])
#5.13 µs ± 14.7 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)
Otherwise, for longer arrays, you should look for a more performing solution.

Controlling multiple bits flipping to represent chromosomes mutation

I am working on an optimization problem. I have X number of ambulance locations, where X ranges from 1-39.
There are 43 numbers [Ambulance Locations] to choose from (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39) , we choose 3 of them since I have 3 ambulances.
To encode my problem for `programming`, I am working on Chromosome representation:
I can only put my ambulance in three locations among 1-39 locations (Restriction).
Assume that I want to put my Ambulance on the 5th, 19th, and 31 positions.
I am doing something like this for a better presentation:
Chromosome 1 = [5, 19, 31] --> 3 Ambulances
Chromosome 1= [000010000000000000100000000000100000000] - Chromosome Presentation
In the above presentation, I am turning on 5-bit, 19-bit, and 31-bit.
To achieve mutation, what can be a good way to randomly change present positions to other positions keeping the range only between 1-39 locations (Restriction)?

Numpy masks behaving differently when explicitly written and when referenced?

I was trying to understand numpy masks better and decided to try a simple fizzbuzz exercise (since np arrays are homogenous, 9993 is "fizz", 9995 = "buzz", 9998 = "fizzbuzz"). However, I noticed behavior I cannot understand and was hoping that someone could explain.
In the first case, I created my masks like that:
In:
a = np.arange(32)
a[(a % 3 == 0) & (a % 5 == 0)] = 9998
a
Out:
array([9998, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
11, 12, 13, 14, 9998, 16, 17, 18, 19, 20, 21,
22, 23, 24, 25, 26, 27, 28, 29, 9998, 31])
In:
a[a % 3 == 0] = 9993
a[a % 5 == 0] = 9995
a
Out:
array([9998, 1, 2, 9993, 4, 9995, 9993, 7, 8, 9993, 9995,
11, 9993, 13, 14, 9998, 16, 17, 9993, 19, 9995, 9993,
22, 23, 9993, 9995, 26, 9993, 28, 29, 9998, 31])
Notice that 9998 has not been overwritten by the subsequent steps, as expected (it divides by neither 3 nor 5). So far so good. However, then I tried to be clever and name my masks:
In:
a = np.arange(32)
fizz = (a % 3 == 0)
buzz = (a % 5 == 0)
fizzbuzz = fizz & buzz
a[fizzbuzz] = 9998
a
Out:
array([9998, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
11, 12, 13, 14, 9998, 16, 17, 18, 19, 20, 21,
22, 23, 24, 25, 26, 27, 28, 29, 9998, 31])
In:
a[fizz] = 9993
a[buzz] = 9995
a
Out:
array([9995, 1, 2, 9993, 4, 9995, 9993, 7, 8, 9993, 9995,
11, 9993, 13, 14, 9995, 16, 17, 9993, 19, 9995, 9993,
22, 23, 9993, 9995, 26, 9993, 28, 29, 9995, 31])
From what I could grasp, it would appear that at the "fizzbuzz = fizz & buzz" step, I create a mask such that it provides me with a copy of the array when applied over it. This is in contrast to just writing the mask out, which appears to work as intended and modify the array directly (15 & 30 remain 9998 even after the % 3 and % 5 masks are applied).
My question is why does this happen? From my perspective the logic is absolutely the same in both cases. Writing it as "a[fizz & buzz]" instead of "a[fizzbuzz]" did not help.

I think your problem is that, when you generate the array in first step you got a = [0,1,2,3,...,31]. and the comparation that you are doing (first snipet) is with the value in the array and not with de index. so when you do the first replace you got a=[9998,1,2,9998,4,9998,6...] then the next replace you are using the values so the compare 9998%3==0 when you are in index 15 is False and 9998%5==0 is also False
In the seccond case you are using boolean array to acccess a then you're acceding with indexs. In this case doesn't mater the value in that index.
if you want the same behavior in both you can modify when you create fizz and buzz
a = np.arange(32)
fizzbuzz = (a % 3 == 0) & (a % 5 == 0)
a[fizzbuzz] = 9998
print(a)
fizz = (a % 3 == 0)
buzz = (a % 5 == 0)
a[fizz] = 9993
a[buzz] = 9995
print(a)
so the thing is that you are creating fizz and buzz with different arrays y both cases
(sorry for potato english)

How to fill a matrix in Python using iteration over rows and columns

So I have an array of 5 integers v and another of 10 integers v.
I have a 5 by 10 matrix P that I would want to fill so that (P)ij = v[i] + u[j]
I tried:
P = np.empty((len(asset_grid),len(asset_grid)))
for i in range(asset_grid):
for j in range(asset_grid):
P[i,j] = asset_grid[i] + asset_grid[j]
but it gives me an error
TypeError: only integer arrays with one element can be converted to an index
How should I be able to do this in Python. I apologize if my approach is too naive, I am used to Matlab and now slowly learning Python. Any help is appreciated.

Broadcasting is what you want to do. Although for small arrays such as yours, it doesn't make a difference, it makes a significant difference with larger arrays:
>>> arr1 = np.arange(5)
>>> arr2 = np.arange(10,20)
>>> arr1[:,None] + arr2
array([[10, 11, 12, 13, 14, 15, 16, 17, 18, 19],
[11, 12, 13, 14, 15, 16, 17, 18, 19, 20],
[12, 13, 14, 15, 16, 17, 18, 19, 20, 21],
[13, 14, 15, 16, 17, 18, 19, 20, 21, 22],
[14, 15, 16, 17, 18, 19, 20, 21, 22, 23]])
Generally with numpy you want to avoid iteration over rows and columns and use vectorized/broadcasted operations. This is where speed improvements actually come from.
So, elaborating based on your comment:
Say P_ij is ith element of x raised to the 4th power minus jth element of y raised to 2nd power
In general, Python supports most arithmetical operations you would want in a vectorized way, using the usual Python operators:
>>> arr1[:, None]**4 - arr2**2
array([[-100, -121, -144, -169, -196, -225, -256, -289, -324, -361],
[ -99, -120, -143, -168, -195, -224, -255, -288, -323, -360],
[ -84, -105, -128, -153, -180, -209, -240, -273, -308, -345],
[ -19, -40, -63, -88, -115, -144, -175, -208, -243, -280],
[ 156, 135, 112, 87, 60, 31, 0, -33, -68, -105]])

How to count the weeks between specific week numbers?

I Have the below data format.It is a dictionary with names as keys, and a list of week numbers as values.
{'Mali': [17, 16, 23, 18, 17, 16, 17, 18, 16],
'Gooki': [7, 8, 8, 15, 7, 7, 8],
'Piata': [85],
'Goerge': [82],
'Samoo': [106, 55],
'Marria: [101,39]}
I would like to count the number of weeks between the week numbers, and change the value of dictionary with the number of weeks instead of week numbers.This means that for instance in name 'Samoo' I have week 55 and week 106. I would like my code to count these two weeks plus the weeks between them(which is equal to 52 weeks ) and set it as the value for dictionary.
I have the below code, but I am not sure to make this above count.
datedict = defaultdict(set)
with open('d:/info.csv', 'r') as csvfile:
filereader = csv.reader(csvfile, 'excel')
#passing the header
read_header = False
start_date=date(year=2009,month=1,day=1)
#print((seen_date - start_date).days)
tdict = {}
for row in filereader:
if not read_header:
read_header = True
continue
# reading the rest rows
name,firstseen = row[0],row[3]
try:
seen_date = datetime.datetime.strptime(firstseen, '%d/%m/%Y').date()
deltadays = (seen_date-start_date).days
deltaweeks = deltadays/7 + 1
key = name
currentvalue = tdict.get(key, set())
currentvalue.add(deltaweeks)
tdict[key] = currentvalue
except ValueError:
print('Date value error')
pass
pprint.pprint(tdict)
Can anyone help me with this?

>>> d = {'Mali': [17, 16, 23, 18, 17, 16, 17, 18, 16],
'Gooki': [7, 8, 8, 15, 7, 7, 8],
'Piata': [85],
'Samoo': [47, 63, 48, 58, 49, 48],
'Goerge': [82],
'Samoo': [106, 55],
'Marria': [101,39]}
>>> dict((name, max(weeks) - min(weeks) + 1) for name, weeks in d.iteritems())
{'Samoo': 52, 'Gooki': 9, 'Mali': 8, 'Goerge': 1, 'Piata': 1, 'Marria': 63}