Related
My problem here is that I need a 2 dimensional array, that holds the value 1 in the center of the matrix and then other numbers continue from that point. For example, when the input is 3, the matrix should look like this:
[[3, 3, 3, 3, 3],
[3, 2, 2, 2, 3],
[3, 2, 1, 2, 3],
[3, 2, 2, 2, 3],
[3, 3, 3, 3, 3]]
This is my code so far:
import pprint
def func(number):
d = [[0] for i in range(number + (number - 1))]
#? Create the matrix with 0 value inside each index and determine the amount of columns
for i in d:
d[d.index(i)] = [number] * (number + (number - 1))
#? Add values to the indexes
#? [value inside] * [length of a row]
centre = len(d) // 2
#? centre is the index value of the list which contains the number 1
pprint.pprint(d)
func(3)
The result is this:
[[3, 3, 3, 3, 3],
[3, 3, 3, 3, 3],
[3, 3, 3, 3, 3],
[3, 3, 3, 3, 3],
[3, 3, 3, 3, 3]]
My approach was to fill the whole table with the number given, then work with the rest of the table without the first and last array, because they won't change and their value will always be the number times amount necessary.
I even know at what index of the 2D array the center is. However, I am stuck here. I'm not sure how to continue from here.
The value at each index is given by the maximum of (distance from the row index to the center, distance from the column index to the center). So, you can do:
n = 3
dim = 2 * n - 1
result = [
[
max(abs(row - dim // 2), abs(col - dim // 2)) + 1
for col in range(dim)
]
for row in range(dim)
]
print(result)
This outputs:
[[3, 3, 3, 3, 3],
[3, 2, 2, 2, 3],
[3, 2, 1, 2, 3],
[3, 2, 2, 2, 3],
[3, 3, 3, 3, 3]]
Fun question!
Here's what I came up with. It's quite shorter than your implementation:
import numpy as np
mdim = (5, 5) # specify your matrix dimensions
value = 3 # specify the outer value
matrix = np.zeros(mdim) # initiate the matrix
for i, v in enumerate(range(value, 0, -1)): # let's fill it!
matrix[i:(mdim[0]-i), i:(mdim[1]-i)] = v
print(matrix)
The output is as desired! It's probably possible to tweak this a little bit more.
Explanations:
It first creates a matrix of zeros with the dimensions of mdim.
Then, it fills the matrix with the values of v=3 from rows 0:5 and columns 0:5.
It successively fills the matrix with the values of v-1=2 from rows 1:4 and columns 1:4, until the centre gets the final value of 1.
Pardon if I couldn't synthesize precisely my problem with my title, however I guess that by explaning it, things will get more clear.
My problem is that I have to perform a calculation utilizing every combination of a set of arrays and then, store the corresponding result in an array for each loop, as well as the arguments utilized for the calculation. Subsequently, I will store the NumPy arrays and the corresponding result of the calculation in a pandas dataframe
I am trying to avoid for loops as much as possible and I am not sure if there are other means to achieve my goal utilizing pandas or python methods which I am still not aware of. The problem that I am trying to solve is naturally more complex and would involve several more arrays as well as more complex data. So, to sum up, my questions is if there are other smart ways to avoid what I have been doing.
The code that I have been working on - written in a very matlab-ish way - has a similar logic to the following(this is just an illustrative simpler example):
max_x = 5
min_x = 1
x_1 = np.linspace(min_x, max_x, 5)
x_2 = np.linspace(min_x, max_x, 5)
x_3 = np.linspace(min_x, max_x, 5)
x_4 = np.linspace(min_x, max_x, 5)
x_result = np.array([])
x1 = np.array([])
x2 = np.array([])
x3 = np.array([])
x4 = np.array([])
for i in range(0,len(x_1)):
for j in range(0,len(x_2)):
for k in range(0,len(x_3)):
for l in range(0, len(x_4)):
x_set = [x_1[i],x_2[j],x_3[k], x_4[l]]
x_calc = calculation_1(arg = x_set)
x1 = np.append(x1, x_1[i])
x2 = np.append(x2, x_2[j])
x3 = np.append(x3, x_3[k])
x4 = np.append(x4, x_4[l])
x_result = np.append(x_result, x_calc)
df_x = pd.DataFrame(np.array([x1, x2, x3, x4, x_result])).T
If I understand it correctly, you want to implement some sort of cartesian product with arrays. We can do this by using np.meshgrid, like:
def cartesian_product(*arrs):
return np.transpose(np.meshgrid(*arrs)).reshape(-1, len(arrs))
For example:
>>> x_1 = [1,2,3]
>>> x_2 = [3,4]
>>> x_3 = [5]
>>> cartesian_product(x_1, x_2, x_3)
array([[1, 3, 5],
[1, 4, 5],
[2, 3, 5],
[2, 4, 5],
[3, 3, 5],
[3, 4, 5]])
You can then channel the items of this cross product through calculation_1 , for example with np.apply_along_axis(..):
np.apply_axis(calculation_1, 1, c)
We can then add that result as a new column, for example with sum:
>>> c = cartesian_product(x_1, x_2, x_3)
>>> np.hstack((c, np.apply_axis(sum, 1, c)[:, None]))
array([[ 1, 3, 5, 9],
[ 1, 4, 5, 10],
[ 2, 3, 5, 10],
[ 2, 4, 5, 11],
[ 3, 3, 5, 11],
[ 3, 4, 5, 12]])
You can rewrite n nested loops with recursion for example.
(And eventually you can write any recursion with single loop using queues)
Read this for details
I'm having trouble understanding how to shape data to evaluate an interpolated view of an nD-array, using scipy.interpolate.RegularGridInterpolator
Considering A a (n1,n2,n3)-shaped numpy array, indexed along the following coordinates :
x = np.linspace(0, 10, 5) # n1 = 5
y = np.linspace(-1, 1, 10) # n2 = 10
z = np.linspace(0, 500, 1000) # n3 = 1000
For this example, you can generate A = ex_array with this bit of code from the documentation :
def f(x,y,z):
return 2 * x**3 + 3 * y**2 - z
ex_array = f(*np.meshgrid(x, y, z, indexing='ij', sparse=True))
Let's imagine I want to interpolate the entire array along each axis.
This is done with :
from scipy.interpolate import RegularGridInterpolator
interpolated = RegularGridInterpolator((x,y,z), ex_array)
Now, the part where my brain starts to hurt hard :
In order to evaluate this interpolator object at any given coordinates, you have to __call__ it on said point like so :
evaluated_pts = interpolated((0,1,0)) # evaluate at (x,y,z) = (5,0.5,300)
print(evaluated_pts)
In order to evaluate it on several points, you can iterate like this :
pts = ((5,0.5,_z) for _z in np.linspace(100,200,50))
evaluated_pts = interpolated(pts)
Now, what if I want to use the same logic as above, and evaluate on an entire new grid, such as :
new_x = np.linspace(2, 3, 128)
new_y = np.linspace(-0.1, 0.1, 100)
new_z = np.linspace(350, 400, 256)
As you can see now, it's not as straightforward as interpolated(new_x, new_y, new_z), and I tried to use np.meshgrid but could not figure it out.
Ideally, I'd want to output a new (128, 100, 256) array in this example.
RegularGridInterpolator input values are located on a grid. The grid points are defined using a tuple of "ticks" along each axis, for instance ((x0, x1, ..., xn), (y0, y1, ..., xm), (z0, z1, ..., zk) ) in 3D. The values are given as an nd-array of shape (n, m, k) in this case.
To evaluate the interpolated function, the assumption that the points are on a grid is no more required. Then, the asked points are defined as a list of points (actually an array of coordinates): ((x1, y1, z1), (x2, y2, z2), ... (xP, yP, zP)) i.e. a nd-array of shape (Number of points, Number of dimension).
To evaluate the interpolation on a new grid, it must be constructed using meshgrid.
reshape and transpose are used to transform arrays from one shape to another (see this question).
For example:
x = [0, 1, 2]
y = [3, 4]
z = [5, 6, 7, 8]
xyz_grid = np.meshgrid(x, y, z, indexing='ij')
xyz_list = np.reshape(xyz_grid, (3, -1), order='C').T
xyz_list
̀xyz_list could be used to call the interpolation function and it looks like that:
array([[0, 3, 5],
[0, 3, 6],
[0, 3, 7],
[0, 3, 8],
[0, 4, 5],
[0, 4, 6],
[0, 4, 7],
[0, 4, 8],
[1, 3, 5],
[1, 3, 6],
[1, 3, 7],
[1, 3, 8],
[1, 4, 5],
[1, 4, 6],
[1, 4, 7],
[1, 4, 8],
[2, 3, 5],
[2, 3, 6],
[2, 3, 7],
[2, 3, 8],
[2, 4, 5],
[2, 4, 6],
[2, 4, 7],
[2, 4, 8]])
I am new to python and I feel like I am using the absolutely wrong strategy for programming in python. Here is an example:
I have a list like this:
selected_parents =
[array([[4, 6, 3, 1, 0, 7, 5, 2]]), array([[0, 2, 7, 3, 5, 4, 1, 6]])]
Now I want to apply crossover to the elements of the list (please see the P.S. for what I mean by crossover and how it is done, but again, my question is how I should avoid so many indexing that I use while programming in python):
crossed_p1 = np.zeros((len(selected_parents[0][0]))).astype(int)
crossed_p2 = np.zeros((len(selected_parents[0][0]))).astype(int)
co_point = rd.sample(range(len(selected_parents[0][0])),1)
if co_point[0] >= len(selected_parents[0][0])/2:
crossed_p1[0:co_point[0]] = selected_parents[0][0][0:co_point[0]]
indeces = []
for i in range(co_point[0],len(selected_parents[0][0])):
a = np.where(selected_parents[1][0] == selected_parents[0][0][i])
indeces.append(a)
indeces = sorted(indeces)
for i in range(len(indeces)):
crossed_p1[i + co_point[0]] = selected_parents[1][0][indeces[i][0][0]]
crossed_p2[0:co_point[0]] = selected_parents[1][0][0:co_point[0]]
indeces = []
for i in range(co_point[0],len(selected_parents[0][0])):
a = np.where(selected_parents[0][0] == selected_parents[1][0][i])
indeces.append(a)
indeces = sorted(indeces)
for i in range(len(indeces)):
crossed_p2[i + co_point[0]] = selected_parents[0][0][indeces[i][0][0]]
else:
crossed_p1[co_point[0]:] = selected_parents[0][0][co_point[0]:]
indeces = []
for i in range(co_point[0]):
a = np.where(selected_parents[1][0] == selected_parents[0][0][i])
indeces.append(a)
indeces = sorted(indeces)
for i in range(len(indeces)):
crossed_p1[i] = selected_parents[1][0][indeces[i][0][0]]
crossed_p2[co_point[0]:] = selected_parents[1][0][co_point[0]:]
indeces = []
for i in range(co_point[0]):
a = np.where(selected_parents[0][0] == selected_parents[1][0][i])
indeces.append(a)
indeces = sorted(indeces)
for i in range(len(indeces)):
crossed_p2[i] = selected_parents[0][0][indeces[i][0][0]]
The code works like a charm, but I hate the way I am writing it! Like I keep questioning myself, do I really have to write something like selected_parents[0][0][indeces[i][0][0]]?! Like is there a better way of doing what I am doing?!
P.S. This is an example of genetic algorithm and I have the two arrays in selected_parents as the first generation parents. Now I want to apply crossover, which means: A cutting point (i.e. co_point in the code) which is a random integer between 1 and the parents length (herein 8), is selected randomly; the first descendant (i.e. crossed_p1) inherits a longer substring from the first parent and replaces the numbers of shorter substring in the order of numbers appeared in the second parent. And similar procedure is repeated for the second descendant (i.e. crossed_p2). For example, based on the current selected_parents list, and for a co_point = 5, the first descendant (i.e. crossed_p1) inherits the substring of 46310 from the first parent, and the remaining substring of 752 is replaced by 275 which is the order of numbers appeared in the second parent. Hence, the first descendant (i.e. crossed_p1) is 46310275 and the second descendant (i.e. crossed_p2) will be 02735461.
Most of the index is to elements of the selected_parents list, which are 2d arrays:
selected_parents[0][0][0:co_point[0]]
arrays can be indexed with one set of []:
selected_parents[0][0, 0:co_point[0]]
notationally it might be convenient to 'name' the 2 elements of the list (unpacking):
p1, p2 = selected_parents
p1[0, 0:co_point[0]]
Generally it is better to use shape than len on an array. Replace
len(selected_parents[0][0])
with
p1.shape[1]
p1.shape is (1,8)
Looks like p1, p2 have the same shape. In which case
np.stack(selected_parents)
should produce a (2,1,8) array, which could be reshaped to (2,8). Or
np.concatenate(selected_parents, axis=0)
producing a (2,8) array.
It seems the simplest way to make a crossover if your gens are 1D lists:
>>> selected_parents = [[4, 6, 3, 1, 0, 7, 5, 2], [0, 2, 7, 3, 5, 4, 1, 6]]
Let's create two parants, and selecet point of crossover:
>>> p1, p2 = selected_parents
>>> cx = random.randint(len(p1))
>>> p1
[4, 6, 3, 1, 0, 7, 5, 2]
>>> p2
[0, 2, 7, 3, 5, 4, 1, 6]
>>> cx
4
First and second chlidrens are conjuctions of two tancated lists
>>> ch1=p1[:cx]+p2[cx:]
>>> ch1
[4, 6, 3, 1, 5, 4, 1, 6]
>>> ch2=p2[:cx]+p1[cx:]
>>> ch2
[0, 2, 7, 3, 0, 7, 5, 2]
>>>
If you need numpy, it is not a problem. The same idea below:
>>> selected_parents = [array([[4, 6, 3, 1, 0, 7, 5, 2]]), array([[0, 2, 7, 3, 5, 4, 1, 6]])]
>>> p1, p2 = selected_parents
>>> p1
array([[4, 6, 3, 1, 0, 7, 5, 2]])
>>> p2
array([[0, 2, 7, 3, 5, 4, 1, 6]])
>>> cx = random.randint(p1.shape[1])
>>> cx
5
>>> ch1=append(p1[0][:cx],p2[0][cx:])
>>> ch1
array([4, 6, 3, 1, 0, 4, 1, 6])
>>> ch2=append(p2[0][:cx],p1[0][cx:])
>>> ch2
array([0, 2, 7, 3, 5, 7, 5, 2])
Here is a vectorized version of your code. One pleasant side effect of vectorization is that it often does away with most of the indices.
This code assumes that the parent vectors are shuffles of 0, 1, 2, .... If that's not the case some more work is needed:
def invperm(p):
out = np.empty_like(p)
idx = np.ogrid[tuple(map(slice, p.shape))]
idx[-1] = p
out[idx] = np.arange(p.shape[-1])
return out
def f_pp(selected_parents):
sp = np.reshape(selected_parents, (2, -1))
_, N = sp.shape
co = np.random.randint(0, N)
out = sp.copy()
slc = np.s_[:co] if 2*co < N else np.s_[co:]
out[::-1, slc] = out[
np.c_[:2], np.sort(invperm(sp)[np.c_[:2], sp[::-1, slc]], axis=1)]
return out
Let N be an odd positive integer. I want to create a square 2D grid of shape N x N where the center element is N, the surrounding 8-neighborhood has value N-1, the elements around that neighborhood have value N-2 etc., so that finally the "outer shell" of the array has value N//2. I can accomplish this using np.pad in a for loop that iteratively adds "shells" to the array:
def pad_decreasing(N):
assert N % 2 == 1
a = np.array([[N]])
for n in range(N-1, N//2, -1):
a = np.pad(a, 1, mode='constant', constant_values=n)
return a
Example output:
In: pad_decreasing(5)
Out:
array([[3, 3, 3, 3, 3],
[3, 4, 4, 4, 3],
[3, 4, 5, 4, 3],
[3, 4, 4, 4, 3],
[3, 3, 3, 3, 3]])
Question. Can I accomplish this in a "vectorized" form that doesn't resort to for loops. This code is fairly slow for N large.
The distance between the center and any point in the grid can be written as np.maximum(np.abs(row - center), np.abs(col - center)); Then N minus this distance gives the onion you need:
N = 5
# the x and y coordinate of the center point
center = (N - 1) / 2
# the coordinates of the square array
row, col = np.ogrid[:N, :N]
# N - distance gives the result
N - np.maximum(np.abs(row - center), np.abs(col - center))
# array([[3, 3, 3, 3, 3],
# [3, 4, 4, 4, 3],
# [3, 4, 5, 4, 3],
# [3, 4, 4, 4, 3],
# [3, 3, 3, 3, 3]])