This question already has answers here:
NumPy selecting specific column index per row by using a list of indexes
(7 answers)
Closed 4 days ago.
I am having a difficulty vectorizing the following for loops in Python.
out = np.zeros((N, d))
dir_int = []
for i in range(N):
dir_int.append(np.random.randint(low=0, high = d))
out[i,dir_int[i]] = 1
#where:
# direct_int has shape (N, )
# u has shape (N, d)
# x has the same shape as u
# A has shape (2d, d) = [I,-I]^T, I the dxd identity
# b has shape (2d, )
bmAx = b - np.concatenate((x,-x), axis=1) #This is b-Ax has shape N x 2d
upper = np.copy(x)
lower = np.copy(x)
temp = np.zeros(2)
for i in range(len(dir_int)):
temp[0] = bmAx[i, dir_int[i]]
temp[1] = -bmAx[i, d + dir_int[i]]
upper[i, dir_int[i]] += np.amax(temp)
lower[i, dir_int[i]] += np.amin(temp)
For the first loop, dir_int can be created as dir_int = np.random.randint(low=0, high = d, size = N). Then for each "row" of out one of its columns should be 1; this column is dir_int[row]. Not sure how to do that in one line.
The second loop is even harder than the first. Any help is much appreaciated.
The first loop comes out as
out = np.zeros((N, d))
dir_int = np.random.randint(0, d, N)
out[np.arange(N), dir_int] = 1
and it it's a bit harder to help with the second one, since b and x are undefined and I'm not sure I'm visualizing the desired output. But you should be able to use dir_int to index into bMax to update an entire N-length column at a time.
Related
I have a math function whose output is defined by two variables, x and y.
The function is e^(x^3 + y^2).
I want to calculate every possible integer combination between 1 and some defined integer for x and y, and place them in an array so that each output is aligned with the cooresponding x value and y value index. So something like:
given:
x = 3
y = 5
output would be an array like this:
f(1,1) f(1,2) f(1,3)
f(2,1) f(2,2) f(2,3)
f(3,1) f(3,2) f(3,3)
f(4,1) f(4,2) f(4,3)
f(5,1) f(5,2) f(5,3)
I feel like this is an easy problem to tackle but I have limited knowledge. The code that follows is the best description.
import math
import numpy as np
equation = math.exp(x**3 + y**2)
#start at 1, not zero
i = 1
j = 1
#i want an array output
output = []
#function
def shape_f (i,j):
shape = []
output.append(shape)
while i < x + 1:
while j < y +1:
return math.exp(i**3 + j**2)
#increase counter
i = i +1
j = j +1
print output
I've gotten a blank array recently but I have also gotten one value (int instead of an array)
I am not sure if you have an indentation error, but it looks like you never do anything with the output of the function shape_f. You should define your equation as a function, rather than expression assignment. Then you can make a function that populates a list of lists as you describes.
import math
def equation(x, y):
return math.exp(x**3 + y**2)
def make_matrix(x_max, y_max, x_min=1, y_min=1):
out = []
for i in range(x_min, x_max+1):
row = []
for j in range(y_min, y_max+1):
row.append(equation(i, j))
out.append(row)
return out
matrix = make_matrix(3, 3)
matrix
# returns:
[[7.38905609893065, 148.4131591025766, 22026.465794806718],
[8103.083927575384, 162754.79141900392, 24154952.7535753],
[1446257064291.475, 29048849665247.426, 4311231547115195.0]]
We can do this very simply with numpy.
First, we use np.arange to generate a range of values from 0 (to simplify indexing) to a maximum value for both x and y. We can perform exponentiation, in a vectorised manner, to get the values of x^3 and y^2.
Next, we can apply np.add on the outer product of x^3 and y^3 to get every possible combination thereof. The final step is taking the natural exponential of the result:
x_max = 3
y_max = 5
x = np.arange(x_max + 1) ** 3
y = np.arange(y_max + 1) ** 2
result = np.e ** np.add.outer(x, y)
print(result[2, 3]) # e^(2 ** 3 + 3 ** 2)
Output:
24154952.753575277
A trivial solution would be to use the broadcasting feature of numpy with the exp function:
x = 3
y = 5
i = np.arange(y).reshape(-1, 1) + 1
j = np.arange(x).reshape(1, -1) + 1
result = np.exp(j**3 + y**2)
The reshape operations make i into a column with y elements and j into a row with x elements. Exponentiation does not change those shapes. Broadcasting happens when you add the two arrays together. The unit dimensions in one array get expanded to the corresponding dimension in the other. The result is a y-by-x matrix.
I have a simple code in MATLAB which I am trying to translate to python, but I am stuck in a simple for loop:
Here is the situation:
Matlab
f0 = constant
fn = (nx1) matrix
b = (nx1) matrix
d and x are constant
mthd = 1 or 2
s = 1:-0.1:0.1;
for i = 1:10
f = fn * s(i)
switch mthd
case 1
v(:,i) = d *(1 + 1./b.*(f0./f)).^x
case 2
v(:,i) = log(f0./f)./b;
v(:,i) = v./(1+v)
end
v(1,:) = min(vp(2,:));
The output in Matlab results v with nx1 matrix
Assuming it is a simple equation with element wise operation in matlab,
I went ahead and wrote a code in python like this:
s = np.linspace(1,0.1,num=10)
for i in range(1,11)
f = fn * s[i]
if mthd ==1:
v = d *(1 + 1/b*(f0/f))^x
elif mthd ==2:
v = log(f0/f)/b;
v = v/(1+v)
Clearly, this is not the right one and I get stuck right from f = fn* s[i]
Any suggestion in this conversion will be of great help.
Thank you
Clearly this is not the right one and I get stuck right from f = fn* s[i]
What error message are you getting here? Make sure your vectors fn and b are numpy arrays and not lists.
for i in range(1,11)
Python uses zero indexing, whereas Matlab uses 1-indexing. Therefore your for loop should use for i in range(10), which iterates from 0 to 9 instead of 1 to 10.
v = d *(1 + 1/b*(f0/f))^x
Assuming fn and b are numpy arrays in your Python implementation, if you really want this to mirror the Matlab code you can still use indexing such as v[:,i]. However you need to initialize v as a numpy array with the correct size first.
v = log(f0/f)/b;
You probably want np.log here.
Hopefully this is helpful, let me know if you still have questions. You may also find this website helpful.
The code block below should be closer to what you want. Here are a few things to look out for:
Phyton arrays are indexed from 0. In base Python you handle powers with ** e.g. 2 ** 2 equals 4
When performing scalar multiplication and divide of arrays, better to use np.multiply and np.divide
Use np.log for logarithm and np.power for exponentiation with numpy matrices.
Use np.add to add a scalar to a numpy array.
import numpy as np
f0 = 5 # constant
fn = np.matrix([[5], [4], [3], [2], [1]]) # 5 x 1 matrix
b = np.matrix([[9], [8], [7], [6], [5]]) # 5 x 1 matrix
# d and x are constant
d = 4
x = 8
# mthd = 1 or 2
mthd = 1
s = np.linspace(1,0.1,num=10)
# python arrays are indexed from 0
for i in range(0,len(s)):
f = fn * s[i]
if mthd == 1:
v = np.power(np.multiply(d, (1 + np.divide(1., np.multiply
(b, np.divide(f0, f) ) ) ) ), x)
elif mthd ==2:
v = np.divide(np.log(np.divide(f0,f)), b);
v = np.divide(v, (np.add(1, v)) )
i want to optimize 2 for loops into single for loop, is there any way as length of array is very large.
A = [1,4,2 6,9,10,80] #length of list is very large
B = []
for x in A:
for y in A:
if x != y:
B.append(abs(x-y))
print(B)
not any better but more pythonic:
B = [abs(x-y) for x in A for y in A if x!=y]
unless you absolutely need duplicates (abs(a-b) == abs(b-a)), you can half your list (and thus computation):
B = [abs(A[i]-A[j]) for i in range(len(A)) for j in range(i+1, len(A))]
finaly you can use the power of numpy to get C++ speedup:
import numpy as np
A = np.array(A)
A.shape = -1,1 # make it a column vector
diff = np.abs(A - A.T) # diff is the matrix of abs differences
# grab upper triangle of order 1 (i.e. less the diagonal)
B = diff[np.triu_indices(len(A), k=1)]
But this will always be O(n^2) no matter what...
I have an ndarray, A,
and I want to multiply this ndarray element wise by another 1D array b where I assume that A.shape[i] = len(b) for some i. I need this generality in my application.
I can do this using np.tile as follows:
A = np.random.rand(2,3,5,9)
b = np.random.rand(5)
i = 2
b_shape = np.ones(len(A.shape), dtype=np.int)
b_shape[i] = len(b)
b_reps = list(A.shape)
b_reps[i] = 1
B = np.tile(b.reshape(b_shape), b_reps)
# Here B.shape = A.shape and
# B[i,j,:,k] = b for all i,j,k
This strikes me as ugly. Is there a better way to do this?
For this particular example, the following code would do the trick:
result = A*b[:, np.newaxis]
For any value of i, try this:
A2, B = np.broadcast_arrays(A, b)
result = A2*B
I have 26 arrays of data, I am working with incredibly large arrays so I would like to speed this process up. This code actually works and achieves what I want it to achieve but it is incredibly slow.
What I want the code to do is:
Look at two arrays and select cells for which a certain conditions are met or not met. If that condition is met then I would like to change the values in those cells by editing and taking values from another array. All the values in the arrays need to be updated to reflect those changes.
My code below may seem a little confusing but here goes. This is my slow code:
active_layer # An array
active_layer_volumes = [] # List of 7 arrays
active_layer_proportions = [] # List of 7 arrays
inactive_layer # Array
inactive_layer_volumes = [] # List of 7 arrays
inactive_layer_proportions = [] # List of 7 arrays
# Calculate the lower and upper limits for the volume of the active layer
al_upper_volume_limit = 5
al_lower_volume_limit = 1
# Count the grainsizes as the model works through them
grain_size_counter = 1
# Set up some empty arrays to hold the new values
new_active_layer_total = np.zeros_like(active_layer)
new_inactive_layer_total = np.zeros_like(inactive_layer)
# Iterate through the 24 arrays in order
for active_layer_proportion, active_layer_volume, inactive_layer_proportion, inactive_layer_volume in izip(active_layer_volumes, active_layer_proportions,inactive_layer_volumes, inactive_layer_proportions):
# Iterate through all of the cells in the active layer checking to see if certain conditions are met
for [i, j], depth in np.ndenumerate(active_layer): # Iterate through the cells
if active_layer[i, j] >= al_upper_volume_limit: # check to see if the volume in that cell is greater than 5m3
inactive_layer_volume[i, j] = (20 * active_layer_proportion[i, j]) + inactive_layer_volume[i, j] # add 20cm proportion of that grainsize to the active layer
active_layer_volume[i, j] = (active_layer[i, j] - 20) * active_layer_proportion[i, j]
elif active_layer[i, j] < al_lower_volume_limit and inactive_layer[i, j] > 0: # check to see if the volume in that cell is greater than 5m3
active_layer_volume[i, j] = (20 * inactive_layer_proportion[i, j]) + active_layer_volume[i, j]
inactive_layer_volume[i, j] = inactive_layer_volume[i, j] - (20 * inactive_layer_proportion[i, j])
# Increment a counter as the loop goes through the arrays
grain_size_counter + 1
# Add the new calculated volumes to a running total array
new_active_layer_total += active_layer_volume
new_inactive_layer_total += inactive_layer_volume
You can replace the inner loop on ndenumerate with the following vectorized expressions:
# Array B contains True/False for the condition and is subsequently
# used as Boolean index.
B = (active_layer >= al_upper_volume_limit)
inactive_layer_volume[B] += 20 * active_layer_proportion[B]
active_layer_volume[B] = (active_layer[B] - 20) * active_layer_proportion[B]
# The "not B" does the "else" part of the elif statement it replaces
B = ~B & (active_layer < al_lower_volume_limit) & (inactive_layer > 0)
active_layer_volume[B] += 20 * inactive_layer_proportion[B]
inactive_layer_volume[B] -= 20 * inactive_layer_proportion[B]