Related
I'm writing a function to find the percentage change using Numpy and function calls. So far what I got is:
def change(a,b):
answer = (np.subtract(a[b+1], a[b])) / a[b+1] * 100
return answer
print(change(a,0))
"a" is the array I have made and b will be the index/numbers I am trying to calculate.
For example:
My Array is
[[1,2,3,5,7]
[1,4,5,6,7]
[5,8,9,10,32]
[3,5,6,13,11]]
How would I calculate the percentage change between 1 to 2 (=0.5) or 1 to 4(=0.75) or 5,7 etc..
Note: I know how mathematically to get the change, I'm not sure how to do this in python/ numpy.
If I understand correctly, that you're trying to find percent change in each row, then you can do:
>>> np.diff(a) / a[:,1:] * 100
Which gives you:
array([[ 50. , 33.33333333, 40. , 28.57142857],
[ 75. , 20. , 16.66666667, 14.28571429],
[ 37.5 , 11.11111111, 10. , 68.75 ],
[ 40. , 16.66666667, 53.84615385, -18.18181818]])
I know you have asked this question with Numpy in mind and got answers above:
import numpy as np
np.diff(a) / a[:,1:]
I attempt to solve this with Pandas. For those who would have the same question but using Pandas instead of Numpy
import pandas as pd
data = [[1,2,3,4,5],
[1,4,5,6,7],
[5,8,9,10,32],
[3,5,6,13,11]]
df = pd.DataFrame(data)
df_change = df.rolling(1,axis=1).sum().pct_change(axis=1)
print(df_change)
I suggest to simply shift the array. The computation basically becomes a one-liner.
import numpy as np
arr = np.array(
[
[1, 2, 3, 5, 7],
[1, 4, 5, 6, 7],
[5, 8, 9, 10, 32],
[3, 5, 6, 13, 11],
]
)
# Percentage change from row to row
pct_chg_row = arr[1:] / arr[:-1] - 1
[[ 0. 1. 0.66666667 0.2 0. ]
[ 4. 1. 0.8 0.66666667 3.57142857]
[-0.4 -0.375 -0.33333333 0.3 -0.65625 ]]
# Percentage change from column to column
pct_chg_col = arr[:, 1::] / arr[:, 0:-1] - 1
[[ 1. 0.5 0.66666667 0.4 ]
[ 3. 0.25 0.2 0.16666667]
[ 0.6 0.125 0.11111111 2.2 ]
[ 0.66666667 0.2 1.16666667 -0.15384615]]
You could easily generalize the task, so that you are not limited to compute the change from one row/column to another, but be able to compute the change for n rows/columns.
n = 2
pct_chg_row_generalized = arr[n:] / arr[:-n] - 1
[[4. 3. 2. 1. 3.57142857]
[2. 0.25 0.2 1.16666667 0.57142857]]
pct_chg_col_generalized = arr[:, n:] / arr[:, :-n] - 1
[[2. 1.5 1.33333333]
[4. 0.5 0.4 ]
[0.8 0.25 2.55555556]
[1. 1.6 0.83333333]]
If the output array must have the same shape as the input array, you need to make sure to insert the appropriate number of np.nan.
out_row = np.full_like(arr, np.nan, dtype=float)
out_row[n:] = arr[n:] / arr[:-n] - 1
[[ nan nan nan nan nan]
[ nan nan nan nan nan]
[4. 3. 2. 1. 3.57142857]
[2. 0.25 0.2 1.16666667 0.57142857]]
out_col = np.full_like(arr, np.nan, dtype=float)
out_col[:, n:] = arr[:, n:] / arr[:, :-n] - 1
[[ nan nan 2. 1.5 1.33333333]
[ nan nan 4. 0.5 0.4 ]
[ nan nan 0.8 0.25 2.55555556]
[ nan nan 1. 1.6 0.83333333]]
Finally, a small function for the general 2D case might look like this:
def np_pct_chg(arr: np.ndarray, n: int = 1, axis: int = 0) -> np.ndarray:
out = np.full_like(arr, np.nan, dtype=float)
if axis == 0:
out[n:] = arr[n:] / arr[:-n] - 1
elif axis == 1:
out[:, n:] = arr[:, n:] / arr[:, :-n] - 1
return out
The accepted answer is close but incorrect if you're trying to take % difference from left to right.
You should get the following percent difference:
1,2,3,5,7 --> 100%, 50%, 66.66%, 40%
check for yourself: https://www.calculatorsoup.com/calculators/algebra/percent-change-calculator.php
Going by what Josmoor98 said, you can use np.diff(a) / a[:,:-1] * 100 to get the percent difference from left to right, which will give you the correct answer.
array([[100. , 50. , 66.66666667, 40. ],
[300. , 25. , 20. , 16.66666667],
[ 60. , 12.5 , 11.11111111, 220. ],
[ 66.66666667, 20. , 116.66666667, -15.38461538]])
import numpy as np
a = np.array([[1,2,3,5,7],
[1,4,5,6,7],
[5,8,9,10,32],
[3,5,6,13,11]])
np.array([(i[:-1]/i[1:]) for i in a])
Combine all your arrays.
Then make a data frame from them.
df = pd.df(data=array you made)
Use the pct_change() function on dataframe. It will calculate the % change for all rows in dataframe.
We have a function f(x,y). We want to calculate the matrix Bij = f(xi,xj) = f(ih,jh) for 1 <= i,j <= n and h=1/(n+1), such as :
If f(x,y)=x+y, then Bij = ih+jh and the matrix becomes (here, n=3) :
I would like to program a function calculating the column vector b that concatenates all the columns of Bij. For example, with my previous example, we would have :
I done, we can change the function and n, here f(x,y)=x+y :
n=3
def f(i,j):
h=1.0/(n+1)
a=((i+1)*h)+((j+1)*h)
return a
B = np.fromfunction(f,(n,n))
print(B)
But I don't know how to do the vector b. And with
np.concatenate((B[:,0],B[:,1],B[:,2],B[:,3])
I get a line vector, and not a column vector. Could you help me ? Sorry for my bad english, and I'm beginner in Python.
The ravel function along with a new axis should do the trick:
import numpy as np
x = np.array([[0.5, 0.75, 1],
[0.75, 1, 1.25],
[1, 1.25, 1.5]])
x.T.ravel()[:, np.newaxis]
# array([[ 0.5 ],
# [ 0.75],
# [ 1. ],
# [ 0.75],
# [ 1. ],
# [ 1.25],
# [ 1. ],
# [ 1.25],
# [ 1.5 ]])
Ravel stitches together all the rows, so we first transpose the matrix (with .T). The result is a row-vector, and we change it to a column vector by adding a new axis.
import numpy as np
# create sample matrix `m`
m = np.matrix([[0.5, 0.75, 1], [0.75, 1, 1.25], [1, 1.25, 1.5]])
# convert matrix `m` to a 'flat' matrix
m_flat = m.flatten()
print(m_flat)
# `m_flat` is still a matrix, in case you need an array:
m_flat_arr = np.squeeze(np.asarray(m_flat))
print(m_flat_arr)
The snippet uses .flatten(), .asarray() and .squeeze() to convert the original matrix m being
matrix([[ 0.5 , 0.75, 1. ],
[ 0.75, 1. , 1.25],
[ 1. , 1.25, 1.5 ]])
into an array m_flat_arr of:
array([ 0.5 , 0.75, 1. , 0.75, 1. , 1.25, 1. , 1.25, 1.5 ])
I'm taking a Computational Neuroscience class on Coursera. So far it's been going great! However, I'm getting a little stuck on one of the quiz problems.
I am not taking this class for a certificate or anything. Solely for fun. I already took the quiz and after awhile, I guessed the answer, so this is not even going to be answering the quiz.
The question is framed as the following:
Suppose that we had a linear recurrent network of 5 input nodes and 5 output nodes. Let us say that our network's weight matrix W is:
W = [0.6 0.1 0.1 0.1 0.1]
[0.1 0.6 0.1 0.1 0.1]
[0.1 0.1 0.6 0.1 0.1]
[0.1 0.1 0.1 0.6 0.1]
[0.1 0.1 0.1 0.1 0.6]
(Essentially, all 0.1, besides 0.6 on the diagonals.)
Suppose that we have a static input vector u:
u = [0.6]
[0.5]
[0.6]
[0.2]
[0.1]
Finally, suppose that we have a recurrent weight matrix M:
M = [-0.25, 0, 0.25, 0.25, 0]
[0, -0.25, 0, 0.25, 0.25]
[0.25, 0, -0.25, 0, 0.25]
[0.25, 0.25, 0, -0.25, 0]
[0, 0.25, 0.25, 0, -0.25]
Which of the following is the steady state output v_ss of the network?
(Hint: See the lecture on recurrent networks, and consider writing some Octave or Matlab code to handle the eigenvectors/values (you may use the "eig" function))'
The notes for the class can be found here. Specifically, the equation for the steady state formula can be found on slides 5 and 6.
I have the following code.
import numpy as np
# Construct W, the network weight matrix
W = np.ones((5,5))
W = W / 10.
np.fill_diagonal(W, 0.6)
# Construct u, the static input vector
u = np.zeros(5)
u[0] = 0.6
u[1] = 0.5
u[2] = 0.6
u[3] = 0.2
u[4] = 0.1
# Connstruct M, the recurrent weight matrix
M = np.zeros((5,5))
np.fill_diagonal(M, -0.25)
for i in range(3):
M[2+i][i] = 0.25
M[i][2+i] = 0.25
for i in range(2):
M[3+i][i] = 0.25
M[i][3+i] = 0.25
# We need to matrix multiply W and u together to get h
# NOTE: cannot use W * u, that's going to do a scalar multiply
# it's element wise otherwise
h = W.dot(u)
print 'This is h'
print h
# Ok then the big deal is:
# h dot e_i
# v_ss = sum_(over all eigens) ------------ e_i
# 1 - lambda_i
eigs = np.linalg.eig(M)
eigenvalues = eigs[0]
eigenvectors = eigs[1]
v_ss = np.zeros(5)
for i in range(5):
v_ss += (np.dot(h,eigenvectors[:, i]))/((1.0-eigenvalues[i])) * eigenvectors[:,i]
print 'This is our steady state v_ss'
print v_ss
The correct answer is:
[0.616, 0.540, 0.609, 0.471, 0.430]
This is what I am getting:
This is our steady state v_ss
[ 0.64362264 0.5606784 0.56007018 0.50057043 0.40172501]
Can anyone spot my bug? Thank you so much! I greatly appreciate it and apologize for the long blog post. Essentially, all you need to look at, is slide 5 and 6 on that top link.
I tryied your solution with my matrices:
W = np.array([[0.6 , 0.1 , 0.1 , 0.1 , 0.1],
[0.1 , 0.6 , 0.1 , 0.1 , 0.1],
[0.1 , 0.1 , 0.6 , 0.1 , 0.1],
[0.1 , 0.1 , 0.1 , 0.6 , 0.1],
[0.1 , 0.1 , 0.1 , 0.1 , 0.6]])
u = np.array([.6, .5, .6, .2, .1])
M = np.array([[-0.75 , 0 , 0.75 , 0.75 , 0],
[0 , -0.75 , 0 , 0.75 , 0.75],
[0.75 , 0 , -0.75 , 0 , 0.75],
[0.75 , 0.75 , 0.0 , -0.75 , 0],
[0 , 0.75 , 0.75 , 0 , -0.75]])
and your code generated the right solution:
This is h
[ 0.5 0.45 0.5 0.3 0.25]
This is our steady state v_ss
[ 1.663354 1.5762684 1.66344153 1.56488258 1.53205348]
Maybe the problem is with the Test on coursera. Have you tryed to contact them on the forum?
I want to calculate Fuzzy Relation Compositions on MATLAB/Python and was wondering if there are any built-in functions in MATLAB or Python for doing this...
here's an example :
R1 = [{ 3, 4, .7}, {3, 5, 0}, { 4, 4, 1}, { 4, 5, .1}]
R2 = [{ 3,3, 0}, { 3, 5, .1}, { 4, 4, 0}, { 4, 5, .6}]
I'm trying to calculate :
R1oR2
I'm looking for 'Max-Min' and 'Max-Product' approaches...
Ok... since no one gave an answer, I ended up writing it by myself... here's the Python Code:
import numpy as np
# Max-Min Composition given by Zadeh
def maxMin(x, y):
z = []
for x1 in x:
for y1 in y.T:
z.append(max(np.minimum(x1, y1)))
return np.array(z).reshape((x.shape[0], y.shape[1]))
# Max-Product Composition given by Rosenfeld
def maxProduct(x, y):
z = []
for x1 in x:
for y1 in y.T:
z.append(max(np.multiply(x1, y1)))
return np.array(z).reshape((x.shape[0], y.shape[1]))
# 3 arrays for the example
r1 = np.array([[1, 0, .7], [.3, .2, 0], [0, .5, 1]])
r2 = np.array([[.6, .6, 0], [0, .6, .1], [0, .1, 0]])
r3 = np.array([[1, 0, .7], [0, 1, 0], [.7, 0, 1]])
print "R1oR2 => Max-Min :\n" + str(maxMin(r1, r2)) + "\n"
print "R1oR2 => Max-Product :\n" + str(maxProduct(r1, r2)) + "\n\n"
print "R1oR3 => Max-Min :\n" + str(maxMin(r1, r3)) + "\n"
print "R1oR3 => Max-Product :\n" + str(maxProduct(r1, r3)) + "\n\n"
print "R1oR2oR3 => Max-Min :\n" + str(maxMin(r1, maxMin(r2, r3))) + "\n"
print "R1oR2oR3 => Max-Product :\n" + str(maxProduct(r1, maxProduct(r2, r3))) + "\n\n"
and here's the answer it gives:
R1oR2 => Max-Min :
[[ 0.6 0.6 0. ]
[ 0.3 0.3 0.1]
[ 0. 0.5 0.1]]
R1oR2 => Max-Product :
[[ 0.6 0.6 0. ]
[ 0.18 0.18 0.02]
[ 0. 0.3 0.05]]
R1oR3 => Max-Min :
[[ 1. 0. 0.7]
[ 0.3 0.2 0.3]
[ 0.7 0.5 1. ]]
R1oR3 => Max-Product :
[[ 1. 0. 0.7 ]
[ 0.3 0.2 0.21]
[ 0.7 0.5 1. ]]
R1oR2oR3 => Max-Min :
[[ 0.6 0.6 0.6]
[ 0.3 0.3 0.3]
[ 0.1 0.5 0.1]]
R1oR2oR3 => Max-Product :
[[ 0.6 0.6 0.42 ]
[ 0.18 0.18 0.126]
[ 0.035 0.3 0.05 ]]
I have a file which, for example, looks like:
1 1 5.5
1 2 6.1
1 3 7.3
2 2 3.4
2 3 9.2
3 3 4.7
This is "half" of a symmetric 3x3 matrix. I would like to create the full symmetric matrix in Python which looks like
[[ 5.5 6.1 7.3]
[ 6.1 3.4 9.2]
[ 7.3 9.2 4.7]]
(of course my actual file is a much bigger 'half' of a NxN matrix so I need a solution other than typing in the values one by one)
I've exhausted all my resources (books and internet) and what I have so far does not really come close. Can anyone please help me with this?
Thank you!
to read the file and load it as a python object, here's a solution:
import numpy
m = numpy.matrix([[0,0,0],[0,0,0],[0,0,0]])
with file('matrix.txt', 'r') as f:
for l in f:
try:
i, j, val = line.split(' ')
i, j, val = int(i), int(j), float(val)
m[i-1,j-1] = val
except:
print("couldn't load line: {}".format(l))
print m
Here is an alternative way to do this completely inside Numpy. Two important remarks:
you can read directly with the np.loadtxt function
you can assign the upper-half values to the correct indexes in one line: N[idxs[:,0] - 1, idxs[:,1] - 1] = vals
Here is the code:
import numpy as np
from StringIO import StringIO
indata = """
1 1 5.5
1 2 6.1
1 3 7.3
2 2 3.4
2 3 9.2
3 3 4.7
"""
infile = StringIO(indata)
A = np.loadtxt(infile)
# A is
# array([[ 1. , 1. , 5.5],
# [ 1. , 2. , 6.1],
# [ 1. , 3. , 7.3],
# [ 2. , 2. , 3.4],
# [ 2. , 3. , 9.2],
# [ 3. , 3. , 4.7]])
idxs = A[:, 0:2].astype(int)
vals = A[:, 2]
## To find out the total size of the triangular matrix, note that there
## are only n * (n + 1) / 2 elements that must be specified (the upper
## half amount for (n^2 - n) / 2, and the diagonal adds n to that).
## Therefore, the length of your data, A.shape[0], must be one solution
## to the quadratic equation: n^2 + 1 - 2 * A.shape[0] = 0
possible_sizes = np.roots([1, 1, -2 * A.shape[0]])
## Let us take only the positive solution to that equation as size of the
## result matrix
size = possible_sizes[possible_sizes > 0]
N = np.zeros([size] * 2)
N[idxs[:,0] - 1, idxs[:,1] - 1] = vals
# N is
# array([[ 5.5, 6.1, 7.3],
# [ 0. , 3.4, 9.2],
# [ 0. , 0. , 4.7]])
## Here we could do a one-liner like
# N[idxs[:,1] - 1, idxs[:,0] - 1] = vals
## But how cool is it to add the transpose and subtract the diagonal? :)
M = N + np.transpose(N) - np.diag(np.diag(N))
# M is
# array([[ 5.5, 6.1, 7.3],
# [ 6.1, 3.4, 9.2],
# [ 7.3, 9.2, 4.7]])
If you know the size of the matrix in advance (and it sounds like you do), then the following would work (in both Python 2 and 3):
N = 3
symmetric = [[None]*N for _ in range(SIZE)] # pre-allocate output matrix
with open('matrix_data.txt', 'r') as file:
for i, j, val in (line.split() for line in file if line):
i, j, val = int(i)-1, int(j)-1, float(val)
symmetric[i][j] = val
if symmetric[j][i] is None:
symmetric[j][i] = val
print(symmetric) # -> [[5.5, 6.1, 7.3], [6.1, 3.4, 9.2], [7.3, 9.2, 4.7]]
If you don't know the sizeNahead of time, you could preprocess the file and determine the maximum index values given.