Hello so I am trying to run this code where i want X_Matrices dictionary to contain all the X_matrix_i matricies. However, all i am getting that X_Matrices end up being the final X_matrix_i of the final loop. I am not quite sure where i am stuck. Any information would be appreciated! thanks.
n = 5
T = 3
p = 5
X_matrix_i = np.zeros((T,p))
X_Matrices = {}
for i in range(n):
X_Matrices["X" + str(i)] = np.zeros((T,p))
for i in range(n):
for t in range(T):
#initial randomness for loop t
ϵ = np.random.normal(0,1,1)
η = np.random.normal(0,1,1)
Covu = np.zeros((p,p))
#Generating X and e of X
for j in range(len(Covu[0])): #covariance matrix for vector x
for l in range(len(Covu)):
Covu[l,j] = 0.7**(np.abs(l-j))
Zerop = np.zeros(p) # mean vector for vector x
x = np.random.multivariate_normal(Zerop,Covu)
X_matrix_i[t] = x
X_Matrices["X" + str(i)] = X_matrix_i
There are two problems here. First one is the conflicting "i" variables in for loops.
One of them should be changed! I replaced the first for loop i variable with k.
for k in range(n):
for t in range(T):
#initial randomness for loop t
The second problem is that by doing
X_Matrices["X" + str(i)] = X_matrix_i
(last line) you are referencing X_Matrices["X" + str(i)] to X_matrix_i! The issue with mutable objects in Python happens! Instead of doing so, you should use shallow copy to assign values to X_Matrices["X" + str(i)]. So a copy of X_matrix_i will be assigned, not X_matrix_i itself.
I replaced the last line with
X_Matrices["X" + str(k)][:] = X_matrix_i
and the problem solved.
[:] acts as shallow copy here.
I think your problem is caused by the redeclaration of i in the second for loop
for j in range(len(Covu[0])): #covariance matrix for vector x
for i in range(len(Covu)):
Covu[i,j] = 0.7**(np.abs(i-j))
Try to rename the i to something else and it should solve your problem.
Related
Hi I'm doing some image processing. I have some problems when code running
maybe... at least 1 hour code running well. But when the times on and on my code speed is getting slower and memory usage increase.
I try to find some information about these problems. people use list comprehension or map func.
Are these the only solutions?
x_array = np.array([])
y_array = np.array([])
present_x_array = np.array([])
present_y_array = np.array([])
cnt = 0
for x in range(curve.shape[1]):
if np.max(curve[:, x]) > 200 :
for y in range(curve.shape[0]):
if curve[y,x] > 200 :
present_x_array = np.append(present_x_array, x)
present_y_array = np.append(present_y_array, y)
else:
continue
if cnt == 0:
x_array = present_x_array
y_array = present_y_array
cnt = cnt + 1
else :
if abs(np.max(y_array) - np.max(present_y_array)) <= 10 :
x_array =np.append(x_array, present_x_array)
y_array =np.append(y_array, present_y_array)
else :
continue
present_x_array = np.array([])
present_y_array = np.array([])
else:
continue
I try to make comprehension but it stucks handle 'cnt == 0' and 'cnt = cnt + 1'
x_array =np.append(x_array, present_x_array) - np.append creates a new copy whenever you append. Therefore, as the array size increase, the copy operation gets expensive.
I would advise replacing np.array with list to avoid unnecessary copy operations. The list append is fast.
Reference: https://github.com/numpy/numpy/issues/17090
From my understanding, there might be some way to do the trick:
1, Replace those if statements with np.where().
2, Turn your whole loop into a function or class.
Hope this could help.
Have a nice coding day.
# indices to calculate pair-wise products (H, V, D1, D2)
shifts = [[0,1], [1,0], [1,1], [-1,1]]
# calculate pairwise components in each orientation
for itr_shift in range(1, len(shifts) + 1):
OrigArr = structdis
reqshift = shifts[itr_shift-1] # shifting index
for i in range(structdis.shape[0]):
for j in range(structdis.shape[1]):
if(i + reqshift[0] >= 0 and i + reqshift[0] < structdis.shape[0] \
and j + reqshift[1] >= 0 and j + reqshift[1] < structdis.shape[1]):
ShiftArr[i, j] = OrigArr[i + reqshift[0], j + reqshift[1]]
else:
ShiftArr[i, j] = 0
If I try to run the code, I get the following error:
NameError: Name 'ShiftArr' is not defined
How can I solve this error?
By the looks of things you have not defined ShiftArr before you have used it. This is what the error is saying.
It looks like you first use ShiftArr in your nested loop, but nowhere before have you said something like ShiftArr = ...
If you add ShiftArr = [] before your first for loop, this should solve your issue I think. It's a little difficult to understand what you're trying to do as your variable names aren't super informative - this might help you when drying to fix errors in your code.
I'm trying to implemnt Minesweeper game , but i'm stuck , i'm getting the list out of range error and i don't where i did wrong
def Int_jeu(largeur,longueur,diff):
global n, m,T
dif = diff
n=largeur
m=longueur
T = [[0 for i in range(n)] for j in range(m)]
d=((n*m)*dif)//100
print(len(T[0]))
for i in range(d):
a=random.randint(0,len(T)-1)
b=random.randint(0,len(T[0])-1)
while (T[a][b] == "2"):
a=random.randint(0,n-1)
b=random.randint(0,m-1)
T[a][b] = "2"
return(T)
this is the line where i got the error
while (T[a][b] == "2"):
i have declared the matrix T out of the int_jeu function
T = []
can anyone explain me why i get the error please.
m is your first dimension, n is your second, but a is used to index the first dimension (m) while being derived from n (same for b, in reverse).
Change:
a=random.randint(0,n-1)
b=random.randint(0,m-1)
to:
a=random.randint(0,m-1) # Or to simplify, random.randrange(m)
b=random.randint(0,n-1) # Or to simplify, random.randrange(n)
to fix.
I am working in SageMath (Python-based), I am quite new to programming and I have the following question. In my computations I have a quadratic form: x^TAx = b , where the matrix A is defined already as a symmetric matrix, and x is defined as
import itertools
X = itertools.product([0,1], repeat = n)
for x in X:
x = vector(x)
print x
as all combination of [0,1] repeated n times. I got a set of values for b the following way:
import itertools
X = itertools.product([0,1], repeat = n)
results = []
for x in X:
x = vector(x)
x = x.row()
v = x.transpose()
b = x * A * v
results.append(b[0, 0])
And then I defined:
U = set(results)
U1 = sorted(U)
A = []
for i in U1:
U2 = round(i, 2)
A.append(U2)
So I have a sorted set to get a few minimal values of my results. I need to extract minimal values from the set and identify what particular x is corresponding to each value of b. I heard that I can use dictionary method and define preimages in there, but I am really struggling to define my dictionary as {key: value}. Could someone help me please, solve the problem or give me the idea of what direction should I think in? Thank you.
This portion I was able to vectorize and get rid of a nested loop.
def EMalgofast(obsdata, beta, pjt):
n = np.shape(obsdata)[0]
g = np.shape(pjt)[0]
zijtpo = np.zeros(shape=(n,g))
for j in range(g):
zijtpo[:,j] = pjt[j]*stats.expon.pdf(obsdata,scale=beta[j])
zijdenom = np.sum(zijtpo, axis=1)
zijtpo = zijtpo/np.reshape(zijdenom, (n,1))
pjtpo = np.mean(zijtpo, axis=0)
I wasn't able to vectorize the portion below. I need to figure that out
betajtpo_1 = []
for j in range(g):
num = 0
denom = 0
for i in range(n):
num = num + zijtpo[i][j]*obsdata[i]
denom = denom + zijtpo[i][j]
betajtpo_1.append(num/denom)
betajtpo = np.asarray(betajtpo_1)
return(pjtpo,betajtpo)
I'm guessing Python is not your first programming language based on what I see. The reason I'm saying this is that in python, normally we don't have to deal with manipulating indexes. You act directly on the value or the key returned. Make sure not to take this as an offense, I do the same coming from C++ myself. It's a hard to remove habits ;).
If you're interested in performance, there is a good presentation by Raymond Hettinger on what to do in Python to be optimised and beautiful :
https://www.youtube.com/watch?v=OSGv2VnC0go
As for the code you need help with, is this helping for you? It's unfortunatly untested as I need to leave...
ref:
Iterating over a numpy array
http://docs.scipy.org/doc/numpy/reference/generated/numpy.true_divide.html
def EMalgofast(obsdata, beta, pjt):
n = np.shape(obsdata)[0]
g = np.shape(pjt)[0]
zijtpo = np.zeros(shape=(n,g))
for j in range(g):
zijtpo[:,j] = pjt[j]*stats.expon.pdf(obsdata,scale=beta[j])
zijdenom = np.sum(zijtpo, axis=1)
zijtpo = zijtpo/np.reshape(zijdenom, (n,1))
pjtpo = np.mean(zijtpo, axis=0)
betajtpo_1 = []
#manipulating an array of numerator and denominator instead of creating objects each iteration
num=np.zeros(shape=(g,1))
denom=np.zeros(shape=(g,1))
#generating the num and denom real value for the end result
for (x,y), value in numpy.ndenumerate(zijtpo):
num[x],denom[x] = num[x] + value *obsdata[y],denom[x] + value
#dividing all at once after instead of inside the loop
betajtpo_1= np.true_divide(num/denom)
betajtpo = np.asarray(betajtpo_1)
return(pjtpo,betajtpo)
Please leave me some feedback !
Regards,
Eric Lafontaine