Good evening,
I was wondering if someone could please provide insight into a problem I'm having. I've written a simple for loop, but it keeps stopping after only 10 iterations. Any thoughts or ideas would be greatly appreciated, thank you. Below is my code:
'''
import pandas as pd
import numpy as np
directory1 = pd.read_csv('/media/Thesis_Maps//testing/JM_rev5.csv', header=None, skiprows=[-1], encoding='utf-8')
results = np.zeros((len(directory_DOE), 3) )
for i in directory1:
x1=directory1.iloc[i,1]
y1=x1+5
results[i,0] = y1
In your example pandas iterates columns, not rows.
You would have to use directory1.iterrows() (or similar functions)
for index, row in directory1.iterrows():
results[index, 0] = row[1] + 5
but you can do the same without iteration
results[:,0] = directory1[1] + 5
Example
import pandas as pd
import numpy as np
import random
random.seed(1) # random will create always the same values
directory1 = pd.DataFrame(
[[random.randint(0, 10) for x in range(10)] for x in range(200)],
)
print('shape:', directory1.shape)
print(directory1.head())
# ----
results = np.zeros((len(directory1), 3))
for index, row in directory1.iterrows():
results[index, 0] = row[1] + 5
print(results[:5])
# ---
results = np.zeros((len(directory1), 3))
results[:,0] = directory1[1] + 5
print(results[:5])
Result
shape: (200, 10)
0 1 2 3 4 5 6 7 8 9
0 2 9 1 4 1 7 7 7 10 6
1 3 1 7 0 6 6 9 0 7 4
2 3 9 1 5 0 0 0 10 8 0
3 6 10 3 6 0 8 3 7 7 8
4 3 5 3 10 3 7 4 0 6 8
# ---
[[14. 0. 0.]
[ 6. 0. 0.]
[14. 0. 0.]
[15. 0. 0.]
[10. 0. 0.]]
# ---
[[14. 0. 0.]
[ 6. 0. 0.]
[14. 0. 0.]
[15. 0. 0.]
[10. 0. 0.]]
Related
For given odd value a, I want to generate two matrices, where values represent the offset from central row/column in x or y direction. Example for a=5:
| -2 -1 0 1 2 | | -2 -2 -2 -2 -2 |
| -2 -1 0 1 2 | | -1 -1 -1 -1 -1 |
X = | -2 -1 0 1 2 | Y = | 0 0 0 0 0 |
| -2 -1 0 1 2 | | 1 1 1 1 1 |
| -2 -1 0 1 2 | | 2 2 2 2 2 |
What is the easiest way to achieve this with Numpy?
Try meshgrid:
n=5
X,Y = np.meshgrid(np.arange(n),np.arange(n))
X -= n//2
Y -= n//2
Or
n = 5
range_ = np.arange(-(n//2), n-n//2)
X,Y = np.meshgrid(range_, range_)
Also check out ogrid.
np.arange and np.repeat will do:
a = 5
limits = -(a//2), a//2 + 1
col = np.c_[np.arange(*limits)]
Y = np.repeat(col, repeats=a, axis=1)
X = Y.T
Just use fancy indexing technique of Numpy module. The following code demonstrates the solution for a 5X5 matrix:
import numpy as np
if __name__=='__main__':
A = np.zeros((5, 5))
A[np.arange(5), :] = np.arange(5)//2 - np.arange(5)[::-1]//2
B = np.zeros((5, 5))
B[:, np.arange(5)] = np.arange(5)//2 - np.arange(5)[::-1]//2
B = B.T
Output
[[-2. -1. 0. 1. 2.]
[-2. -1. 0. 1. 2.]
[-2. -1. 0. 1. 2.]
[-2. -1. 0. 1. 2.]
[-2. -1. 0. 1. 2.]]
[[-2. -2. -2. -2. -2.]
[-1. -1. -1. -1. -1.]
[ 0. 0. 0. 0. 0.]
[ 1. 1. 1. 1. 1.]
[ 2. 2. 2. 2. 2.]]
Cheers.
Python: get all possible array attributions of nd arrays. Use itertools.product?
If so, how?
In Python, I have two n dimensions numpy arrays A and B (B is a zero array).
Such way A.shape[i]<=B.shape[i], for any i between 0 and n.
I want to create a for loop in such way every iteration I attribute A to a different subset of B, in such way every possible position in occupied until the end of the for loop.
for instance, with A = np.array([[1,1,1],[1,1,1]]) and B = np.zeros((3,4)), I would get these(one of these for each iteration):
1 1 1 0 0 1 1 1 0 0 0 0 0 0 0 0
1 1 1 0 0 1 1 1 1 1 1 0 0 1 1 1
0 0 0 0 0 0 0 0 1 1 1 0 0 1 1 1
For a fixed n dimension it is trivial, just use nested for loops for each dimension.
However, I want it for a generic n dimensions.
My approach was to use the itertools.product to get all combinations of indexes.
In the above example, product([0,1],[0,1]), would iterate over (0,0),(0,1),(1,0),(1,1), and I would have my indexes.
However, I don't know how to pass the values of the parameters to product function for a generic n.
Any idea? There are better ways of doing so?
itertools product should work.
import numpy as np
from itertools import product
A = np.ones((2,3))
B = np.zeros((3,4))
r_rng = range(B.shape[0]-A.shape[0]+1)
c_rng = range(B.shape[1]-A.shape[1]+1)
for i,j in product(r_rng, c_rng):
C = B.copy()
C[i:i+A.shape[0],j:j+A.shape[1]]=A
print(C,'\n')
Output:
[[1. 1. 1. 0.]
[1. 1. 1. 0.]
[0. 0. 0. 0.]]
[[0. 1. 1. 1.]
[0. 1. 1. 1.]
[0. 0. 0. 0.]]
[[0. 0. 0. 0.]
[1. 1. 1. 0.]
[1. 1. 1. 0.]]
[[0. 0. 0. 0.]
[0. 1. 1. 1.]
[0. 1. 1. 1.]]
Here is an example. You can use the * operator to unpack a variable number of argument from a list and give it to itertools.product():
import itertools
size1 = (3,5,6)
size2 = (2,2,2)
N = len(size1)
coords = []
for i in range(N):
delta = size1[i]-size2[i]
coords.append(list(range(delta)))
print(coords)
it = itertools.product(*coords)
arr = np.array(list(it))
print(arr)
Output:
[[0 0 0]
[0 0 1]
[0 0 2]
[0 0 3]
[0 1 0]
[0 1 1]
[0 1 2]
[0 1 3]
[0 2 0]
[0 2 1]
[0 2 2]
[0 2 3]]
Im going to post the solution I obtained:
import numpy as np
from itertools import product
A=np.ones((2,3,2))
B=np.zeros((3,4,4))
coords=[]
for i in range(len(B.shape)):
delta = B.shape[i]-A.shape[i]+1
coords.append(list(range(delta)))
print(coords)
for start_idx in product(*coords):
idx=tuple(slice(start_idx[i], start_idx[i]+A.shape[i]) for i in range(len(A.shape)))
m=np.zeros(B.shape)
m.__setitem__(tuple(idx), A)
print(m)
ps: Indexing the nd arrays was very tricky
I'm trying to use Numpy to create a y vector that will correspond to the following plot:
The x values will run from 0 to 24, the y values should be:
0 to 6 will be 0
6 to 18 will be sort of parabola
18 to 24 will be 0 again
What is a good way to do it? I don't have any practical ideas yet (I thought about some sort of interpolation).
Thank you!
I have done it assuming that you want a circle shape instead of a parabola (based on your scheme).
import numpy as np
length = 24
radius = 6
x = np.arange(length)
y = np.sqrt(radius**2-(x-(length/2))**2)
y = np.nan_to_num(y)
print(x)
# [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23]
print(y)
# [0. 0. 0. 0. 0. 0.
# 0. 3.31662479 4.47213595 5.19615242 5.65685425 5.91607978
# 6. 5.91607978 5.65685425 5.19615242 4.47213595 3.31662479
# 0. 0. 0. 0. 0. 0. ]
I Have a matrix with int inside it.
I need to replace the 111 with the median of its immediate 4 neighbourhood if any of the neighbors are 111 then they are ignored.
For eg:-
matrix = 1 2 3 4 5 6
101 111 1 3 44 3
111 3 4 4 5 6
1 2 4 5 7 7
after replacing expected op
1 2 3 4 5 6
101 2.5 1 3 44 3
3 3 4 4 5 6
1 2 4 5 7 7
My code is pretty bad and probably very slow. any help appreciated
def median_fil_mat(matrix):
rows,columns= np.where(matrix==111)
r,c=np.shape(matrix)
for each_row in rows:
for each_colmn in columns:
if each_row==r-1:
r1=[each_row-1]
elif each_row>0 & each_row!=r-1:
r1= [each_row-1,each_row+1]
else:
r1=[each_row+1]
if each_colmn ==c-1:
c1=[each_colmn-1]
elif each_colmn >0 & each_colmn!=c-1:
c1=[each_colmn-1,each_colmn+1]
else:
c1=[each_colmn+1]
med_lis=list()
for rr in r1:
for cc in c1:
med_lis.append(matrix[rr,cc])
med_lis=[x for x in med_lis if x!=111 ]
matrix[each_row,each_colmn]= np.median(med_lis)
return matrix
def func(array):
if array[2]==111:
return np.median(array[array!=111])
else:
return array[2]
fp = np.array([[0, 1, 0], [1, 1, 1], [0, 1, 0]])
a = np.fromstring("""1 2 3 4 5 6
101 111 1 3 44 3
111 3 4 4 5 6
1 2 4 5 7 7""", sep=" ").reshape((4, 6))
generic_filter(a, func, footprint=fp, mode='nearest')
Returns
array([[ 1. , 2. , 3. , 4. , 5. , 6. ],
[ 101. , 2.5, 1. , 3. , 44. , 3. ],
[ 3. , 3. , 4. , 4. , 5. , 6. ],
[ 1. , 2. , 4. , 5. , 7. , 7. ]])
I prefer python over R for my work. From time to time, I need to use R
functions, and I start to try Rpy2 for that purpose.
I tried but failed to find out how to replicate following with Rpy2
design <- model.matrix(~Subject+Treat)
I have gone as far as this:
import rpy2.robjects as robjects
fmla = robjects.Formula('~subject+treatment')
env = fmla.environment
env['subject'] = sbj_group
env['treatment'] = trt_group
from what I saw here.
But I could not find how to perform model.matrix. I tried a couple of different ways:
robjects.r.model_matrix(fmla)
robjects.r('model.matrix(%s)' %fmla.r_repr())
As you can see none of them is right.
I am new to Rpy2, and fairly inexperienced in R. Any help would be appreciated!
You could evaluate strings as R code:
import numpy as np
import rpy2.robjects as ro
import rpy2.robjects.numpy2ri
ro.numpy2ri.activate()
R = ro.r
subject = np.repeat([1,2,3], 4)
treatment = np.tile([1,2,3,4], 3)
R.assign('subject', subject)
R.assign('treatment', treatment)
R('subject <- as.factor(subject)')
R('treatment <- as.factor(treatment)')
R('design <- model.matrix(~subject+treatment)')
R('print(design)')
yields
(Intercept) subject2 subject3 treatment2 treatment3 treatment4
1 1 0 0 0 0 0
2 1 0 0 1 0 0
3 1 0 0 0 1 0
4 1 0 0 0 0 1
5 1 1 0 0 0 0
6 1 1 0 1 0 0
7 1 1 0 0 1 0
8 1 1 0 0 0 1
9 1 0 1 0 0 0
10 1 0 1 1 0 0
11 1 0 1 0 1 0
12 1 0 1 0 0 1
attr(,"assign")
[1] 0 1 1 2 2 2
attr(,"contrasts")
attr(,"contrasts")$subject
[1] "contr.treatment"
attr(,"contrasts")$treatment
[1] "contr.treatment"
R(...) returns objects which you can manipulate on the Python side.
For example,
design = R('model.matrix(~subject+treatment)')
assigns a rpy2.robjects.vectors.Matrix to design.
arr = np.array(design)
makes arr the NumPy array
[[ 1. 0. 0. 0. 0. 0.]
[ 1. 0. 0. 1. 0. 0.]
[ 1. 0. 0. 0. 1. 0.]
[ 1. 0. 0. 0. 0. 1.]
[ 1. 1. 0. 0. 0. 0.]
[ 1. 1. 0. 1. 0. 0.]
[ 1. 1. 0. 0. 1. 0.]
[ 1. 1. 0. 0. 0. 1.]
[ 1. 0. 1. 0. 0. 0.]
[ 1. 0. 1. 1. 0. 0.]
[ 1. 0. 1. 0. 1. 0.]
[ 1. 0. 1. 0. 0. 1.]]
The column names can be accessed with
np.array(design.colnames)
# array(['(Intercept)', 'subject2', 'subject3', 'treatment2', 'treatment3',
# 'treatment4'],
# dtype='|S11')