Related
I have a list of numpy arrays and want to modify some numbers of arrays. This is my simplified list:
first_list=[np.array([[1.,2.,0.], [2.,1.,0.], [6.,8.,3.], [8.,9.,7.]]),
np.array([[1.,0.,2.], [0.,0.,2.], [5.,5.,1.], [0.,6.,2.]])]
I have a factor which defines how many splits I have in each arrays:
spl_array=2.
it means each array of the list can be splited into 2 ones. I want to add a fixed value (3.) into last column of each split of each array and also copy the last split and subtract this value (3.) from the third column of this copied split. Finally I want to have it as following:
final_list=[np.array([[1.,2.,3.], [2.,1.,3.], [6.,8.,6.], [8.,9.,10.], \
[6.,8.,0.], [8.,9.,4.]]), # copied and subtracted
np.array([[1.,0.,5.], [0.,0.,5.], [5.,5.,4.], [0.,6.,5.], \
[5.,5.,-2.], [0.,6.,-1.]])] # copied and subtracted
I tried some for loops but I totaly lost. In advance , I do appreciate any help.
final_list=[]
for i in first_list:
each_lay=np.split (i, spl_array)
for j in range (len(each_lay)):
final_list.append([each_lay[j][:,0], each_lay[j][:,1], each_lay[j][:,2]+3])
Is it what you expect:
m = np.asarray(first_list)
m = np.concatenate((m, m[:, 2:]), axis=1)
m[:, :4, 2] += 3
m[:, 4:, 2] -= 3
final_list = m.tolist()
>>> m
array([[[ 1., 2., 3.],
[ 2., 1., 3.],
[ 6., 8., 6.],
[ 8., 9., 10.],
[ 6., 8., 0.],
[ 8., 9., 4.]],
[[ 1., 0., 5.],
[ 0., 0., 5.],
[ 5., 5., 4.],
[ 0., 6., 5.],
[ 5., 5., -2.],
[ 0., 6., -1.]]])
i have this array in a text file:
[[0, 1, 3, 1, 3, 3, 1, 5, 2, 4, 4,],
[1, 0, 4, 1, 3, 1, 1, 3, 2, 2, 3,],
[4, 3, 1, 3, 4, 5, 2, 5, 4, 5, 0,]]
my code is written with python and i want to read it and use it as a np-array in my code, how can i do that.
The problem here is the formatting; you can clean that up with regex and then simply load the whole array with np.loadtxt(). Assuming your file is called 'array.txt':
import numpy as np
import re
with open('array.txt','r') as f:
txt = f.read()
nums = re.findall(r'\[([^][]+)\]', txt)
arr = np.loadtxt(nums)
print(arr)
You can use regular expressions to extract runs of numbers without square brackets, then split each run, and convert each substring into a number:
import re
runs = re.findall(r'\[([^][]+)\]', data)
np.array([[float(n) for n in run.split()]
for run in runs])
#array([[0., 1., 3., 1., 3., 3., 1., 5., 2., 4., 4.],
# [1., 0., 4., 1., 3., 1., 1., 3., 2., 2., 3.],
# [4., 3., 1., 3., 4., 5., 2., 5., 4., 5., 0.]])
Here a solution I've seen elsewhere (Read printed numpy array)
import re
import numpy as np
from ast import literal_eval
dat = """[[0. 1. 3. 1. 3. 3. 1. 5. 2. 4. 4.]
[1. 0. 4. 1. 3. 1. 1. 3. 2. 2. 3.]
[4. 3. 1. 3. 4. 5. 2. 5. 4. 5. 0.]]"""
a = re.sub('\s+', ',', dat)
a = np.array(literal_eval(a))
output:
array([[0., 1., 3., 1., 3., 3., 1., 5., 2., 4., 4.],
[1., 0., 4., 1., 3., 1., 1., 3., 2., 2., 3.],
[4., 3., 1., 3., 4., 5., 2., 5., 4., 5., 0.]])
Suppose I have a n × m array, i.e.:
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.]])
And I what to generate a 3D array k × n × m, where all the arrays in the new axis are equal, i.e.: the same array but now 3 × 3 × 3.
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.]],
[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.]],
[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.]]])
How can I get it?
Introduce a new axis at the start with None/np.newaxis and replicate along it with np.repeat. This should work for extending any n dim array to n+1 dim array. The implementation would be -
np.repeat(arr[None,...],k,axis=0)
Sample run -
In [143]: arr
Out[143]:
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.]])
In [144]: np.repeat(arr[None,...],3,axis=0)
Out[144]:
array([[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.]],
[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.]],
[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.]]])
View-output for memory-efficiency
We can also generate a 3D view and achieve virtually free runtime with np.broadcast_to. More info - here. Hence, simply do -
np.broadcast_to(arr,(3,)+arr.shape) # repeat 3 times
if you have:
a = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
You can use a list comprehension to generate the duplicate array:
b = [a for x in range(3)]
Then (for numpy):
c = array(b)
One possibility would be to use default broadcasting to replicate your array:
a = np.arange(1, 10).reshape(3,3)
n = 3
b = np.ones((n, 3, 3)) * a
Which results in the array you wanted:
array([[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.]],
[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.]],
[[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.]]])
This won't work by default if you want to replicate it along another axis. In that case you would need to be explicit with the dimensions to ensure correct broadcasting.
I think this answer is exactly the answer of Divakar, but the syntax might be a bit easier to understand for a beginner(at least in my case, it is):
a = np.array([[1,2,3],[4,5,6]])
a[np.newaxis,:,:].repeat(3,axis=0)
results in:
array([[[1, 2, 3],
[4, 5, 6]],
[[1, 2, 3],
[4, 5, 6]],
[[1, 2, 3],
[4, 5, 6]]])
I learned about np.newaxis here: What is numpy.newaxis and when to use it.
And about numpy.repeat here: numpy.repeat
Here's an example usage I needed this for:
k = np.array([[[111,121,131,141,151],[211,221,231,241,251]],\
[[112,122,132,142,152],[212,222,232,242,252]],\
[[113,123,133,143,153],[213,223,233,243,253]]])
filter = np.array([[True,True,True,True,False],
[True,False,False,True,False]])
k[filter[None,...].repeat(3,axis=0)] = 0
print(k)
results in:
[[[ 0 0 0 0 151]
[ 0 221 231 0 251]]
[[ 0 0 0 0 152]
[ 0 222 232 0 252]]
[[ 0 0 0 0 153]
[ 0 223 233 0 253]]]
Consider the following numpy.arrays:
a = np.array([1., 2., 3.])
b = np.array([4., 5.])
c = np.array([6., 7.])
I need to combine these so I end up with the following:
[(1., 4., 6.), (1., 5., 7.), (2., 4., 6.), (2., 5., 7.), (3., 4., 6.), (3., 5., 7.)]
Note that in this case, the array a happens to be the largest array. This is not guaranteed however. Nor is the length guaranteed. In other words, any array could be the longest and each array is of arbitrary length.
I tried using itertools.izip_longest but I can only use fillvalue for the tuple with 3. which will not work. I tried itertools.product also but my result is not a true cartesian product.
You can transpose b and c and then create a product of the a with the transposed array using itertools.product:
>>> from itertools import product
>>> [np.insert(j,0,i) for i,j in product(a,np.array((b,c)).T)]
[array([ 1., 4., 6.]), array([ 1., 5., 7.]), array([ 2., 4., 6.]), array([ 2., 5., 7.]), array([ 3., 4., 6.]), array([ 3., 5., 7.])]
>>>
Let's say you have:
a = np.array([4., 5.])
b = np.array([1., 2., 3.])
c = np.array([6., 7.])
d = np.array([5., 1])
e = np.array([3., 2.])
Now, if you know before-hand which one is the longest array, which is b in this case, you can use an approach based upon np.meshgrid -
# Concatenate elements from identical positions from the equal arrays
others = np.vstack((a,c,d,e)).T # If you have more arrays, edit this line
# Get grided version of the longest array and
# grided-indices for indexing into others array
X,Y = np.meshgrid(np.arange(others.shape[0]),b)
# Concatenate grided longest array and grided indexed others for final output
out = np.hstack((Y.ravel()[:,None],others[X.ravel()]))
Sample run -
In [47]: b
Out[47]: array([ 1., 2., 3.])
In [48]: a
Out[48]: array([ 4., 5.])
In [49]: c
Out[49]: array([ 6., 7.])
In [50]: d
Out[50]: array([ 5., 1.])
In [51]: e
Out[51]: array([ 3., 2.])
In [52]: out
Out[52]:
array([[ 1., 4., 6., 5., 3.],
[ 1., 5., 7., 1., 2.],
[ 2., 4., 6., 5., 3.],
[ 2., 5., 7., 1., 2.],
[ 3., 4., 6., 5., 3.],
[ 3., 5., 7., 1., 2.]])
If the length differences are not extreme (check inputs first) I'd be tempted to pad out the shorter lists to the length of the longest with None and generate all the permutations (27 of them for 3 lists of 3 elements). Then
results = []
for candidate in possibles:
if not (None in candidate): results.append(candidate)
Reasons not to do this: if the cube of the length of the longest list is significant in terms of memory usage (space to store N cubed possibles) or CPU usage.
I would like to get several values whose i have the coordinates.
My coordinates are given by "Coord" (shape : (3, 3, 2, 3) : X and Y during 3 times and with 2 because of 2 coordinates) and my values are given by "Values" (shape : (3, 3, 3) for 3 times)
In other words, i would like to concatenate values in time with "slices" for each positions...
I dont know how to undertake that...Here there is a little part of the arrays.
import numpy as np
Coord = np.array([[[[ 4., 6., 10.],
[ 1., 3., 7.]],
[[ 3., 5., 9.],
[ 1., 3., 7.]],
[[ 2., 4., 8.],
[ 1., 3., 7.]]],
[[[ 4., 6., 10.],
[ 2., 4., 8.]],
[[ 3., 5., 9.],
[ 2., 4., 8.]],
[[ 2., 4., 8.],
[ 2., 4., 8.]]],
[[[ 4., 6., 10.],
[ 3., 5., 9.]],
[[ 3., 5., 9.],
[ 3., 5., 9.]],
[[ 2., 4., 8.],
[ 3., 5., 9.]]]])
Values = np.array([[[-4.24045246, 0.97551048, -5.78904502],
[-3.24218504, 0.9771782 , -4.79103141],
[-2.24390519, 0.97882129, -3.79298771]],
[[-4.24087775, 1.97719843, -5.79065966],
[-3.24261128, 1.97886271, -4.7926441 ],
[-2.24433235, 1.98050192, -3.79459845]],
[[-4.24129055, 2.97886284, -5.79224713],
[-3.24302502, 2.98052345, -4.79422942],
[-2.24474697, 2.98215901, -3.79618161]]])
EDIT LATER
I try in case of a simplified problem (without time first). I have used a "for loop" but
somes errors seems subsist...do you think it s the best way to treat this problem? because my arrays are important... 400x300x100
Coord3 = np.array([[[ 2, 2.],
[ 0., 1.],
[ 0., 2.]],
[[ 1., 0.],
[ 2., 1.],
[ 1., 2.]],
[[ 2., 0.],
[ 1., 1.],
[ 0., 0.]]])
Coord3 = Coord3.astype(int)
Values2 = np.array([[0., 1., 2.],
[3., 4., 5.],
[6., 7., 8.]])
b = np.zeros((3,3))
for i in range(Values2.shape[0]):
for j in range(Values2.shape[1]):
b[Coord3[i,j,0], Coord3[i,j,1]] = Values2[i,j]
b
Your second example is relatively easy to do with fancy indexing:
b = np.zeros((3,3), values2.dtype)
b[coord3[..., 0], coord3[..., 1]] = values2
The origial problem is a bit harder to do, but I think this takes care of it:
coord = coord.astype(int)
x_size = coord[..., 0, :].max() + 1
y_size = coord[..., 1, :].max() + 1
# x_size, y_size = coord.max(axis=(0, 1, 3)) + 1
nt = coord.shape[3]
b = np.zeros((x_size, y_size, nt), values.dtype)
b[coord[..., 0, :], coord[..., 1, :], np.arange(nt)] = values