I have a big array and a part of that is shown in small example below.
in each list, the first number is start and the 2nd number is end
(so there is a range) but each element is a part of a sequence and
each sequence starts from zero.
. what I want to do is:
small example:
array([[ 469, 1300],
[ 171, 1440],
[ 187, 1564],
[ 204, 1740],
[ 40, 1363],
[ 56, 1457],
[ 132, 606],
[1175, 2096],
[ 484, 2839],
[ 132, 4572],
[ 166, 1693],
[ 69, 3300],
[ 142, 1003],
[2118, 2118],
[ 715, 1687],
[ 301, 1006],
[ 48, 2142],
[ 63, 330],
[ 479, 2411]], dtype=uint32)
I want to take from the begining of each sequence which is indexed 0
(not included in the ranges) until -20 before
the start of each range.
I tried this code:
cds = np.column_stack([cdspos[:, 0] - cdspos[:, 0], cdspos[:, 0] - 20])
but it gives this error:
y = _nx.arange(0, num, dtype=dt)
MemoryError
but I try this one:
cds = np.column_stack([cdspos[:, 0] - 100, cdspos[:, 0] - 20])
it works perfectly. the problem is that I don't have the same range
before the starting point (all of them are not 100).
so, I want to get a smaller ranges (lists) like this: (0) to
(start-20). for example the first element would be like this:
[ 0, 449]
I also tried
cds = np.column_stack([0, cdspos[:, 0] - 20])
but did not work.
for the small example, the output I am looking for would be like this:
array([[0, 449],
[0, 151],
[0, 167],
[0, 184],
[0, 20],
[0, 36],
[0, 112],
[0, 1155],
[0, 464],
[0, 112],
[0, 146],
[0, 49],
[0, 122],
[0, 2098],
[0, 695],
[0, 281],
[0, 28],
[0, 43],
[0, 459]], dtype=uint32)
do you guys know how to do that?
What about
In [17]: numpy.array([[0, start -20] for start, end in a], dtype=numpy.uint32)
Out[17]:
array([[ 0, 449],
[ 0, 151],
[ 0, 167],
[ 0, 184],
[ 0, 20],
[ 0, 36],
[ 0, 112],
[ 0, 1155],
[ 0, 464],
[ 0, 112],
[ 0, 146],
[ 0, 49],
[ 0, 122],
[ 0, 2098],
[ 0, 695],
[ 0, 281],
[ 0, 28],
[ 0, 43],
[ 0, 459]], dtype=uint32)
Here is one way:
In [22]: np.column_stack((np.zeros(a.shape[0], dtype=np.int8), (a[:,0]- 20).astype(np.int16)))
Out[22]:
array([[ 0, 449],
[ 0, 151],
[ 0, 167],
[ 0, 184],
[ 0, 20],
[ 0, 36],
[ 0, 112],
[ 0, 1155],
[ 0, 464],
[ 0, 112],
[ 0, 146],
[ 0, 49],
[ 0, 122],
[ 0, 2098],
[ 0, 695],
[ 0, 281],
[ 0, 28],
[ 0, 43],
[ 0, 459]], dtype=int16)
Note that in this case I used np.int8 for zeros and np.int16 (or use np.uint32 if your numbers are larger to be fit on int16)for second column numbers. If you get memory error again this means that you can't preserve your arrays at once in your RAM. You can divide your array to multiple parts and apply the operations on different parts separately and process them separately too. If this is not possible
Related
my lists look something like this:
A_List= [0,1,2,3,4,5,6,7,8]
B_List=[0,10,20,30,40,50,60,70,80]
C_List=[0,100,200,300,400,500,600,700,800]
D_List = ...
And so on.
Each value is type np.float64.
The values here are random, I only wanted to show that they are all the same length.
I now tried to write a loop that changes these lists starting at a specific index in that way, that all numbers including and above index 4 are subtracted by the value written in index 4.
i=4
while i <= len(A_List):
A_List[i] = A_List[i]+A_List[4]
B_List[i] = B_List[i]+B_List[4]
C_List[i] = C_List[i]+C_List[4]
D_List[i] = D_List[i]+D_List[4]
...
i=i+1
Which just won't work. Error: can only concatenate str (not "numpy.float64") to str). I don't quite understand that, because I thought I'm substituting a float value with another float value.
Why don't you simply use numpy and not lists?
First create a 2D array:
A_List= [0,1,2,3,4,5,6,7,8]
B_List=[0,10,20,30,40,50,60,70,80]
C_List=[0,100,200,300,400,500,600,700,800]
a = np.c_[A_List, B_List, C_List]
array([[ 0, 0, 0],
[ 1, 10, 100],
[ 2, 20, 200],
[ 3, 30, 300],
[ 4, 40, 400],
[ 5, 50, 500],
[ 6, 60, 600],
[ 7, 70, 700],
[ 8, 80, 800]])
Then perform your subtraction:
>>> a-a[4]
array([[ -4, -40, -400],
[ -3, -30, -300],
[ -2, -20, -200],
[ -1, -10, -100],
[ 0, 0, 0],
[ 1, 10, 100],
[ 2, 20, 200],
[ 3, 30, 300],
[ 4, 40, 400]])
If you want to apply your transform only on rows ≥ 4:
mask = np.tile((np.arange(len(a))>=4), (a.shape[1], 1)).T
np.where(mask, a-a[4], a)
output:
array([[ 0, 0, 0],
[ 1, 10, 100],
[ 2, 20, 200],
[ 3, 30, 300],
[ 0, 0, 0],
[ 1, 10, 100],
[ 2, 20, 200],
[ 3, 30, 300],
[ 4, 40, 400]])
I'm trying to create list from existing matrix:
a = matrix([[ 0, 0, 0, ..., 82, 140, 165],
[ 0, 0, 0, ..., 30925, 30830, 27075],
[ 0, 0, 0, ..., 628, 678, 528],
...,
[ 0, 0, 0, ..., 988, 930, 878],
[ 0, 1, 0, ..., 21140, 24720, 22681],
[ 0, 0, 0, ..., 1809, 1655, 1560]])
b = matrix([[ 0, 0, 0, ..., 26, 18, 14],
[ 0, 0, 0, ..., 1473, 1005, 904],
[ 0, 0, 0, ..., 138, 61, 72],
...,
[ 0, 0, 0, ..., 21, 18, 21],
[ 0, 0, 0, ..., 1036, 921, 670],
[ 0, 0, 0, ..., 176, 357, 204]])
c = matrix([[ 0, 0, 0, ..., 89, 64, 78],
[ 0, 0, 0, ..., 16346, 11977, 10209],
[ 0, 0, 0, ..., 678, 890, 1013],
...,
[ 0, 0, 0, ..., 926, 1622, 1922],
[ 0, 0, 0, ..., 5744, 6704, 9143],
[ 0, 0, 0, ..., 7, 9, 2]])
a.shape
(13, 136)
b.shape
(13, 136)
c.shape
(13, 136)
expected output: I would like to create a list of list like that
[ [a[0],b[0],c[0]], ... , [a[n],b[n],c[n]] ]
example with first and last row of each matrix:
[ [[ 0, 0, 0, ..., 82, 140, 165],[ 0, 0, 0, ..., 26, 18, 14],[ 0, 0, 0, ..., 89, 64, 78]] , ... ,[[ 0, 0, 0, ..., 1809, 1655, 1560],[ 0, 0, 0, ..., 176, 357, 204], [ 0, 0, 0, ..., 7, 9, 2]] ]
Thank you for your help
a = np.matrix([[1, 2, 3],
[12, 9, 0],
[2, 45, 2]])
b = np.matrix([[12, 9, 0],
[2, 67, 94],
[2, 45, 2]])
x = [list(t) for t in zip(a.tolist(),b.tolist())]
print(x)
Out:
[[[1, 2, 3], [12, 9, 0]], [[12, 9, 0], [2, 67, 94]], [[2, 45, 2], [2, 45, 2]]]
Let's say I have the following array:
a = np.random.randint(5, size=(2000, 2000, 1))
a = np.repeat(a, 3, axis=2) # Using this method to have a (m,n,3) array with the same values
and the next arrays:
val_old = np.array([[0, 0, 0], [3, 3, 3]])
val_new = np.array([[12, 125, 13], [78, 78, 0]])
What I want to do is to replace the values from the array a with the values specified in the array val_new. So, all [0,0,0] arrays would become [12,125,13] and all [3,3,3] would become [78, 78, 0].
I can't find an efficient way to do this... I tried to adapt this solution but it's only for 1-d arrays...
Does anyone know a fast way/method to replace these values ?
Assuming you have a "map" for each integer, you can use a (2000, 2000) index on a (5,) array to broadcast to a (2000,2000, 5) array. example:
val_new = np.array([[12, 125, 13], [0,0,0], [1,3,3], [78, 78, 0]]) #0, 1, 2, 3
a = np.random.randint(4,size=(4,5))
val_new[a] # (4,5,3) shaped array
>>array([[[ 0, 0, 0],
[ 78, 78, 0],
[ 78, 78, 0],
[ 12, 125, 13],
[ 0, 0, 0]],
....
[[ 12, 125, 13],
[ 12, 125, 13],
[ 0, 0, 0],
[ 12, 125, 13],
[ 0, 0, 0]]])
I have a .png image with four colors in it. If I convert the image to a numpy array I get an array with the following dimensions: [length X height X 3], with length == height.
How can I reduce the dimension with mapping the colors?
This is the current structure:
array([[[ 0, 65, 101],
[ 0, 65, 101],
[ 0, 65, 101],
...,
[ 0, 65, 101],
[ 0, 65, 101],
[ 0, 65, 101]],
[[ 0, 65, 101],
[163, 219, 232],
[163, 219, 232],
...,
[ 0, 65, 101],
[163, 219, 232],
[ 0, 65, 101]],
[[ 0, 65, 101],
[163, 219, 232],
[ 0, 65, 101],
...,
[ 0, 65, 101],
[163, 219, 232],
[ 0, 65, 101]],
...,
[[ 0, 65, 101],
[163, 219, 232],
[ 0, 65, 101],
...,
[ 0, 65, 101],
[ 0, 65, 101],
[ 0, 65, 101]],
[[ 0, 65, 101],
[163, 219, 232],
[163, 219, 232],
...,
[163, 219, 232],
[163, 219, 232],
[ 0, 65, 101]],
[[ 0, 65, 101],
[ 0, 65, 101],
[ 0, 65, 101],
...,
[ 0, 65, 101],
[ 0, 65, 101],
[ 0, 65, 101]]], dtype=uint8)
And I would like an array with two dimensions, and every value in the i'th row and j'th column would correspond to the color it had in the third dimension. So if the original image had 7 X 7 X 3 dimension with four colors, the output would be something like this:
array([[0, 1, 1, 3, 3, 3, 0],
[0, 2, 1, 1, 1, 1, 0],
[0, 2, 0, 1, 2, 1, 0],
[0, 3, 1, 1, 3, 1, 0],
[0, 1, 0, 0, 3, 0, 0],
[0, 1, 1, 1, 1, 1, 0],
[0, 0, 0, 0, 0, 0, 0]])
The values in the forementioned arrays are all made up, so they don't correspond to each other, I have just tried to represent the concept.
I read the picture as:
from PIL import Image
import numpy as np
img = Image.open('image.png')
imgarray = np.asarray(img)
print(imgarray)
You can use numpy.unique for this. For example, here's a 3x5 image that has just three colors:
In [105]: img
Out[105]:
array([[[10, 20, 30],
[ 5, 5, 0],
[ 5, 5, 0],
[ 5, 5, 0],
[ 0, 0, 0]],
[[ 5, 5, 0],
[ 5, 5, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0]],
[[10, 20, 30],
[10, 20, 30],
[10, 20, 30],
[10, 20, 30],
[ 5, 5, 0]]])
Call numpy.unique on the reshaped image. The first two dimensions are flattened into a single dimension, and then axis=0 is used so we get the unique colors. inv will holds the array of "inverses", i.e. the indices into colors of the original values.
In [106]: colors, inv = np.unique(img.reshape(-1, 3), axis=0, return_inverse=True)
In [107]: colors
Out[107]:
array([[ 0, 0, 0],
[ 5, 5, 0],
[10, 20, 30]])
In [108]: inv
Out[108]: array([2, 1, 1, 1, 0, 1, 1, 0, 0, 0, 2, 2, 2, 2, 1])
Reshape inv to get the array of indices into colors with the same shape as the original image:
In [109]: inv.reshape(img.shape[:2])
Out[109]:
array([[2, 1, 1, 1, 0],
[1, 1, 0, 0, 0],
[2, 2, 2, 2, 1]])
I have a big 2D array. I would like to replace specific array element by another array. To be more clear, here is an example that I would like to do in python:
a = np.random.randint(0, 255, size=(10, 3))
print "a : \n", a
a[a<200]=0
a[a>=200]=255
print "Normalize a : \n", a
It gives any random output like:
a :
[[119 19 16]
[226 33 102]
[163 188 235]
[ 22 85 176]
[ 25 188 85]
[ 78 146 141]
[229 10 37]
[ 38 79 92]
[142 107 213]
[107 228 145]]
Normalize a :
[[ 0 0 0]
[255 0 0]
[ 0 0 255]
[ 0 0 0]
[ 0 0 0]
[ 0 0 0]
[255 0 0]
[ 0 0 0]
[ 0 0 255]
[ 0 255 0]]
Now, I would like to replace all [255 0 0] with [255 255 255]. It is possible to use any loop like for loop but very time consume if it is a very large array. Is there any simple why to use like np.where or someting else that I used to replace like all elements below 200 by using a[a<200]=0 ?
IIUC, yes, you can use a similar method as you did above, with np.all:
Example:
a = np.random.randint(0, 255, size=(10, 3))
>>> a
array([[ 10, 113, 91],
[ 99, 63, 164],
[ 96, 130, 35],
[193, 38, 11],
[193, 150, 200],
[242, 195, 28],
[ 6, 205, 168],
[156, 35, 242],
[246, 84, 61],
[111, 187, 18]])
# Use `np.where` to make your normalization
b = np.where(a >= 200, 255, 0)
>>> b
array([[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 255],
[255, 0, 0],
[ 0, 255, 0],
[ 0, 0, 255],
[255, 0, 0],
[ 0, 0, 0]])
# Replace [255,0,0] with [255,255,255]
b[np.all(b==[255,0,0], axis=1)] = [255,255,255]
>>> b
array([[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 255],
[255, 255, 255],
[ 0, 255, 0],
[ 0, 0, 255],
[255, 255, 255],
[ 0, 0, 0]])
Explanation:
np.all(b==[255,0,0],axis=1) Returns where your array matches all elements of [255,0,0]:
array([False, False, False, False, False, True, False, False, True,
False])
So you can index your array by that to return those vectors:
>>> b[np.all(b==[255,0,0],axis=1)]
array([[255, 0, 0],
[255, 0, 0]])
And replace them in a similar manner as you had above.