I have a data np.array "A" and np.array with ranges[from-to index] "I" to be obtained from A.
How to create a new np array/or list ?
A=[1 161 51 105 143 2 118 127 37 19 4 29 13 136 129 128 129
250 52 53 57 53 49 53 57 49 55 177 84 69 85 210 6 43 128
194 253 0 236 129 131 53 54 56 54 50 48 182 128 52 113 13 169
57 41 233 128 254 160 128 9 81 75 166 89 178 128 128 128 128 128
128 177 128 84 81 84 197 206]
I=[[ 0 2]
[ 2 5]
[ 5 8]
[ 8 14]
...
]
The new array should be like this:
[[1 161 nul] [51 105 143] ... ]
I am not sure why you have a 'null' in one of your intervals. But you can do this using a list comprehension:
import numpy as np
A=np.array([1, 161, 51, 105, 143, 2, 118, 127 , 37, 19, 4 , 29 , 13, 136, 129, 128, 129])
I=[[ 0, 2],
[ 2 ,5],
[ 5 ,8],
[ 8, 14]]
res = [A[i[0]:i[1]] for i in I]
Output:
[array([ 1, 161]),
array([ 51, 105, 143]),
array([ 2, 118, 127]),
array([ 37, 19, 4, 29, 13, 136])]
Related
I have a list of, say, size 144. I need every 16 elements to be stored in a 2-D array. In the end I would want to have a 16x9 2-D array. How do I pull these elements out in a loop and rearrange them? Here is what I have so far...
yuH = np.zeros([N, t])
i = 0
j = 0
for i in range(t):
for j in range(len(p)):
print(p[0+j:16+j])
yuH[:,i] = p[0+j:16+j] # change to N
j = j + 16
Where N in this case is 16 and t is 9. p is my list of 144. yuH is the matrix I am trying to create. alas it is not working...
I get the ValueError "setting an array element with a sequence" at the line yuH[:,i] = p[0+j:16+j] after many runs.
Something like...
p = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]
p = [[1, [5, 9, [13, [17,
2, 6, 10, 14, 18,
3, 7, 11, 15, 19,
4], 8] 12], 16], 20]]
If what you want is just reorganize you array p you can just do this:
import numpy as np
N = 16
t = 9
p = np.arange(t*N) # example array p
yuH = p.reshape([t,N])
print(yuH.shape)
>>> (9, 16)
print(yuH)
>>>
[[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15]
[ 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31]
[ 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47]
[ 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63]
[ 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79]
[ 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95]
[ 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111]
[112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127]
[128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143]]
I am trying to separate the pixel values of an image in python which are in a numpy array of 'object' data-type in a single quote like this:
['238 236 237 238 240 240 239 241 241 243 240 239 231 212 190 173 148 122 104 92 .... 143 136 132 127 124 119 110 104 112 119 78 20 17 19 20 23 26 31 30 30 32 33 29 30 34 39 49 62 70 75 90']
The shape of the numpy array is coming as 1.
There are a total of 784 numbers but I cannot access them individually.
I wanted something like:
[238, 236, 237, ......, 70, 75, 90] of dtype int or float.
There are 1000 such numpy arrays like the one above.
Thanks in advance.
You can use str.split
Ex:
l = ['238 236 237 238 240 240 239 241 241 243 240 239 231 212 190 173 148 122 104 92 143 136 132 127 124 119 110 104 112 119 78 20 17 19 20 23 26 31 30 30 32 33 29 30 34 39 49 62 70 75 90']
print( list(map(int, l[0].split())) )
Output:
[238, 236, 237, 238, 240, 240, 239, 241, 241, 243, 240, 239, 231, 212, 190, 173, 148, 122, 104, 92, 143, 136, 132, 127, 124, 119, 110, 104, 112, 119, 78, 20, 17, 19, 20, 23, 26, 31, 30, 30, 32, 33, 29, 30, 34, 39, 49, 62, 70, 75, 90]
I believe using np.ndarray.item() is idiomatic to retrieve a single item from a numpy array.
import numpy as np
your_numpy_array = np.asarray(['238 236 237 238 240 240 239 241 241 243 240 239 231 212 190 173 148 122 104 92 143 136 132 127 124 119 110 104 112 119 78 20 17 19 20 23 26 31 30 30 32 33 29 30 34 39 49 62 70 75 90'] )
values = your_numpy_array.item().split(' ')
new_numpy_array = np.asarray(values, dtype='int')
Note that values here is a list of strings. np.asarray can construct an array of integers from list of string values, we just need to specify the dtype (as suggested by hpaulj)
I'm trying to convert an image to grayscale without using OpenCV or Numpy
Example :
The matrix of my image is
[[[116 116 117]
[115 115 116]
[117 115 115]
...,
[135 138 142]
[137 139 139]
[137 139 139]]
[[116 116 116]
[116 116 116]
[114 116 116]
...,
[135 139 140]
[135 137 138]
[135 137 138]]
[[115 118 114]
[115 118 114]
[112 116 115]
...,
[132 141 141]
[134 137 141]
[133 136 140]]
...,
[[ 35 44 163]
[ 31 40 159]
[ 10 33 158]
...,
[ 14 48 53]
[ 24 56 55]
[ 27 59 58]]
[[ 24 38 156]
[ 19 33 151]
[ 7 28 145]
...,
[ 25 55 66]
[ 15 59 61]
[ 17 61 63]]
[[ 0 27 131]
[ 0 26 130]
[ 0 34 113]
...,
[ 11 39 55]
[ 6 28 56]
[ 4 26 54]]]
Using the cv2.COLOR_BGR2GRAY function the matrix is :
[[150 150 150 ..., 150 150 150]
[150 173 175 ..., 97 91 89]
[150 176 179 ..., 95 89 82]
...,
[150 66 67 ..., 152 154 152]
[150 62 59 ..., 152 152 152]
[150 62 64 ..., 155 154 151]]
First, i don't get why it returns a 36 integers when i gave 108 and how can i do it manually
Thanks for your help :)
Question 1
You are getting 36 integers instead of 108 because your original image had 3 channels (red, green, blue, a.k.a RGB). When you convert it to grayscale, you are flattening it to only one channel. 108 / 3 = 36, seems legit!
Quesion 2
Converting it by hand: There are multiple ways to convert an RGB image to grayscale, but the most straightforward would be to take the average of the three channels, basically (red_values + green_values + blue values) / 3. So take your original image array, and run:
gray_image = (image[:,:,0] + image[:,:,1] + image[:,:,2]) / 3
The above is assuming that your image array is in the form image.shape = (pixel_height, pixel_width, number_of_channels), and that the first 3 channels represent your R,G, and B channels but that's pretty standard
I have a array as:
[0 1 2 3 4 5 6 7 8 ....] #2560 positions
I'm trying to reshape this 1D array to 3D as:
A = np.arange(2560) #generating example data
B = np.reshape(A,(16,16,10)) # here, i's expect to be 16 rows 16 x columns x 10 "frames"
C = B[:,:,0]
However, the result is giving something like this
C = [[0, 10, 20, 30, .., 150]
[160, 170, 180, ..., 310]
...
[2510,...2550]
The correct would be
print(B[:,:,0])
[[0, 1, 2, 3, .., 15]
[16, 17, 18, ..., 31]
...
[240,241,..., 255]
print(B[:,:,1])
[[256, 257, , .., 271]
...
[496,497,..., 511]
What I'm doing wrong?
The idea is to not use for's in order to make it faster
I think you don't understand how your data is structured.
It is a 3D matrix.
First you have 16 rows, each row contains 16 columns, each columns contains 10 lists
However, the rows and columns do not contains any numbers, rows contain columns and columns contain lists. Only lists contain number.
So when you execute: print(B[:,:,0]), you ask: "I want the first number of the list contains in every columns of every row". So you obtain exactely what you ask for.
You can not obtain this:
print(B[:,:,0])
[[0, 1, 2, 3, .., 15]
[16, 17, 18, ..., 31]
...
[240,241,..., 255]
print(B[:,:,1])
[[256, 257, , .., 271]
...
[496,497,..., 511]
This is not the way data are stored, you have to manipulate your data to make them appear in this way.
You can do the following:
A = np.arange(2560)
B = np.reshape(A,(10,16,16))
print(B[0,:,:])
[[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15]
[ 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31]
[ 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47]
[ 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63]
[ 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79]
[ 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95]
[ 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111]
[112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127]
[128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143]
[144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159]
[160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175]
[176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191]
[192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207]
[208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223]
[224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239]
[240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255]]
Simply reshaping won't give you the desired format, as you found out yourself. To solve it, we need to reshape differently and then permute axes. The general theory could be followed here - Reshape and permute axes. The solution would be -
A.reshape(-1,16,16).transpose(1,2,0)
Sample run -
In [465]: A = np.arange(2560)
In [466]: B = A.reshape(-1,16,16).transpose(1,2,0)
In [467]: B[:,:,0]
Out[467]:
array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,
13, 14, 15],
[ 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,
29, 30, 31] ...
In [468]: B[:,:,1]
Out[468]:
array([[256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268,
269, 270, 271],
[272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284,
285, 286, 287]...
We can also use np.moveaxis to perform the permuting -
np.moveaxis(A.reshape(-1,16,16), 0, -1)
I'm trying to write some numpy arrays in python to lmdb:
import numpy as np
import lmdb
def write_lmdb(filename):
lmdb_env = lmdb.open(filename, map_size=int(1e9))
lmdb_txn = lmdb_env.begin(write=True)
X= np.array([[1.0, 0.0], [0.1, 2.0]])
y= np.array([1.4, 2.1])
#Put first pair of arrays
lmdb_txn.put('X', X)
lmdb_txn.put('y', y)
#Put second pair of arrays
lmdb_txn.put('X', X+1.6)
lmdb_txn.put('y', y+1.2)
def read_lmdb(filename):
lmdb_env = lmdb.open(filename)
lmdb_txn = lmdb_env.begin()
lmdb_cursor = lmdb_txn.cursor()
for key, value in lmdb_cursor:
print type(key)
print type(value)
print key
print value
write_lmdb('temp.db')
read_lmdb('temp.db')
but read_lmdb prints nothing, what is the proper way to write numpy arrays to lmdb?
Update:
Based on #frankyjuang answer I manage to do it, howewer not in very elegant way: multidimensional array lose it's shape, each array should have it's own name.
import numpy as np
import lmdb
def write_lmdb(filename):
print 'Write lmdb'
lmdb_env = lmdb.open(filename, map_size=int(1e9))
n_samples= 2
X= (255*np.random.rand(n_samples,3,4,3)).astype(np.uint8)
y= np.random.rand(n_samples).astype(np.float32)
for i in range(n_samples):
with lmdb_env.begin(write=True) as lmdb_txn:
lmdb_txn.put('X_'+str(i), X)
lmdb_txn.put('y_'+str(i), y)
print 'X:',X
print 'y:',y
def read_lmdb(filename):
print 'Read lmdb'
lmdb_env = lmdb.open(filename)
lmdb_txn = lmdb_env.begin()
lmdb_cursor = lmdb_txn.cursor()
n_samples=0
with lmdb_env.begin() as lmdb_txn:
with lmdb_txn.cursor() as lmdb_cursor:
for key, value in lmdb_cursor:
print key
if('X' in key):
print np.fromstring(value, dtype=np.uint8)
if('y' in key):
print np.fromstring(value, dtype=np.float32)
n_samples=n_samples+1
print 'n_samples',n_samples
write_lmdb('temp.db')
read_lmdb('temp.db')
Test script output should be something like:
Write lmdb
X: [[[[ 48 224 119]
[ 76 87 174]
[ 14 88 183]
[ 76 234 56]]
[[234 223 65]
[ 63 85 175]
[184 252 125]
[100 7 225]]
[[134 159 41]
[ 2 146 221]
[ 99 74 225]
[169 57 59]]]
[[[100 202 3]
[ 88 204 131]
[ 96 238 243]
[103 58 30]]
[[157 125 107]
[238 207 99]
[102 220 64]
[ 27 240 33]]
[[ 74 93 131]
[107 88 206]
[ 55 86 35]
[212 235 187]]]]
y: [ 0.80826157 0.01407595]
X: [[[[ 48 224 119]
[ 76 87 174]
[ 14 88 183]
[ 76 234 56]]
[[234 223 65]
[ 63 85 175]
[184 252 125]
[100 7 225]]
[[134 159 41]
[ 2 146 221]
[ 99 74 225]
[169 57 59]]]
[[[100 202 3]
[ 88 204 131]
[ 96 238 243]
[103 58 30]]
[[157 125 107]
[238 207 99]
[102 220 64]
[ 27 240 33]]
[[ 74 93 131]
[107 88 206]
[ 55 86 35]
[212 235 187]]]]
y: [ 0.80826157 0.01407595]
Read lmdb
X_0
[ 48 224 119 76 87 174 14 88 183 76 234 56 234 223 65 63 85 175
184 252 125 100 7 225 134 159 41 2 146 221 99 74 225 169 57 59
100 202 3 88 204 131 96 238 243 103 58 30 157 125 107 238 207 99
102 220 64 27 240 33 74 93 131 107 88 206 55 86 35 212 235 187]
X_1
[ 48 224 119 76 87 174 14 88 183 76 234 56 234 223 65 63 85 175
184 252 125 100 7 225 134 159 41 2 146 221 99 74 225 169 57 59
100 202 3 88 204 131 96 238 243 103 58 30 157 125 107 238 207 99
102 220 64 27 240 33 74 93 131 107 88 206 55 86 35 212 235 187]
y_0
[ 0.80826157 0.01407595]
y_1
[ 0.80826157 0.01407595]
n_samples 4
Wrap your transactions under with. And remember to convert the value from bytes (string) back to numpy array using np.fromstring.
To be honest, it is not a good idea to store numpy array in lmdb since conversion from array to bytes back to array will lose some informations (ex. shape). You can try storing a dict of numpy arrays using pickle.
def write_lmdb(filename):
...
with lmdb_env.begin(write=True) as lmdb_txn:
...
def read_lmdb(filename):
...
with lmdb_env.begin() as lmdb_txn:
with lmdb_txn.cursor() as lmdb_cursor:
...
print np.fromstring(value, dtype=np.float64)