In my program I current create a numpy array full of zeros and then for loop through each element replacing it with the desired value. Is there a more efficient way of doing this?
Below is an example of what I am doing however, instead of a int I have a list of each row which needs put into the numpy array. Is there a way to put replace whole rows and is that more efficient.
import numpy as np
from tifffile import imsave
image = np.zeros((5, 2160, 2560), 'uint16')
num =0
for pixel in np.nditer(image, op_flags=['readwrite']):
pixel = num
num += 1
imsave('multipage.tif', image)
Just assign to the whole row using slicing
import numpy as np
from tifffile import imsave
list_of_rows = ... # all items in list should have same length
image = np.zeros((len(list_of_rows),'uint16')
for row_idx, row in enumerate(list_of_rows):
image[row_idx, :] = row
imsave('multipage.tif', image)
Numpy slicing is extremely powerful and nice. I recommend reading through this documentation to get a feeling of what is possible.
You could simply generate a vector of length 5*2160*2560 and apply this to image.
image=np.arange(5*2160*2560)
image.shape=5,2160,-1
Related
I'm having some trouble trying to check if a python tuple is in a one dimensional numpy array. I'm working on a loop that will record all the colors present in an image and store them into an array. It worked well using normal lists, but the image is very large and I think NumPy Arrays will speed up the loop as it took several minutes to complete the loop.
Here's what the code looks like:
from PIL import Image
import numpy as np
img = Image.open("bg.jpg").convert("RGB")
pixels = img.load()
colors = np.array([])
for h in range(img.size[1]):
for w in range(img.size[0]):
if pixels[w,h] not in colors:
colors = np.append(colors, pixels[w,h])
else:
continue
When I run this, I get the following error:
DeprecationWarning: elementwise comparison failed; this will raise an error in the future.
if pixels[w,h] in colors:
Thanks in advance, and if you know a faster way to do this please let me know.
I'm not sure what you need exactly. But i hope the next piece of code will help you.
import numpy as np
image = np.arange(75).reshape(5, 5, 3) % 8
# Get the set of unique pixles
pixel_list = image.reshape(-1, 3)
unique_pixels = np.unique(pixel_list, axis = 0)
# Test whether a pixel is in the list of pixels:
i = 0
pixel_in_list = (unique_pixels[i] == pixel_list).all(1).any(0)
# all(1) - all the dimensions (rgb) of the pixels need to match
# any(0) - test if any of the pixels match
# Test whether any of the pixels in the set is in the list of pixels:
compare_all = unique_pixels.reshape(-1, 1, 3) == pixel_list.reshape(1, -1, 3)
pixels_in_list = compare_all.all(2).any()
# all(2) - all the dimensions (rgb) of the pixels need to match
# any() - test if any of the pixelsin the set matches any of the pixels in the list
I found a faster way to make my loop run faster without NumPy and that is by using sets, which is way faster than using lists or NumPy. This is what the code looks like now:
from PIL import Image
img = Image.open("bg.jpg").convert("RGB")
pixels = img.load()
colors = set({})
for h in range(img.size[1]):
for w in range(img.size[0]):
if pixels[w,h] in colors:
continue
else:
colors.add(pixels[w,h])
This solves my initial problem of the lists being too slow to loop through, and it goes around the second problem of NumPy unable to compare the tuples. Thanks for all the replies, have a good day.
Assuming pixels is of shape (3, w, h) or (3, h, w) (i.e., the color channels are along the first axis), and assuming all you're after are the unique colors in the image:
channels = (channel.flatten() for channel in pixels)
colors = set(zip(*channels))
If you want a list instead of a set, colors = list(set(zip(*channels))).
You seem to be misunderstanding where numpy comes in handy. A numpy array of tuples is not going to be any faster than a a Python list of tuples. The speed of numpy comes into play in numerical computation on matrices and vectors. A numpy array of tuples cannot take advantage of any of the things that make numpy so fast.
What you're trying to do is simply not appropriate for numpy, and won't help speed up your code at all.
How would I efficiently initialise a fixed-size pyarray.ListArray
from a suitably prepared numpy array?
The documentation of pyarray.array indicates that a nested iterable input structure works, but in practice that does not work if the outer iterable is a numpy array:
import numpy as np
import pyarrow as pa
n = 1000
w = 3
data = np.arange(n*w,dtype="i2").reshape(-1,w)
# this works:
pa.array(list(data),pa.list_(pa.int16(),w))
# this fails:
pa.array(data,pa.list_(pa.int16(),w))
# -> ArrowInvalid: only handle 1-dimensional arrays
It seems ridiculus to split an input array directly matching the Arrow specification into n separate arrays and then re-assemble from there.
pyarray.ListArray.from_arrays seems to require an offsets argument, which only has a meaning for variable-size lists.
I believe you are looking for pyarrow.FixedSizeListArray.from_arrays which, regrettably, appears undocumented (I went ahead and filed a JIRA ticket)
You'll want to reshape your numpy array as a contiguous array first.
import numpy as np
import pyarrow as pa
len = 10
width = 3
# Or just skip the initial reshape but keeping it in to simulate real data
arr = np.arange(len*width,dtype="i2").reshape(-1,width)
arr.shape = -1
pa.FixedSizeListArray.from_arrays(arr, width)
In c/c++ we use to declare three-dimensional using the following syntax.
`long long dp[20][180][2]; `
memset(dp, -1, sizeof(dp));
My code:
import numpy as np
x = np.zeros((20,180,2))
How can we declare and initialize a three-dimensional array in python?
If you want all the values initialized to -1 like in your memset example, then you'd want np.full instead of np.zeros
import numpy as np
x = np.full((20,180,2), -1)
I have a 3D numpy array that is a stack of 2D (m,n) images at certain timestamps, t. So my array is of shape (t, m, n). I want to plot the value of one of the pixels as a function of time.
e.g.:
import numpy as np
import matplotlib.pyplot as plt
data_cube = []
for i in xrange(10):
a = np.random(100,100)
data_cube.append(a)
So my (t, m, n) now has shape (10,100,100). Say I wanted a 1D plot the value of index [12][12] at each of the 10 steps I would do:
plt.plot(data_cube[:][12][12])
plt.show()
But I'm getting index out of range errors. I thought I might have my indices mixed up, but every plot I generate seems to be in the 'wrong' axis, i.e. across one of the 2D arrays, but instead I want it 'through' the vertical stack. Thanks in advance!
Here is the solution: Since you are already using numpy, convert you final list to an array and just use slicing. The problem in your case was two-fold:
First: Your final data_cube was not an array. For a list, you will have to iterate over the values
Second: Slicing was incorrect.
import numpy as np
import matplotlib.pyplot as plt
data_cube = []
for i in range(10):
a = np.random.rand(100,100)
data_cube.append(a)
data_cube = np.array(data_cube) # Added this step
plt.plot(data_cube[:,12,12]) # Modified the slicing
Output
A less verbose version that avoids iteration:
data_cube = np.random.rand(10, 100,100)
plt.plot(data_cube[:,12,12])
I have an image's numpy array of shape (224,224,4). Each pixel has 4 dimension - r,g,b,alpha. I need to extract the (r,g,b) values for each pixel where it's alpha channel is 255.
I thought to first delete all elements in the array where alpha value is <255, and then extract only the first 3 values(r,g,b) of these remaining elements, but doing it in simple loops in Python is very slow. Is there a fast way to do it using numpy operations?
Something similar to this? https://stackoverflow.com/a/21017621/4747268
This should work: arr[arr[:,:,3]==255][:,:,:3]
something like this?
import numpy as np
x = np.random.random((255,255,4))
y = np.where(x[:,:,3] >0.5)
res = x[y][:,0:3]
where you have to fit > 0.5 to your needs (e.g. ==255). The result will be a matrix with all pixels stacked vertically