We Keep Coding

Numpy Extract Data from Compressed Sparse Column Format

I have a mat file with sparse data for around 7000 images with 512x512 dimensions stored in a flattened format (so rows of 262144) and I’m using scipy’s loadmat method to turn this sparse information into a Compressed Sparse Column format. The data inside of these images is a smaller image that’s usually around 25x25 pixels somewhere inside of the 512x512 region , though the actual size of the smaller image is not consitant and changes for each image. I want to get the sparse information from this format and turn it into a numpy array with only the data in the smaller image; so if I have an image that’s 512x512 but there’s a circle in a 20x20 area in the center I want to just get the 20x20 area with the circle and not get the rest of the 512x512 image. I know that I can use .A to turn the image into a non-sparse format and get a 512x512 numpy array, but this option isn’t ideal for my RAM.
Is there a way to extract the smaller images stored in a sparse format without turning the sparse data into dense data?
I tried to turn the sparse data into dense data, reshape it into a 512x512 image, and then I wrote a program to find the top, bottom, left, and right edges of the image by checking for the first occurrence of data from the top, bottom, left, and right but this whole processes seemed horribly inefficient.

Sorry about the little amount of information I provided; I ended up figuring it out.Scipy's loadmat function when used to extract sparse data from a mat file returns a csc_matrix, which I then converted to numpy's compressed sparse column format. Numpy's format has a method .nonzero() that will return the index of every non_zero element in that matrix. I then reshaped the numpy csc matrix into 512x512, and then used .nonzero() to get the non-zero elements in 2D, then used used those indexes to figure out the max height and width of my image I was interested in. Then I created a numpy matrix of zeros the size of the image I wanted, and set the elements in that numpy matrix to the elements to the pixels I wanted by indexing into my numpy csc matrix (after I called .tocsr() on it)

Template matching two 3D numpy array

I have an 3D numpy array, start_array, I do some processing to it by applying some random translations and rotations to get transform_arry. I have access to both the arrays only. I want to identify the random transforms done to the arrays. What would be a quick and easy way to do it using python?

Loading 3D Model but getting 2D Array in Python

I`ve downloaded a sample .stl file from here: [https://www.thingiverse.com/thing:156207]
Then I've used this code to get a numpy array for further image processing with matplotlib:
import numpy as np
from stl import mesh
np.set_printoptions(threshold=np.nan)
# Using an existing stl file:
your_mesh = mesh.Mesh.from_file('300_polygon_sphere_100mm.stl')
data = np.array(your_mesh)
print(data.shape)
Unfortunately, this is an array with only two dimensions. I've checked the .stl file with my editor and there are three dimensions.
Can someone help me? My goal is to create a code with that i can slice 3D models to get acces to the sliced 2d images.
Thanks.
EDIT: I've tried to reshape it:
data_reshaped = np.reshape(data, (550, 3, 3))
But i guess this totally wrong. And i don't know if the pattern is (Z, X, Y).
I want to do some slicing operations on the 3d array to get XY images like this guy is very easily doing https://www.youtube.com/watch?v=5jQVQE6yfio&list=PLT66ZlnovHPYzny9TYM1mx02k5Xnw_kjw&t=215s&index=3

You won't be able to just load the .stl file into a numpy array and perform slicing as shown in the video you linked. In the video, they load a model that is stored as a 3D numpy array.
However, the model you are trying to load consists of a polygonal mesh. This means you only have the coordinate values of the vertices. You can open the .stl file in a text editor to see its contents. (By converting the loaded mesh into a numpy array you just extract those coordinate values. You can actually compare the values in the numpy array and the text file, they are the same.) The resulting numpy array has shape (550, 9). The first dimension is defined by the number of faces in the model (in this case, the model has 550 faces). As each face has three vertices, which have three coordinate values each, hence you have 9 numbers per face. So the third dimension is not lost. It's just stored in a different manner.
Simply reshaping the array won't create you a model of which you can get slices of, as shown in the video. To achieve this, you have to convert the meshed model into a rasterized one. You could do this by initializing an empty 3D array that contains the whole model and then determining for each pixel if it intersects with the geometry of the mesh you loaded.

Numpy Concatenate Images into Array

I have a bunch of images that I want to store into an array.
The problem is that all my images are different sizes and I don't want to necessarily change their size, because some will be square and some aren't.
I tried using np.concatenate but someone online said it was better to construct a zero matrix and fill it.
However, using
image = misc.imread(filename)
from the scipy library. The image is returned as a 3 dimensional array. How should I construct my numpy ndarray if I want to store all the images in it?

If I'm understanding the question correctly, you are trying to store a bunch of images of different sizes that are each stored as separate numpy arrays. If your images are gray scale (meaning 2D, as opposed to RGB which are 3D - a channel for R, G and B), you could store the images as the third dimension, filling in the absent pixels with 0s. But the best way would be to just use a python list (or tupple maybe) that stores a list of your numpy array images. That way they can be different sizes. i.e.: img_list = img1, img2, img3, etc.

storing them in a list may be easier, the list will store them as array() objects and size wont matter, when you do operations on them, just reference the list elements.

Image filtering with scikit-image?

I'm moving to python from a Matlab background, and there are a few elementary operations I've yet to conquer in Python/skimage:
How can I apply a user-generated linear filter (given as a small 2d array) to an image? I can do it with scipy.ndimage.convolve, but is there a method in skimage?
In Matlab, image filtering always returns a result of the same numeric type as its input, be it uint8 or float. Does skimage behave the same way?
Does skimage include unsharp masking somewhere? (I've found an unsharp masking filter in PIL but that's a bit of a pain, as PIL uses its own Image class, rather than ndarrays).
Is there a method, maybe similar to Matlab's "colfilt" by which a user can apply a non-linear filter to an image? The idea is that the user supplies a function which produces a single number from a 3x3 array, say; then that function is applied across the image as a spatial filter.

How can I apply a user-generated linear filter (given as a small 2d array) to an image? I can do it with scipy.ndimage.convolve, but is there a method in skimage?
The goal of scikit-image (and the scikits, in general) is to extend the functionality of scipy. (Smaller, more focused projects tend to evolve more rapidly than larger ones.) It tries not to duplicate any functionality, and only does so if it can improve upon that functionality.
In Matlab, image filtering always returns a result of the same numeric type as its input, be it uint8 or float. Does skimage behave the same way?
No, there is no such guarantee. Sometimes it's just more efficient to convert to a single type. (Sometimes, it's just a lack of time/man-power.) Here's some documentation on the matter:
http://scikit-image.org/docs/0.9.x/user_guide/data_types.html#output-types
There are convenient methods (e.g. img_as_float, img_as_ubyte) for transforming images if you need a certain type (and they check if the input type is the desired type, so you don't go wasting time with unnecessary conversion).
Does skimage include unsharp masking somewhere? (I've found an unsharp masking filter in PIL but that's a bit of a pain, as PIL uses its own Image class, rather than ndarrays).
Not that I know of, but you could roll your own. Something like the following would work:
from skimage import data
from skimage import filter
from skimage import img_as_float
import matplotlib.pyplot as plt
unsharp_strength = 0.8
blur_size = 8 # Standard deviation in pixels.
# Convert to float so that negatives don't cause problems
image = img_as_float(data.camera())
blurred = filter.gaussian_filter(image, blur_size)
highpass = image - unsharp_strength * blurred
sharp = image + highpass
fig, axes = plt.subplots(ncols=2)
axes[0].imshow(image, vmin=0, vmax=1)
axes[1].imshow(sharp, vmin=0, vmax=1)
plt.show()
There are, however, many ways to implement unsharp masking.
Is there a method, maybe similar to Matlab's "colfilt" by which a user can apply a non-linear filter to an image? The idea is that the user supplies a function which produces a single number from a 3x3 array, say; then that function is applied across the image as a spatial filter.
Not in scikit-image, but there's generic filtering capability in scipy.ndimage:
https://docs.scipy.org/doc/scipy-0.19.0/reference/generated/scipy.ndimage.generic_filter.html