I'm trying to make an array of 4x4 (16) pixel black and white images with all possible combinations. I made the following array as a template:
template = [[0,0,0,0], # start with all white pixels
[0,0,0,0],
[0,0,0,0],
[0,0,0,0]]
I then want to iterate through the template and changing the 0 to 1 for every possible combination.
I tried to iterate with numpy and itertools but can only get 256 combinations, and with my calculations there should be 32000 (Edit: 65536! don't know what happened there...). Any one with mad skills that could help me out?
As you said, you can use the itertools module to do this, in particular the product function:
import itertools
import numpy as np
# generate all the combinations as string tuples of length 16
seq = itertools.product("01", repeat=16)
for s in seq:
# convert to numpy array and reshape to 4x4
arr = np.fromiter(s, np.int8).reshape(4, 4)
# do something with arr
You would have a total of 65536 such combinations of such a (4 x 4) shaped array. Here's a vectorized approach to generate all those combinations, to give us a (65536 x 4 x 4) shaped multi-dim array -
mask = ((np.arange(2**16)[:,None] & (1 << np.arange(16))) != 0)
out = mask.astype(int).reshape(-1,4,4)
Sample run -
In [145]: out.shape
Out[145]: (65536, 4, 4)
In [146]: out
Out[146]:
array([[[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]],
[[1, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]],
[[0, 1, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]],
...,
[[1, 0, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1]],
[[0, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1]],
[[1, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1]]])
One possibility which relies on a for loop
out = []
for i in range(2**16):
out.append(np.frombuffer("{:016b}".format(i).encode('utf8')).view(np.uint8).reshape(4,4)-48)
Obviously you could make that a list comprehension if you like.
It takes advantage of Python string formatting which is able to produce the binary representation of integers. The format string instructs it to use 16 places filling with zeros on the left. The string is then encoded to give a bytes object which numpy can interpret as an array.
In the end we subtract the code for the character "0" to get a proper 0. Luckily, "1" sits just above "0", so that's all we need to do.
First I'll iterate for all numbers from 0 to (2^16)-1. Then I'll create a 16 character binary string for each of those numbers and thus covering all possible combinations
After that I converted the string to a list and made a 2d list out of it using list comprehension and slicing.
all_combinations = []
for i in xrange(pow(2,16))
binary = '{0:016b}'.format(i) ## Converted number to binary string
binary = map(int,list(binary)) ## String to list ## list(map(int,list(binary))) in py 3
template = [binary[i:i+4] for i in xrange(0, len(binary), 4)] #created 2d list
all_combinations.append(template)
Related
Let's say, now I have a 1x1 matrix, like:
M = Matrix([[2]])
How can I create a new 2x2 matrix from this, filling the all blanks with 0s? Which is:
N = Matrix([[2, 0], [0, 0]])
If it were numpy, I could use np.newaxis; however, it seems that there is no newaxis in sympy.
So, I tried:
N = M.reshape(2, 2)
I got the following error:
ValueError: Invalid reshape parameters 2 2
I found that the following expression works:
N = Matrix(2, 2, [D[0], 0, 0, 0])
However, this is a bit awkward.
Is there any better way?
Please note that a scalar multiplication N = D[0] * Matrix(2, 2, [1, 0, 0, 0]) is not acceptable, since next time I may ask you to convert 2x2 to 3x3.
Use sympy.diag.
>>> import sympy as sp
>>> m = sp.Matrix([[2]])
>>> sp.diag(m, 0)
Matrix([
[2, 0],
[0, 0]])
>>> sp.diag(m, 0, 0)
Matrix([
[2, 0, 0],
[0, 0, 0],
[0, 0, 0]])
>>> sp.diag(sp.Matrix([[1, 2], [3, 4]]), 0)
Matrix([
[1, 2, 0],
[3, 4, 0],
[0, 0, 0]])
All, I have an application that requires returning a numpy ndarray, rather than a simple sum, when multiplying two matrices; e.g.:
import numpy as np
x = np.array([[1, 1, 0], [0, 1, 1]])
y = np.array([[1, 0, 0, 1], [1, 0, 1, 0], [0, 0, 0, 0]])
w = x # y
>>> array([[2, 0, 1, 1],
[1, 0, 1, 0]])
However, the requirement is to return an ndarray (in this case..):
array([[[1,1,0], [0,0,0], [0,1,0], [1,0,0]],
[[0,1,0], [0,0,0], [0,1,0], [0,0,0]]])
Note that the matrix multiplication operation may be repeated; the output will be used as the left-side matrix of ndarrays for the next matrix multiplication operation, which would yield a higher-order ndarray after the second matrix multiplication operation, etc..
Any way to achieve this? I've looked at overloading __add__, and __radd__ by subclassing np.ndarray as discussed here, but mostly got dimension incompatibility errors.
Ideas?
Update:
Addressing #Divakar's answer E.g., for chained operation, adding
z = np.array([[1, 1, 0], [0, 0, 0], [1, 0, 0], [0, 1, 0]])
s1 = x[...,None] * y
s2 = s1[...,None] * z
results in an undesired output.
I suspect the issue starts with s1, which in the case above returns s1.shape = (2,3,4). It should be (2,4,3) since [2x3][3x4] = [2x4], but we're not really summing here, just return an array of length 3.
Similarly, s2.shape should be (2,3,4,3), which [incidentally] it is, but with undesired output (it's not 'wrong', just not what we're looking for).
To elaborate, s1*z should be [2x4][4x3] = [2x3] matrix. Each element of the matrix is itself an ndarray, of [4x3] since we have 4 rows in z to multiply the elements in s1, and each element in s1 is itself 3 elements long (again, we're not arithmetically adding elements, but return ndarrays with the extended dimension being the row count in the R-matrix of the operation.
Ultimately, the desired output would be:
s2 = array([[[[1, 1, 0],
[0, 0, 0],
[0, 1, 0],
[0, 0, 0]],
[[1, 1, 0],
[0, 0, 0],
[0, 0, 0],
[1, 0, 0]],
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]]],
[[[0, 1, 0],
[0, 0, 0],
[0, 1, 0],
[0, 0, 0]],
[[0, 1, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]]]])
Extend them to 3D and leverage broadcasting -
x[:,None] * y.T
Or with np.einsum -
np.einsum('ij,jk->ikj',x,y)
Going by OP's comment and the quote from the question :
... matrix multiplication operation may be repeated; the output will
be used as the left-side matrix of ndarrays for the next matrix
multiplication operation, which would yield a higher-order ndarray
after the second matrix multiplication operation, etc..
It seems, we need to do something along these lines -
s1 = x[...,None] * y
s2 = s1[...,None] * z # and so on.
Though, the order of the axes would be different in this case, but it seems to be the simplest way to extend the solution to a generic number of incoming 2D arrays.
Following the edits in the question, seems like you are placing the incoming arrays from the first axis onwards for element-wise multiplication. So, if I got that right, you can swap axes to get the correct order, like so -
s1c = (x[...,None] * y).swapaxes(1,-1)
s2c = (s1c.swapaxes(1,-1)[...,None] * z).swapaxes(1,-1) # and so on.
If you are only interested in the final output, swap axes only at the final stage and skip those in the intermediate ones.
import numpy as np
data = np.array([[0, 0, 1, 1, 2, 2],
[1, 0, 0, 1, 2, 2],
[1, 0, 1, 0, 0, 0],
[1, 1, 0, 0, 2, 0]])
How can I do the followings?
Within 2 by 2 patch:
if any element is 2: put 2
if any element is 1: put 1
if all elements are 0: put 0
The expected result is:
np.array([[1, 1, 2],
[1, 1, 2]])
Using extract_patches from scikit-learn you can write this as follows (copy and paste-able code):
import numpy as np
from sklearn.feature_extraction.image import extract_patches
data = np.array([[0, 0, 1, 1, 2, 2],
[1, 0, 0, 1, 2, 2],
[1, 0, 1, 0, 0, 0],
[1, 1, 0, 0, 2, 0]])
patches = extract_patches(data, patch_shape=(2, 2), extraction_step=(2, 2))
output = patches.max(axis=-1).max(axis=-1)
Explanation: extract_patches gives you a view on patches of your array, of size patch_shape and lying on a grid of extraction_step. The result is a 4D array where the first two axes index the patch and the last two axes index the pixels within the patch. We then evaluate the maximum over the last two axes to obtain the maximum per patch.
EDIT This is actually very much related to this question
I'm not sure where you get your input from or where you are supposed to leave the output, but you can adapt this.
import numpy as np
data = np.array([[0, 0, 1, 1, 2, 2],
[1, 0, 0, 1, 2, 2],
[1, 0, 1, 0, 0, 0],
[1, 1, 0, 0, 2, 0]])
def patchValue(i,j):
return max([data[i][j],
data[i][j+1],
data[i+1][j],
data[i+1][j+1]])
result = np.array([[0, 0, 0],
[0, 0, 0]])
for (v,i) in enumerate(range(0,4,2)):
for (w,j) in enumerate(range(0,6,2)):
result[v][w] = patchValue(i,j)
print(result)
Here's a rather lengthy one-liner that relies solely on reshaping, transposes, and taking maximums along different axes. It is fairly fast too.
data.reshape((-1,2)).max(axis=1).reshape((data.shape[0],-1)).T.reshape((-1,2)).max(axis=1).reshape((data.shape[1]/2,data.shape[0]/2)).T
Essentially what this does is reshape to take the maximum in pairs of two horizontally, then shuffle things around again and take the maximum in pairs of two vertically, ultimately giving the maximum of each block of 4, matching your desired output.
If the original array is large, and performance is an issue, the loops can be pushed down to numpy C code by manipulating the shape and strides of the original array to create the windows that you are acting on:
import numpy as np
from numpy.lib.stride_tricks import as_strided
data = np.array([[0, 0, 1, 1, 2, 2],
[1, 0, 0, 1, 2, 2],
[1, 0, 1, 0, 0, 0],
[1, 1, 0, 0, 2, 0]])
patch_shape = (2,2)
data_shape = np.array(data.shape)
# transform data to a 2x3 array of 2x2 patches/windows
# final shape of the computation on the windows can be calculated with:
# tuple(((data_shape-patch_shape) // patch_shape) + 1)
final_shape = (2,3)
# the shape of the windowed array can be calculated with:
# final_shape + patch_shape
newshape = (2, 3, 2, 2)
# the strides of the windowed array can be calculated with:
# tuple(np.array(data.strides) * patch_shape) + data.strides
newstrides = (48, 8, 24, 4)
# use as_strided to 'transform' the array
patch_array = as_strided(data, shape = newshape, strides = newstrides)
# flatten the windowed array for iteration - dim of 6x2x2
# the number of windows is the product of the 'first' dimensions of the array
# which can be calculated with:
# (np.product(newshape[:-len(patch_shape)])) + (newshape[-len(patch_array):])
dim = (6,2,2)
patch_array = patch_array.reshape(dim)
# perfom computations on the windows and reshape to final dimensions
result = [2 if np.any(patch == 2) else
1 if np.any(patch == 1) else
0 for patch in patch_array]
result = np.array(result).reshape(final_shape)
A generalized 1-d function for creating the windowed array can be found at Efficient rolling statistics with NumPy
A generalised multi-dimension function and a nice explanation can be found at Efficient Overlapping Windows with Numpy
I am trying to generate a numpy array of length 100 randomly filled with sets of 5 1s and 0s as such:
[ [1,1,1,1,1] , [0,0,0,0,0] , [0,0,0,0,0] ... [1,1,1,1,1], [0,0,0,0,0] ]
Essentially there should be a 50% chance that at each position there will be 5 1s and a 50% chance there will be 5 0's
Currently, I have been messing about with numpy.random.binomial(), and tried running:
numpy.random.binomial(1, .5 , (100,5))
but this creates an array as such:
[ [0,1,0,0,1] , [0,1,1,1,0] , [1,1,0,0,1] ... ]
I need each each set of elements to be consistent and not random. How can I do this?
Use numpy.random.randint to generate a random column of 100 1s and 0s, then use tile to repeat the column 5 times:
>>> numpy.tile(numpy.random.randint(0, 2, size=(100, 1)), 5)
array([[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[1, 1, 1, 1, 1],
[0, 0, 0, 0, 0],
[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1],
[0, 0, 0, 0, 0],
...
You want to create a temporary array of zeros and ones, and then randomly index into that array to create a new array. In the code below, the first line creates an array whose 0'th row contains all zeros and whose 1st row contains all ones. The function randint returns a random sequence of zeros and ones, which can be used as indices into the temporary array.
import numpy as np
...
def make_coded_array(n, k=5):
_ = np.array([[0]*k,[1]*k])
return _[np.random.randint(2, size=500)]
import numpy as np
import random
c = random.sample([[0,0,0,0,0],[1,1,1,1,1]], 1)
for i in range(99):
c = np.append(c, random.sample([[0,0,0,0,0],[1,1,1,1,1]], 1))
Not the most efficient way though
Use numpy.ones and numpy.random.binomial
>>> numpy.ones((100, 5), dtype=numpy.int64) * numpy.random.binomial(1, .5, (100, 1))
array([[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1],
...
I have a sparse matrix (numpy.array) and I would like to have the index of the nonzero elements in it.
In Matlab I would write:
[i, j] = find(CM)
and in Python what should I do?
I have tried numpy.nonzero (but I don't know how to take the indices from that) and flatnonzero (but it's not convenient for me, I need both the row and column index).
Thanks in advance!
Assuming that by "sparse matrix" you don't actually mean a scipy.sparse matrix, but merely a numpy.ndarray with relatively few nonzero entries, then I think nonzero is exactly what you're looking for. Starting from an array:
>>> a = (np.random.random((5,5)) < 0.10)*1
>>> a
array([[0, 0, 0, 0, 0],
[0, 0, 0, 0, 1],
[0, 0, 1, 0, 0],
[1, 0, 0, 0, 0],
[0, 0, 0, 0, 0]])
nonzero returns the indices (here x and y) where the nonzero entries live:
>>> a.nonzero()
(array([1, 2, 3]), array([4, 2, 0]))
We can assign these to i and j:
>>> i, j = a.nonzero()
We can also use them to index back into a, which should give us only 1s:
>>> a[i,j]
array([1, 1, 1])
We can even modify a using these indices:
>>> a[i,j] = 2
>>> a
array([[0, 0, 0, 0, 0],
[0, 0, 0, 0, 2],
[0, 0, 2, 0, 0],
[2, 0, 0, 0, 0],
[0, 0, 0, 0, 0]])
If you want a combined array from the indices, you can do that too:
>>> np.array(a.nonzero()).T
array([[1, 4],
[2, 2],
[3, 0]])
(there are lots of ways to do this reshaping; I chose one almost at random.)
This goes slightly beyond what you as and I only mention it since I once faced a similar problem. If you want the indices to access some other array there is some very simple sytax:
import numpy as np
array = np.random.randint(0, 2, size=(3, 3))
data = np.random.random(size=(3, 3))
Now array looks something like
>>> print array
array([[0, 1, 0],
[1, 0, 1],
[1, 1, 0]])
while data could be
>>> print data
array([[ 0.92824816, 0.43605604, 0.16627849],
[ 0.00301434, 0.94342538, 0.95297402],
[ 0.32665135, 0.03504204, 0.86902492]])
Then if we want the elements of data which are zero:
>>> print data[array==0]
array([ 0.92824816, 0.16627849, 0.94342538, 0.86902492])
Which is nice and simple.