add field to memmaped numpy record array - python

With normal memmapped numpy arrays, you can "add" a new column by opening the memmap file with an additional column in the shape.
k = np.memmap('input', dtype='int32', shape=(10, 2), mode='r+', order='F')
k[:] = 1
l = np.memmap('input', dtype='int32', shape=(10, 3), mode='r+', order='F')
print(k)
print(l)
[[1 1]
[1 1]
[1 1]
[1 1]
[1 1]
[1 1]
[1 1]
[1 1]
[1 1]
[1 1]]
[[1 1 0]
[1 1 0]
[1 1 0]
[1 1 0]
[1 1 0]
[1 1 0]
[1 1 0]
[1 1 0]
[1 1 0]
[1 1 0]]
Is it possible to make a similar move with record arrays? Seems possible with rows, but can't find a way to do so with a new field, if the dtype has heterogeneous types.

Related

How to save iterations in a loop as follows?

I use two for loop to go through two arrays. I have set the conditions to print me only those that meet my conditions.
the individual positions in are in the array comb_x and comb_y. this results in conditions in the for loop
This is fine and works as it should.
This my code
import os
import numpy as np
from itertools import combinations
a=np.array([0, 0, 0, 1, 1, 1, 2, 2, 2])
b=np.array([0, 1, 2, 0, 1, 2, 0, 1, 2])
z1=np.array([1, 1])
z2=np.array([1, 1])
comb_x=np.array([0, 0, 1, 1])
comb_y=np.array([0, 1, 0, 1])
for (j), (k) in zip(a,b):
z1[:]=0
z1[:j]=1
x12=z1
z2[:]=0
z2[:k]=1
y12=z2
#print(x12,y12)
for (h),(n),(r) in zip(comb_x,comb_y,np.arange(0,4)):
if x12[h]==1 and y12[n]==1:
print('pravda',x12,y12)
My output sorts those numbers by values
my output:
[1 0] [1 0]
[1 0] [1 1]
[1 0] [1 1]
[1 1] [1 0]
[1 1] [1 0]
[1 1] [1 1]
[1 1] [1 1]
[1 1] [1 1]
[1 1] [1 1]
I need them to appear as well as follow in each iteration
required output
[1 0] [1 0]
[1 0] [1 1]
[1 1] [1 0]
[1 1] [1 1]
[1 0] [1 1]
[1 1] [1 1]
[1 1] [1 0]
[1 1] [1 1]
[1 1] [1 1]
To make it easier to see, I'll show below what the individual iterations and their results should look like
first iteration:
[1 0] [1 0]
[1 0] [1 1]
[1 1] [1 0]
[1 1] [1 1]
second iteration:
[1 0] [1 1]
[1 1] [1 1]
third iteration:
[1 1] [1 0]
[1 1] [1 1]
fourth iteration:
[1 1] [1 1]
But as you can see, it does not save them by iteration, but sorts them in ascending order by value
Could it be arranged according to iterations so that it is still a numpy array?

How to check some elements of a matrix and generate a set of matrices accordingly?

I'm trying to check elements of a 2D array (matrix) and generate a number of matrices (of equal size) depending on some conditions as below:
Consider my matrix:
x = [[1, 0, 2],[7, 0, 7],[1, 1, 1]]
I need to check for the (2) and generate two matrices where the position of (2) will be replaced by 0 and 1 respectively. I also need to check for the 7's and generate 3 combinations of the matrix with values of 7 being (0,1),(1,0),(1,1) respectively. This mean the total number of matrices generated are 6 as follows:
[[1, 0, 0],[0, 0, 1],[1, 1, 1]]
[[1, 0, 0],[1, 0, 0],[1, 1, 1]]
[[1, 0, 0],[1, 0, 1],[1, 1, 1]]
[[1, 0, 1],[0, 0, 1],[1, 1, 1]]
[[1, 0, 1],[1, 0, 0],[1, 1, 1]]
[[1, 0, 1],[1, 0, 1],[1, 1, 1]]
There can be more than 1 (2), and the position of 7's can be vertical or hirizental.
I've tried a naiive way just looping through x looking for 2's and appending:
for i in range(len(x)):
for j in range(len(x[0])):
if x[i][j] == 2:
inter[i][j] = 0
test.append(inter)
inter2[i][j] = 1
test.append(inter2)
But that only works if I have the value of 2 only. I've also tried converting to numpy array and using where() to find the indexes of 2's and 7's, but then don't know how that can be used to generate the required outcome. Any thoughts?
The conditions described are very vague. If I understand correctly, you want this:
sevens = [[0,1],[1,0],[1,1]]
twos = [0,1]
for i in twos:
for j in sevens:
m = x.copy()
m[m==2] = i
m[m==7] = j
print(m)
output:
[[1 0 0]
[0 0 1]
[1 1 1]]
[[1 0 0]
[1 0 0]
[1 1 1]]
[[1 0 0]
[1 0 1]
[1 1 1]]
[[1 0 1]
[0 0 1]
[1 1 1]]
[[1 0 1]
[1 0 0]
[1 1 1]]
[[1 0 1]
[1 0 1]
[1 1 1]]
UPDATE: per OP's comment for (2)s multiplicities:
x = np.array([[2, 0, 2],[7, 0, 7],[1, 1, 1]])
sevens = [[0,1],[1,0],[1,1]]
v = (x==2).sum()*([0,1],)
twos = np.array(np.meshgrid(*v)).T.reshape(-1,2)
for i in twos:
for j in sevens:
m = x.copy()
m[m==2] = i
m[m==7] = j
print(m)
output:
[[0 0 0]
[0 0 1]
[1 1 1]]
[[0 0 0]
[1 0 0]
[1 1 1]]
[[0 0 0]
[1 0 1]
[1 1 1]]
[[0 0 1]
[0 0 1]
[1 1 1]]
[[0 0 1]
[1 0 0]
[1 1 1]]
[[0 0 1]
[1 0 1]
[1 1 1]]
[[1 0 0]
[0 0 1]
[1 1 1]]
[[1 0 0]
[1 0 0]
[1 1 1]]
[[1 0 0]
[1 0 1]
[1 1 1]]
[[1 0 1]
[0 0 1]
[1 1 1]]
[[1 0 1]
[1 0 0]
[1 1 1]]
[[1 0 1]
[1 0 1]
[1 1 1]]

Numpy.array indexing

import numpy as np
arr = np.array([[0, 1, 0],
[1, 0, 0],
[1, 0, 0]])
mask = arr
print('boolean mask is:')
print(mask)
print('arr[mask] is:')
print(arr[mask])
Result:
boolean mask is:
[[0 1 0]
[1 0 0]
[1 0 0]]
arr[mask] is:
[[[0 1 0]
[1 0 0]
[0 1 0]]
[[1 0 0]
[0 1 0]
[0 1 0]]
[[1 0 0]
[0 1 0]
[0 1 0]]]
I know how indexing works when the mask is 2-D, but confused when the mask is 3-D.
Anyone can explain it?
import numpy as np
l = [[0,1,2],[3,5,4],[7,8,9]]
arr = np.array(l)
mask = arr[:,:] > 5
print(mask) # shows boolean results
print(mask.sum()) # shows how many items are > 5
print(arr[:,1]) # slicing
print(arr[:,2]) # slicing
print(arr[:, 0:3]) # slicing
output
[[False False False]
[False False False]
[ True True True]]
3
[1 5 8]
[2 4 9]
[[0 1 2]
[3 5 4]
[7 8 9]]

How to scan through tensor not at dimension 0?

The tensorflow document states that tf.scan scans on the list of tensors unpacked from elems on dimension 0.
The simplest version of scan repeatedly applies the callable fn to a sequence of elements from first to last. The elements are made of the tensors unpacked from elems on dimension 0.
My question is:
How to scan on the list of tensors on other dimension instead of dimension 0?
For example,
I have a tensor, ref, defined as below.
>>> ref = tf.Variable(tf.ones([2,3,3],tf.int32))
....
>>> print(ref.eval())
[[[1 1 1]
[1 1 1]
[1 1 1]]
[[1 1 1]
[1 1 1]
[1 1 1]]]
I want to scan through the ref[1,0], ref[1,1], ref[1,2] and apply a function to each of the, ,say add 1.
That is to say, I want ref be after the operation
>>> print(ref.eval())
[[[1 1 1]
[1 1 1]
[1 1 1]]
[[2 2 2]
[2 2 2]
[2 2 2]]]
Can I use tf.scan to do that? If yes, how?
If not, any how to do in other way?
Thanks.
You can tf.transpose() your tensor before the tf.scan() and transpose back after.
Also, if you want the variable ref to contain the new value after the operation, you need to tf.assign() the value back.
Please note your desired example value is achieved just by applying a straight-up scan, with no transpose, however. Refer to this code (tested) (see another example with transpose at the bottom of answer):
import tensorflow as tf
ref = tf.Variable(tf.ones([2,3,3],tf.int32))
ref2 = tf.scan( lambda y, x: x + 1, ref )
with tf.Session() as sess:
sess.run( tf.global_variables_initializer() )
print ( sess.run( ref2 ) ) # ref2 is calculated
print ( "====================")
print ( sess.run( ref ) ) # the original ref is unchanged
print ( "====================")
sess.run( tf.assign( ref, ref2 ) ) # assign the value back to ref
print ( sess.run( ref ) )
outputs:
[[[1 1 1]
[1 1 1]
[1 1 1]]
[[2 2 2]
[2 2 2]
[2 2 2]]]
====================
[[[1 1 1]
[1 1 1]
[1 1 1]]
[[1 1 1]
[1 1 1]
[1 1 1]]]
====================
[[[1 1 1]
[1 1 1]
[1 1 1]]
[[2 2 2]
[2 2 2]
[2 2 2]]]
If you use tf.transpose(), you can achieve a scan along any dimension:
import tensorflow as tf
ref = tf.Variable(tf.ones([2,3,3],tf.int32))
ref2 = tf.transpose( tf.scan( lambda y, x: x + 1,
tf.transpose( ref, [ 1, 0, 2 ] ) ),
[ 1, 0, 2 ] )
with tf.Session() as sess:
sess.run( tf.global_variables_initializer() )
sess.run( tf.assign( ref, ref2 ) )
print ( sess.run( ref ) )
will output:
[[[1 1 1]
[2 2 2]
[2 2 2]]
[[1 1 1]
[2 2 2]
[2 2 2]]]

python: Find subsets coordinate is in

I have a set of coordinates and try to find those subsets the coordinate is in.
import numpy as np
a=np.array([[[0,1,1],[1,1,1]],[[0,1,1],[2,1,1]],[[3,3,3],[2,2,2]]])
If I try things like:
print(np.argwhere(a==[[0,1,1]]))
print(a[[0,1,1]])
print(np.isin([0,1,1],a))
I get:
[[0 0 0]
[0 0 1]
[0 0 2]
[0 1 1]
[0 1 2]
[1 0 0]
[1 0 1]
[1 0 2]
[1 1 1]
[1 1 2]]
[[[0 1 1]
[1 1 1]]
[[0 1 1]
[2 1 1]]
[[0 1 1]
[2 1 1]]]
[ True True True]
But I expect something like:
[true,true,false]
EDIT
The best case would be If I get an array where only all other coordinates which are members of the founded subsets are in like:
out = [[1,1,1],[2,1,1]]
Use all(-1) to assert the array equal at the last axis and then any(1) to check if such condition exists at the second axis:
(a == [0,1,1]).all(-1).any(1)
# array([ True, True, False], dtype=bool)
On the update:
mask = (a == [0,1,1]).all(-1)
a[mask.any(1)[:,None] & ~mask]
#array([[1, 1, 1],
# [2, 1, 1]])
I got the results you're looking for by doing this:
[[0,1,1] in b for b in a]
I'll try to figure out why isin didnt work.

Categories