I have an input tensor as follow:
a = tf.constant([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
and the 'multiple' tensor:
mul= tf.constant([1, 3, 2])
Is it possible to tile a 3D tensor with the first element of a appears once, the second appears 3 times, the last element appears twice?
result = [
[[1, 2, 3]],
[[4, 5, 6],[4, 5, 6],[4, 5, 6]],
[[7, 8, 9], [7, 8, 9]]
]
Tensorflow 0.12
Thank you very much.
No, this is not possible. Read about tensors and shapes from the docs.
To understand why it is not possible imagine the matrix that has different number of elements in each row. It will clearly be not a matrix.
You can use numpy
import numpy as np
import tensorflow as tf
a = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
mul = np.array([1,3,2])
result = []
for i in range(len(mul)):
result.append(np.tile(a[i], (mul[i], 1)))
result = np.array(result)
I am sure you cannot have non-rectangular tensors in tensorflow. That is what is causing the problem. Otherwise I just extended the code of #Kris to run wholly on tensorflow.
import tensorflow as tf
sess = tf.InteractiveSession()
a = tf.constant([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
mul = tf.constant([1,3,2])
result = []
for i in range(3):
result.append(tf.tile(a[i],[mul[i]]))
print([r.eval() for r in result])
#r_tensor = tf.stack(0,[r for r in result]) # Not possible
Related
I have an n-by-m matrix X and an n-by-r index matrix I. I am wondering what are the relevant TensorFlow operators that allow me to get an n-by-r matrix R such that R[i,j] = X[i,I[i,j]]. As an example, let's say
X = tf.constant([[1,2,3],
[4,5,6],
[7,8,9]])
I = tf.constant([[1,2],
[1,0],
[0,2]])
The desired result would be a tensor
R = [[2, 3],
[5, 4],
[7, 9]]
I tried to use each column of the matrix I as the index and do tf.diag_part(tf.gather(X', index)), which seems to give me one column of R if I has the same number of row as X. For example,
idx = tf.transpose(I)[0] #[1,1,0]
res = tf.diag_part(tf.gather(tf.transpose(X), idx))
# res will be [2,5,7], i,e, first colum of R
Another attempt:
res = tf.transpose(tf.gather(tf.transpose(X), I),[0,2,1])
print(res.eval())
array([[[2, 3],
[5, 6],
[8, 9]],
[[2, 1],
[5, 4],
[8, 7]],
[[3, 1],
[6, 4],
[7, 9]]], dtype=int32)
From here i just need to be able to select the "diagonal entries" res[0,0], res[1,1] and res[2,2] to get R. I get stuck here though...
Use tf.gather with batch_dims argument:
res = tf.gather(X, I, batch_dims=1)
I need to create a 2-D numpy array using only list comprehension, but it has to follow the following format:
[[1, 2, 3],
[2, 3, 4],
[3, 4, 5],
[4, 5, 6],
[5, 6, 7]]]
So far, all I've managed to figure out is:
two_d_array = np.array([[x+1 for x in range(3)] for y in range(5)])
Giving:
array([[1, 2, 3],
[1, 2, 3],
[1, 2, 3],
[1, 2, 3],
[1, 2, 3]])
Just not very sure how to change the incrementation. Any help would be appreciated, thanks!
EDIT: Accidentally left out [3, 4, 5] in example. Included it now.
Here's a quick one-liner that will do the job:
np.array([np.arange(i, i+3) for i in range(1, 6)])
Where 3 is the number of columns, or elements in each array, and 6 is the number of iterations to perform - or in this case, the number of arrays to create; which is why there are 5 arrays in the output.
Output:
array([[1, 2, 3],
[2, 3, 4],
[3, 4, 5],
[4, 5, 6],
[5, 6, 7]])
Change the code, something like this can work:
two_d_array = np.array([[(y*3)+x+1 for x in range(3)] for y in range(5)])
>>> [[1,2,3],[4,5,6],...]
two_d_array = np.array([[y+x+1 for x in range(3)] for y in range(5)])
>>> [[1,2,3],[2,3,4],...]
You've got a couple of good comprehension answers, so here are a couple of numpy solutions.
Simple addition:
np.arange(1, 6)[:, None] + np.arange(3)
Crazy stride tricks:
base = np.arange(1, 8)
np.lib.stride_tricks.as_strided(base, shape=(5, 3), strides=base.strides * 2).copy()
Reshaped cumulative sum:
base = np.ones(15)
base[3::3] = -1
np.cumsum(base).reshape(5, 3)
I have a list of lists(2000x1000), but as example consider this one(10x3):
l = [[8, 7, 6], [5, 3, 1], [4, 5, 9], [1, 5, 1], [3, 5, 7], [8, 2, 5], [1, 9, 2], [8, 7, 6], [9, 9, 9], [4, 5, 9]]
In this example, every list corresponds to 3 measurements of each instant:
t0 -> [8,7,6]
t1 -> [5,3,1]
and so on.
I would like to compare the measurements with a window of 4 instants for position and take the max value which is in the 99 percentile of the peak to peak value.
EXAMPLE
Lets consider the first window:
[8, 7, 6], [5, 3, 1], [4, 5, 9], [1, 5, 1] :
[8,5,4,1] -> peak to peak: 8-1=7
[7,3,5,5] -> ptp=4
[6,1,9,1] -> ptp=8
with these 3 values [7,4,8] I want to take the max in the 99percentile, in this case 7
For the second window:
[5, 3, 1], [4, 5, 9], [1, 5, 1], [3, 5, 7]:
[5,4,1,3] -> ptp=4
[3,5,5,5] -> ptp=2
[1,9,1,7] -> ptp=8
max in 99percentile -> 4
After I do that for all the windows of size 4, I want to make a list with these values.
My code is this the following one, but it is slow. Is there a fast way to implement this?
NOTE: I cannot use pandas, and Numpy version should be <=1.6
num_meas = 4
m = []
for index, i in enumerate(l):
if index < len(l) - num_meas + 1:
p = []
for j in range(len(i)):
t = []
for k in range(num_meas):
t.append(l[index + k][j])
t = [x for x in t if ~np.isnan(x)]
try:
a = np.ptp(t)
except ValueError:
a = 0
p.append(a)
perce = np.percentile(p, 99)
p = max([el for el in p if el < perce])
m.append(p)
print m
The output:
[7, 4, 7, 6, 5, 7, 7]
Please check if the following code works with NumPy 1.6:
import numpy as np
l = [[8, 7, 6], [5, 3, 1], [4, 5, 9], [1, 5, 1], [3, 5, 7], [8, 2, 5],
[1, 9, 2], [8, 7, 6], [9, 9, 9], [4, 5, 9]]
l = np.array(l)
# range matrix
mat_ptp = np.zeros((l.shape[0]-3, l.shape[1]))
for i in range(l.shape[0]-3):
l[i:i+4].ptp(axis=0, out=mat_ptp[i])
percentiles = np.percentile(mat_ptp, 99, axis=1)
greater_pos = np.greater_equal(mat_ptp, percentiles.reshape(-1, 1))
mat_ptp[greater_pos] = -np.inf
result = np.max(mat_ptp, axis=1)
For enhancing performance, you can try to vetorize your operations as much as possible using numpy. It could be much faster than using for loops and the append function.
EDIT
Sorry, I didn't notice that you wanted the selected elements strictly less than the percentile. Here is the correct version.
BENCHMARK
Just to validate the question about performance, here is the result with:
l = np.random.randint(0, 100, size=(200, 100))
run 100 times with timeit:
OP code: 0.5197743272900698 ms in average
Code above: 0.0021439407201251015 in average
I have a 3 dimensional numpy array similar to this:
a = np.array([[[1, 2],
[3, 4]],
[[5, 6],
[7, 8]],
[[9, 10],
[11, 12]]])
What I'd like to do is intersperse each 2D array contained inside the outer array to produce this result:
t = np.array([[[1, 2], [5, 6], [9, 10]],
[[3, 4], [7, 8], [11, 12]]])
I could do this in Python like this, but I'm hoping there's a more efficient, numpy version:
t = np.empty((a.shape[1], a.shape[0], a.shape[2]), a.dtype)
for i, x in np.ndenumerate(a):
t[i[1], i[0], i[2]] = x
As #UdayrajDeshmukh said, you can use the transpose method (which, despite the name that evokes the "transpose" operator in linear algebra, is better understood as "permuting the axes"):
>>> t = a.transpose(1, 0, 2)
>>> t
array([[[ 1, 2],
[ 5, 6],
[ 9, 10]],
[[ 3, 4],
[ 7, 8],
[11, 12]]])
The newly created object t is a shallow array looking into a's data with a different permutation of indices. To replicate your own example, you need to copy it, e.g. t = a.transpose(1, 0, 2).copy()
Try the transpose function. You simply change the first two axes.
t = np.transpose(a, axes=(1, 0, 2))
I'm very new to python and I've become a bit unstuck.
I'm trying to run a for loop which adds 1 to every element of a 2x2 array. However, I want to store each matrix after each loop. I am using .append() to do this but it seems to only store the final result multiple times.
result = []
def f():
x= np.matrix([[1, 2],[3, 4]])
for i in range(5):
result.append(x)
x += 1
return result
function = f()
print function
But the output is:
[matrix([[6, 7],
[8, 9]]), matrix([[6, 7],
[8, 9]]), matrix([[6, 7],
[8, 9]]), matrix([[6, 7],
[8, 9]]), matrix([[6, 7],
[8, 9]])]'
Which is just a repeat of the last iteration instead of logging all the results.
Any help would be appreciated.
The reason that is happening is that in the backend, Python handles everything through pointers. Hence, whenever you are appending x, you are just appending a pointer to x. But, then, at each stage, you are just appending a pointer which points to the same location. Hence, they all point to a matrix which stores the final value of the matrix x. Instead, you should create a copy of the matrix by appending x.copy()`. The modified code would be :
result = []
def f():
x= np.matrix([[1, 2],[3, 4]])
for i in range(5):
result.append(x.copy())
x += 1
return result
function = f()
print(function)
When I try it on my Python 3, I get the output
[matrix([[1, 2],
[3, 4]]), matrix([[2, 3],
[4, 5]]), matrix([[3, 4],
[5, 6]]), matrix([[4, 5],
[6, 7]]), matrix([[5, 6],
[7, 8]])]
Try the following:
import numpy as np
result = []
def f():
x = np.matrix([[1, 2],[3, 4]])
for i in range(5):
result.append(x + i)
return result
res = f()
Result:
>>> res
[matrix([[1, 2],
[3, 4]]), matrix([[2, 3],
[4, 5]]), matrix([[3, 4],
[5, 6]]), matrix([[4, 5],
[6, 7]]), matrix([[5, 6],
[7, 8]])]