Numpy has this helper function, np.empty, which will:
Return a new array of given shape and type, without initializing entries.
I find it pretty useful when I want to create a tensor using tf.concat since:
The number of dimensions of the input tensors must match, and all dimensions except axis must be equal.
So it comes in handy to start with an empty tensor of an expected shape. Is there any way to achieve this in tensorflow?
[edit]
A simplified example of why I want this
netInput = np.empty([0, 4])
netTarget = np.empty([0, 4])
inputWidth = 2
for step in range(data.shape.as_list()[-2]-frames_width-1):
netInput = tf.concat([netInput, data[0, step:step + frames_width, :]], -2)
target = tf.concat([target, data[0, step + frames_width + 1:step + frames_width + 2, :]], -2)
In this example, if netInput or netTarget are initialized, I'll be concatenating an extra example with that initialization. And to initialize them with the first value, I need to hack the loop. Nothing mayor, I just wondered if there is a 'tensorflow' way to solve this.
In TF 2,
tensor = tf.reshape(tf.convert_to_tensor(()), (0, n))
worked for me.
If you're creating an empty tensor, tf.zeros will do
>>> a = tf.zeros([0, 4])
>>> tf.concat([a, [[1, 2, 3, 4], [5, 6, 7, 8]]], axis=0)
<tf.Tensor: shape=(2, 4), dtype=float32, numpy=
array([[1., 2., 3., 4.],
[5., 6., 7., 8.]], dtype=float32)>
The closest thing you can do is create a variable that you do not initialize. If you use tf.global_variables_initializer() to initialize your variables, disable putting your variable in the list of global variables during initialization by setting collections=[].
For example,
import numpy as np
import tensorflow as tf
x = tf.Variable(np.empty((2, 3), dtype=np.float32), collections=[])
y = tf.Variable(np.empty((2, 3), dtype=np.float32))
sess = tf.InteractiveSession()
tf.global_variables_initializer().run()
# y has been initialized with the content of "np.empty"
y.eval()
# x is not initialized, you have to do it yourself later
x.eval()
Here np.empty is provided to x only to specify its shape and type, not for initialization.
Now for operations such as tf.concat, you actually don't have (indeed cannot) manage the memory yourself -- you cannot preallocate the output as some numpy functions allow you to. Tensorflow already manages memory and does smart tricks such as reusing memory block for the output if it detects it can do so.
Related
I have a numpy array of integers. In my code I need to append some other integers from a list, which works fine and gives me back an array of dtype int64 as expected. But it may happen that the list of integers to append is empty. In that case, numpy returns an array of float64 values. Exemplary code below:
import numpy as np
a = np.arange(10, dtype='int64')
np.append(a, [10]) # dtype is int64
# array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
np.append(a, []) # dtype is float64
# array([0., 1., 2., 3., 4., 5., 6., 7., 8., 9.])
Is this expected behaviour? If so, what is the rationale behind this? Could this even be a bug?
The documentation for np.append states that the return value is
A copy of arr with values appended to axis.
Since there are no values to append, shouldn't it just return a copy of the array?
(Numpy version: 1.22.4, Python version: 3.8.0)
The default dtype for a numpy array constructed from an empty list is float64:
>>> np.array([])
array([], dtype=float64)
A float64 dtype will "win" over the int64 dtype and promote everything to float64, because converting the other way around would cause loss of precision (i.e. truncate any float64). Of course this is an extreme case because there is no value to truncate in an empty array, but the check is only done on the dtype. More info on this is in the doc for numpy.result_type().
In fact:
>>> a = np.array(dtype='int64')
>>> a
array([], dtype=int64)
>>> b = np.array([])
>>> b
array([], dtype=float64)
>>> np.result_type(a, b)
dtype('float64')
The np.promote_types() function is used to determine the type to promote to:
>>> np.promote_types('int64', 'float64')
dtype('float64')
See also: How the dtype of numpy array is calculated internally?
You can cast your array when appending and specify the type you want:
import numpy as np
a = np.arange(10, dtype="int64")
# array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=int64)
np.append(a, [])
array([0., 1., 2., 3., 4., 5., 6., 7., 8., 9.], dtype=float64)
np.append(a, np.array([], dtype=a.dtype))
# array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=int64)
In the source code for numpy.append, we can see that it calls numpy.concatenate. Looking at the documentation for this method, we see that it accepts two type-related arguments: dtype and casting, the default of which is None. Since numpy.append does not provide values for either of these, the defaults are assumed. The default for casting is same-kind, which allows for safe casting within a kind, as described here. Since all of this is done in C, my guess is that concatenate tried to guess the type of the array you provided. Since it was empty, however, no type could be assumed so it took the widest possible type that would fit both inputs and assigned that to the output.
If you want to avoid this, you should probably append using numpy arrays instead of Python lists:
np.append(a, np.array([], dtype = 'int64'))
Can you avoid appending the the empty list with some sort of if statement?
I think its expected behaviour as [] is still a value that gets appended. Obviously [] can't be an integer so it probably assumes its a float (default casting).
I converted a PyTorch Tensor into a NumPy ndarray, and as it's shown below 'a' and 'b' both have different ids so they point to different memory locations.
I know in PyTorch underscore indicates in-place operation so a.add_(1) does not change the memory location of a, but why does it change the content of the array b (due to the difference in id)?
import torch
a = torch.ones(5)
b = a.numpy()
a.add_(1)
print(id(a), id(b))
print(a)
print(b)
Results:
139851293933696 139851293925904
tensor([2., 2., 2., 2., 2.])
[2. 2. 2. 2. 2.]
PyTorch documentation:
Converting a torch Tensor to a NumPy array and vice versa is a breeze.
The torch Tensor and NumPy array will share their underlying memory
locations, and changing one will change the other.
But why?
(They have different IDs so must be independent) :(
If you use numpy.ndarray.__array_interface__ and torch.Tensor.data_ptr(), you can find the memory location of their data_array.
a = torch.ones(5)
b = a.numpy()
a.add_(1)
print(b.__array_interface__)
# > {'data': (1848226926144, False),
# 'strides': None,
# 'descr': [('', '<f4')],
# 'typestr': '<f4',
# 'shape': (5,),
# 'version': 3}
print(a.data_ptr())
# > 1848226926144
a.data_ptr() == b.__array_interface__["data"][0]
# > True
Both the array and the tensor share the same data_array.
We can reasonably conclude that the method torch.Tensor.numpy() creates a numpy array from the Tensor by passing it the reference to its data_array.
b[0] = 4
print(a)
# > tensor([4., 2., 2., 2., 2.])
My guess is b is a pointer to a, and the memory addresses id() returns are just where they are stored in memory. Try using copy.deepcopy() to create a from b and that should make them entirely separate.
I have a problem with a numpy array.
In particular, suppose to have a matrix
x = np.array([[1., 2., 3.], [4., 5., 6.]])
with shape (2,3), I want to convert the float numbers into list so to obtain the array [[[1.], [2.], [3.]], [[4.], [5.], [6.]]] with shape (2,3,1).
I tried to convert each float number to a list (i.e., x[0][0] = [x[0][0]]) but it does not work.
Can anyone help me? Thanks
What you want is adding another dimension to your numpy array. One way of doing it is using reshape:
x = x.reshape(2,3,1)
output:
[[[1.]
[2.]
[3.]]
[[4.]
[5.]
[6.]]]
There is a function in Numpy to perform exactly what #Valdi_Bo mentions. You can use np.expand_dims and add a new dimension along axis 2, as follows:
x = np.expand_dims(x, axis=2)
Refer:
np.expand_dims
Actually, you want to add a dimension (not level).
To do it, run:
result = x[...,np.newaxis]
Its shape is just (2, 3, 1).
Or save the result back under x.
You are trying to add a new dimension to the numpy array. There are multiple ways of doing this as other answers mentioned np.expand_dims, np.new_axis, np.reshape etc. But I usually use the following as I find it the most readable, especially when you are working with vectorizing multiple tensors and complex operations involving broadcasting (check this Bounty question that I solved with this method).
x[:,:,None].shape
(2,3,1)
x[None,:,None,:,None].shape
(1,2,1,3,1)
Well, maybe this is an overkill for the array you have, but definitely the most efficient solution is to use np.lib.stride_tricks.as_strided. This way no data is copied.
import numpy as np
x = np.array([[1., 2., 3.], [4., 5., 6.]])
newshape = x.shape[:-1] + (x.shape[-1], 1)
newstrides = x.strides + x.strides[-1:]
a = np.lib.stride_tricks.as_strided(x, shape=newshape, strides=newstrides)
results in:
array([[[1.],
[2.],
[3.]],
[[4.],
[5.],
[6.]]])
>>> a.shape
(2, 3, 1)
I want to convert a list of tuples into a numpy array. For example:
items = [(1, 2), (3, 4)]
using np.asarray(items) I get:
array([[1, 2],
[3, 4]])
but if I try to append the items individually:
new_array = np.empty(0)
for item in items:
new_array = np.append(new_array, item)
the new_array loses the original shape and becomes:
array([1., 2., 3., 4.])
I can get it to the shape I wanted using new_array.reshape(2, 2):
array([[1., 2.],
[3., 4.]])
but how would I get that shape without reshaping?
Firstly you need to provide a correct shape to the array so that numpy could understand how to interpret the values provided to the append method.
Then, to prevent automatic flattening, specify the axis you wish to append on.
This code does what you intended to do:
import numpy as np
items = [(1,2),(3,4)]
new_array = np.ndarray((0,2))
for item in items:
new_array = np.append(new_array, [item], axis=0)
print(new_array) # [[1. 2.]
# [3. 4.]]
If you have a list of tuples, and you've decided you hate the standard array constructors (np.array, np.asarray, etc, which, as #JohnZwinck pointed out are probably the best answer) for some reason, the most efficient approach would be to preallocate the entire array and then assign to it:
items = [(1, 2), (3, 4)]
arr = np.empty((len(items), len(items[0])))
arr[...] = items
Even if what you want is to grow an array over time, row-by-row, it has been shown through detailed timings that you're usually better off just allocating a whole new array and then copying over the old values.
So given the above arr, by this approach the most efficient way to append a row would be:
newitem = (5, 6)
oldarr = arr
arr = np.empty((oldarr.shape[0] + 1, *oldarr.shape[1:]))
arr[:-1,:] = oldarr
arr[-1,:] = newitem
In order to understand a tensor in tensorflow clearly, I need to have a clear understanding of how is the shape a tensor defined.
These are some examples from the tensorflow document:
3 # a rank 0 tensor; this is a scalar with shape []
[1. ,2., 3.] # a rank 1 tensor; this is a vector with shape [3]
[[1., 2., 3.], [4., 5., 6.]] # a rank 2 tensor; a matrix with shape [2, 3]
[[[1., 2., 3.]], [[7., 8., 9.]]] # a rank 3 tensor with shape [2, 1, 3]
Is the below understanding of mine correct:
In order to find the shape of the tensor, we start from the outermost list and count the number of elements (or lists) inside. This count makes the first dimension. We then repeat this procedure for the inner lists and find the next dimensions of the tensor.
Please correct me if I am wrong.
Yes, your understanding is correct. If you have a valid tensor, your algorithm will return you the correct dimensions of the tensor. You can write it in python in the following way
def get_shape(arr):
res = []
while isinstance((arr), list):
res.append(len(arr))
arr = arr[0]
return res
Notice that in case of the arbitrary value of arr, you also need to make sure that dimensions match ([[1, 2, 3], [4, 5]] is not a valid tensor)