I'm trying to make a nested list in Python to contain information about points in a video, and I'm having a lot of trouble creating an array for the results to be saved in to. The structure of the list is simple: the top level is a reference to the frame, the next level is a reference to a marker, and the last level is the point of the marker. So for example, the list is setup as such:
markerList # a long list of every marker in every frame
markerList[0] # every marker in the first frame
markerList[0][0] # the first marker of the first frame
markerList[0][0][0] # the x value of the first marker of the first frame
Calling markerList[0] looks like this:
array([[ 922.04443359, 903. ],
[ 987.83850098, 891.38830566],
[ 843.27374268, 891.70471191],
[ 936.38446045, 873.34661865],
[ 965.52880859, 840.44445801],
[ 822.19567871, 834.06298828],
[ 903.48956299, 830.62268066],
[ 938.70031738, 825.71557617],
[ 853.09545898, 824.47247314],
[ 817.84277344, 816.05029297],
[ 1057.91186523, 815.52935791],
[ 833.23632812, 787.48504639],
[ 924.24224854, 755.53997803],
[ 836.07800293, 720.02764893],
[ 937.83880615, 714.11199951],
[ 813.3493042 , 720.30566406],
[ 797.09521484, 705.72729492],
[ 964.31713867, 703.246521 ],
[ 934.9864502 , 697.27099609],
[ 815.1550293 , 688.91473389],
[ 954.94085693, 685.88171387],
[ 797.70239258, 672.35119629],
[ 877.05749512, 659.94250488],
[ 962.24786377, 659.26495361],
[ 843.66131592, 618.83868408],
[ 901.50476074, 585.42541504],
[ 863.41851807, 584.4977417 ]], dtype=float32)
The problem is that every frame contains a different number of markers. I want to create an empty array the same length as markerList (i.e., the same number of frames) in which every element is the same size as the largest frame in markerList. Some important caveats: first,I want to save the results into a .mat file where the final array (which I'll call finalStack) is a cell of cells. Second, I need to be able to reference and assign to any specific part of finalStack. So if I want to move a point to finalStack[0][22], I need to be able to do so without conflict. This basically just means I can't use append methods anywhere, but it's also unearthed my first problem - finding a way to create finalStack that doesn't cause every new assignment to be duplicated throughout the entire parent list. I've tried to do this a few different ways, and none work correctly.
Attempts at a solution:
Following another SO question, I attempted to create finalStack iteratively, but to no avail. I created the following function:
def createFinalStack(numMarkers, numPoints, frames):
step = [[0]*numPoints for x in xrange(numMarkers)]
finalStack = [step]*frames
return finalStack
However, this causes all assignments to be copied across the parent list, such that assigning finalStack[0][12] leads to finalStack[2][12] == finalStack[20][12] == finalStack[0][12]. In this example, numMarkers= 40, numPoints = 2 (just x & y), and frames= 200. (So the final array should be 200 x 40 x 2.)
That said, this seems like the most straightforward way to do what I want, I just can't get past the copy error (I know it's a reference issue, I just don't know how to avoid it in this context).
Another seemingly simple solution would be to copy markerList using copy.deepcopy(markerList), and pad any frames with less than 40 markers to get them to numMarkers = 40, and zero out anything else. But I can't come up with a good way to cycle through all of the frames, add points in the correct format, and then empty out everything else.
If this isn't enough information to work with, I can try to provide greater context and some other not-good-methods that didn't work at all. I've been stuck on this long enough that I'm convinced the solution is horribly simple, and I'm just missing the obvious. I hope you can prove me right!
Thanks!
This illustrates what is going on:
In [1334]: step=[[0]*3 for x in range(3)]
In [1335]: step
Out[1335]: [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
In [1336]: stack=[step]*4
In [1337]: stack
Out[1337]:
[[[0, 0, 0], [0, 0, 0], [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]]
In [1338]: stack[0]
Out[1338]: [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
In [1339]: stack[0][2]=3
In [1340]: stack
Out[1340]:
[[[0, 0, 0], [0, 0, 0], 3],
[[0, 0, 0], [0, 0, 0], 3],
[[0, 0, 0], [0, 0, 0], 3],
[[0, 0, 0], [0, 0, 0], 3]]
In [1341]: step
Out[1341]: [[0, 0, 0], [0, 0, 0], 3]
When you use alist*n to create new list, the new list contains multiple pointers to the same underlying object. As a general rule, using *n to replicate a list is dangerous if you plan on changing values later on.
If instead I make an array of the right dimensions I don't have this problem:
In [1342]: np.zeros((4,3,3),int)
Out[1342]:
array([[[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
...
[0, 0, 0]]])
Or in list form:
In [1343]: np.zeros((4,3,3),int).tolist()
Out[1343]:
[[[0, 0, 0], [0, 0, 0], [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]]
If I assign a value in this list, I only change one item:
In [1344]: stack=np.zeros((4,3,3),int).tolist()
In [1345]: stack[0][2]=3
In [1346]: stack
Out[1346]:
[[[0, 0, 0], [0, 0, 0], 3],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]]
I really should have used stack[0][2][1]=3, but you get the idea. If I make the same assignment in the array form I end up changing a whole row
In [1347]: stack=np.zeros((4,3,3),int)
In [1348]: stack[0][2]=3
In [1349]: stack
Out[1349]:
array([[[0, 0, 0],
[0, 0, 0],
[3, 3, 3]],
[[0, 0, 0],
...
[0, 0, 0]]])
I should have used an expression like stack[0,2,:]=4.
It's probably possible to construct a triply next list like this where all initial values are independent. But this array approach is simpler.
Related
I need to pick up the clicked item from a list, my input is like below:
item_arr=tf.constant([['item001', 'item010', 'item020', 'item030', 'item041', 'item051'],
['item101', 'item110', 'item120', 'item130', 'item140', 'item151']])
clicked_arr=tf.constant([[1, 0, 0, 0, 1, 1], [1, 0, 0, 0, 0, 1]])
the item_arr is batch data of items, and clicked_arr is clicked flag(1 is clicked, 0 is no clicked) corresponding to item_arr.
I hope to get the output that pick up the clicked items like that:
clicked_item_arr(for example, output shape is [2,4]):
[
['item001', 'item041', 'item051', 'item_placeholder'],
['item101', 'item151', 'item_placeholder', 'item_placeholder']
]
I also need to get to no-clicked items but the same solution of clicked items should be used.
I tried to use gather_nd and sparse_to_dense:
index_arr=tf.where(tf.equal(clicked_arr, 1))
>>> array([[0, 0],
[0, 4],
[0, 5],
[1, 0],
[1, 5]])
sparse_item_value= tf.gather_nd(item_arr, index_arr)
>>> array(['item001', 'item041', 'item051', 'item101', 'item151'],
dtype=object)
but I can't get the result I want, because I need sparse_indices like:
array([[0, 0],
[0, 1],
[0, 2],
[1, 0],
[1, 1]])
so that I can use:
tf.sparse_to_dense(
sparse_indices=sparse_indices,
sparse_values=sparse_item_value,
output_shape=[2, 10],
default_value='item_placeholder'
)
But I don't know how to get the sparse_indices. Hope you can give me some advice and thanks in advance.
This question already has answers here:
List of lists changes reflected across sublists unexpectedly
(17 answers)
Closed 2 years ago.
Let's take the following code
board = [[0,0,0],[0,0,0],[0,0,0]]
versionsOfBoard = []
versionsOfBoard.append(board); #this will keep the versions of board array
print(versionsOfBoard);
# output
# [[[0,0,0],[0,0,0],[0,0,0]]]
# later I change few values in the board and append 'board' to my versions array
board[0][0] = 1
versionsOfBoard.append(board)
print(versionsOfBoard)
# output
# [[[1, 0, 0], [0, 0, 0], [0, 0, 0]], [[1, 0, 0], [0, 0, 0], [0, 0, 0]]]
# if you notice here, the 0th index value is also changed
board[1][2] = 1
versionsOfBoard.append(board)
print(versionsOfBoard)
# output
# [[[1, 0, 0], [0, 0, 1], [0, 0, 0]], [[1, 0, 0], [0, 0, 1], [0, 0, 0]], [[1, 0, 0], [0, 0, 1], [0, 0, 0]]]
Every index value displays the latest state of 'board'. it is referencing the board.
I have tried with .append(board[:]) and .append(board.copy()). Both these options too do not solve the problem.
Any help would be highly appreciated
Try this
import copy
board = [[0,0,0],[0,0,0],[0,0,0]]
versionsOfBoard = []
versionsOfBoard.append(copy.deepcopy(board))
print(versionsOfBoard);
board[0][0] = 1
versionsOfBoard.append(copy.deepcopy(board))
print(versionsOfBoard)
board[1][2] = 1
versionsOfBoard.append(copy.deepcopy(board))
print(versionsOfBoard)
All, I have an application that requires returning a numpy ndarray, rather than a simple sum, when multiplying two matrices; e.g.:
import numpy as np
x = np.array([[1, 1, 0], [0, 1, 1]])
y = np.array([[1, 0, 0, 1], [1, 0, 1, 0], [0, 0, 0, 0]])
w = x # y
>>> array([[2, 0, 1, 1],
[1, 0, 1, 0]])
However, the requirement is to return an ndarray (in this case..):
array([[[1,1,0], [0,0,0], [0,1,0], [1,0,0]],
[[0,1,0], [0,0,0], [0,1,0], [0,0,0]]])
Note that the matrix multiplication operation may be repeated; the output will be used as the left-side matrix of ndarrays for the next matrix multiplication operation, which would yield a higher-order ndarray after the second matrix multiplication operation, etc..
Any way to achieve this? I've looked at overloading __add__, and __radd__ by subclassing np.ndarray as discussed here, but mostly got dimension incompatibility errors.
Ideas?
Update:
Addressing #Divakar's answer E.g., for chained operation, adding
z = np.array([[1, 1, 0], [0, 0, 0], [1, 0, 0], [0, 1, 0]])
s1 = x[...,None] * y
s2 = s1[...,None] * z
results in an undesired output.
I suspect the issue starts with s1, which in the case above returns s1.shape = (2,3,4). It should be (2,4,3) since [2x3][3x4] = [2x4], but we're not really summing here, just return an array of length 3.
Similarly, s2.shape should be (2,3,4,3), which [incidentally] it is, but with undesired output (it's not 'wrong', just not what we're looking for).
To elaborate, s1*z should be [2x4][4x3] = [2x3] matrix. Each element of the matrix is itself an ndarray, of [4x3] since we have 4 rows in z to multiply the elements in s1, and each element in s1 is itself 3 elements long (again, we're not arithmetically adding elements, but return ndarrays with the extended dimension being the row count in the R-matrix of the operation.
Ultimately, the desired output would be:
s2 = array([[[[1, 1, 0],
[0, 0, 0],
[0, 1, 0],
[0, 0, 0]],
[[1, 1, 0],
[0, 0, 0],
[0, 0, 0],
[1, 0, 0]],
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]]],
[[[0, 1, 0],
[0, 0, 0],
[0, 1, 0],
[0, 0, 0]],
[[0, 1, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]]]])
Extend them to 3D and leverage broadcasting -
x[:,None] * y.T
Or with np.einsum -
np.einsum('ij,jk->ikj',x,y)
Going by OP's comment and the quote from the question :
... matrix multiplication operation may be repeated; the output will
be used as the left-side matrix of ndarrays for the next matrix
multiplication operation, which would yield a higher-order ndarray
after the second matrix multiplication operation, etc..
It seems, we need to do something along these lines -
s1 = x[...,None] * y
s2 = s1[...,None] * z # and so on.
Though, the order of the axes would be different in this case, but it seems to be the simplest way to extend the solution to a generic number of incoming 2D arrays.
Following the edits in the question, seems like you are placing the incoming arrays from the first axis onwards for element-wise multiplication. So, if I got that right, you can swap axes to get the correct order, like so -
s1c = (x[...,None] * y).swapaxes(1,-1)
s2c = (s1c.swapaxes(1,-1)[...,None] * z).swapaxes(1,-1) # and so on.
If you are only interested in the final output, swap axes only at the final stage and skip those in the intermediate ones.
I am moving from c to python.
I want to initialize an array of different lengths: map((2,3,(5,6,7)))
I want each entry of an array np.zeros((2,3)) to be a list with different lengths.
I would like the python equivalent to the c:
for(ii=0; ii<N; ii++) {
dim[ii] = ii**2;
for(jj=0; jj<Z; jj++) {
map[jj][ii] = (long*) calloc(dim[ii], sizeof(long));
}
}
I've tried several things but none is what I need. I would like to access later to the data easily and with lists I haven't managed to.
Thanks!
I have managed to do it:
>>> x=[[0]*3 for count in range(2)]
>>> for ii in range(2):
... for jj in range(3):
... x[ii][jj]=[0]*(jj+1)**2
...
>>> x
[[[0], [0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0]], [[0], [0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0]]]
Although I'm sure there is a better way, Thank you!
I've been given the pseudo-code:
for i= 1 to 3
for j = 1 to 3
board [i] [j] = 0
next j
next i
How would I create this in python?
(The idea is to create a 3 by 3 array with all of the elements set to 0 using a for loop).
If you really want to use for-loops:
>>> board = []
>>> for i in range(3):
... board.append([])
... for j in range(3):
... board[i].append(0)
...
>>> board
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]
But Python makes this easier for you:
>>> board = [[0]*3 for _ in range(3)]
>>> board
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]
arr=[[0,0,0] for i in range(3)] # create a list with 3 sublists containing [0,0,0]
arr
Out[1]: [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
If you want an list with 5 sublists containing 4 0's:
In [29]: arr=[[0,0,0,0] for i in range(5)]
In [30]: arr
Out[30]:
[[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]]
The range specifies how many sublists you want, ranges start at 0, so ranges 4 is 0,1,2,3,4.
gives you five [0,0,0,0]
Using the list comprehension is the same as:
arr=[]
for i in range(5):
arr.append([0,0,0,0])
arr
[[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]]
If you want something closer to your pseudocode:
board = []
for i in range(3):
board.append([])
for j in range(3):
board[i].append(0)
numpy has something for this:
numpy.zeros((3,3))
You can use the style of pseudocode given or simply just use a python one liner
chess_board = [[x]*3 for _ in range(y)] --> list comprehension
or you can use the plain loop style of other languages like java. I prefer the one liner as it looks much nicer and cleaner.