Related
This question already has answers here:
List of lists changes reflected across sublists unexpectedly
(17 answers)
Closed 3 years ago.
I'm using python to illustrate DP algorithm. During memoization which keeps updating the table, I found that the table was not updated correctly.
I've narrowed down to the line "dp[t_id][_p] = max(choice_1, choice_2)", and I don't see what's the problem there.
tasks = [1,2,2,3]
dp = [[0]*(p+1)]*len(tasks)
for t_id in range(len(dp)):
for _p in range(p+1):
choice_1 = 1
choice_2 = 2
print dp
dp[t_id][_p] = max(choice_1, choice_2)
print dp
I expect the dp table should be updated one cell at a time, whereas dp[0][0] = 2, then dp[0][1] = 2, etc. However, it's updating as dp[every_column][0] = 2. The two prints in code should showcase the issue.
screenshot
dp = [[0]*(p+1)]*len(tasks)
This creates a list of references to the same list. Let's do a simpler example:
dp = [[0] * 5] * 5
dp[0][0] = 42
print(dp)
Output:
[[42, 0, 0, 0, 0], [42, 0, 0, 0, 0], [42, 0, 0, 0, 0], [42, 0, 0, 0, 0], [42, 0, 0, 0, 0]]
As you can see, it looks like the first element of each row is set to 42. This is because each row is the same list. Instead, you need to create several independent lists with a list comprehension or a for loop. For example:
dp = [[0] * 5 for _ in range(6)]
dp[0][0] = 42
print(dp)
Output:
[[42, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0]]
Now only the first element of the first row is set.
Suppose, I have a numpy vector with n elements, so I'd like to encode numbers in this vector as a binary notation, so resulting shape will be (n,m) where m is log2(maxnumber) for example:
x = numpy.array([32,5,67])
Because max number I have is 67, I need numpy.ceil(numpy.log2(67)) == 7 bits to encode this vector, so shape of the result will be (3,7)
array([[1, 0, 0, 0, 0, 1, 1],
[0, 0, 0, 0, 1, 0, 1],
[0, 1, 0, 0, 0, 0, 0]])
The problem rises because I have no quick way to move binary notation from
function numpy.binary_repr to numpy array. Now I have to iterate over result, and put each bit severally:
brepr = numpy.binary_repr(x[i],width=7)
j = 0
for bin in brepr:
X[i][j] = bin
j += 1
It's very timecost and stupid way, how to make it efficient?
Here is one way using np.unpackbits and broadcasting:
>>> max_size = np.ceil(np.log2(x.max())).astype(int)
>>> np.unpackbits(x[:,None].astype(np.uint8), axis=1)[:,-max_size:]
array([[0, 1, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 1, 0, 1],
[1, 0, 0, 0, 0, 1, 1]], dtype=uint8)
You can use numpy byte string.
For the case you have in hand:
res = numpy.array(len(x),dtype='S7')
for i in range(len(x)):
res[i] = numpy.binary_repr(x[i])
Or more compactly
res = numpy.array([numpy.binary_repr(val) for val in x])
I'm trying to make a nested list in Python to contain information about points in a video, and I'm having a lot of trouble creating an array for the results to be saved in to. The structure of the list is simple: the top level is a reference to the frame, the next level is a reference to a marker, and the last level is the point of the marker. So for example, the list is setup as such:
markerList # a long list of every marker in every frame
markerList[0] # every marker in the first frame
markerList[0][0] # the first marker of the first frame
markerList[0][0][0] # the x value of the first marker of the first frame
Calling markerList[0] looks like this:
array([[ 922.04443359, 903. ],
[ 987.83850098, 891.38830566],
[ 843.27374268, 891.70471191],
[ 936.38446045, 873.34661865],
[ 965.52880859, 840.44445801],
[ 822.19567871, 834.06298828],
[ 903.48956299, 830.62268066],
[ 938.70031738, 825.71557617],
[ 853.09545898, 824.47247314],
[ 817.84277344, 816.05029297],
[ 1057.91186523, 815.52935791],
[ 833.23632812, 787.48504639],
[ 924.24224854, 755.53997803],
[ 836.07800293, 720.02764893],
[ 937.83880615, 714.11199951],
[ 813.3493042 , 720.30566406],
[ 797.09521484, 705.72729492],
[ 964.31713867, 703.246521 ],
[ 934.9864502 , 697.27099609],
[ 815.1550293 , 688.91473389],
[ 954.94085693, 685.88171387],
[ 797.70239258, 672.35119629],
[ 877.05749512, 659.94250488],
[ 962.24786377, 659.26495361],
[ 843.66131592, 618.83868408],
[ 901.50476074, 585.42541504],
[ 863.41851807, 584.4977417 ]], dtype=float32)
The problem is that every frame contains a different number of markers. I want to create an empty array the same length as markerList (i.e., the same number of frames) in which every element is the same size as the largest frame in markerList. Some important caveats: first,I want to save the results into a .mat file where the final array (which I'll call finalStack) is a cell of cells. Second, I need to be able to reference and assign to any specific part of finalStack. So if I want to move a point to finalStack[0][22], I need to be able to do so without conflict. This basically just means I can't use append methods anywhere, but it's also unearthed my first problem - finding a way to create finalStack that doesn't cause every new assignment to be duplicated throughout the entire parent list. I've tried to do this a few different ways, and none work correctly.
Attempts at a solution:
Following another SO question, I attempted to create finalStack iteratively, but to no avail. I created the following function:
def createFinalStack(numMarkers, numPoints, frames):
step = [[0]*numPoints for x in xrange(numMarkers)]
finalStack = [step]*frames
return finalStack
However, this causes all assignments to be copied across the parent list, such that assigning finalStack[0][12] leads to finalStack[2][12] == finalStack[20][12] == finalStack[0][12]. In this example, numMarkers= 40, numPoints = 2 (just x & y), and frames= 200. (So the final array should be 200 x 40 x 2.)
That said, this seems like the most straightforward way to do what I want, I just can't get past the copy error (I know it's a reference issue, I just don't know how to avoid it in this context).
Another seemingly simple solution would be to copy markerList using copy.deepcopy(markerList), and pad any frames with less than 40 markers to get them to numMarkers = 40, and zero out anything else. But I can't come up with a good way to cycle through all of the frames, add points in the correct format, and then empty out everything else.
If this isn't enough information to work with, I can try to provide greater context and some other not-good-methods that didn't work at all. I've been stuck on this long enough that I'm convinced the solution is horribly simple, and I'm just missing the obvious. I hope you can prove me right!
Thanks!
This illustrates what is going on:
In [1334]: step=[[0]*3 for x in range(3)]
In [1335]: step
Out[1335]: [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
In [1336]: stack=[step]*4
In [1337]: stack
Out[1337]:
[[[0, 0, 0], [0, 0, 0], [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]]
In [1338]: stack[0]
Out[1338]: [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
In [1339]: stack[0][2]=3
In [1340]: stack
Out[1340]:
[[[0, 0, 0], [0, 0, 0], 3],
[[0, 0, 0], [0, 0, 0], 3],
[[0, 0, 0], [0, 0, 0], 3],
[[0, 0, 0], [0, 0, 0], 3]]
In [1341]: step
Out[1341]: [[0, 0, 0], [0, 0, 0], 3]
When you use alist*n to create new list, the new list contains multiple pointers to the same underlying object. As a general rule, using *n to replicate a list is dangerous if you plan on changing values later on.
If instead I make an array of the right dimensions I don't have this problem:
In [1342]: np.zeros((4,3,3),int)
Out[1342]:
array([[[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
...
[0, 0, 0]]])
Or in list form:
In [1343]: np.zeros((4,3,3),int).tolist()
Out[1343]:
[[[0, 0, 0], [0, 0, 0], [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]]
If I assign a value in this list, I only change one item:
In [1344]: stack=np.zeros((4,3,3),int).tolist()
In [1345]: stack[0][2]=3
In [1346]: stack
Out[1346]:
[[[0, 0, 0], [0, 0, 0], 3],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]]
I really should have used stack[0][2][1]=3, but you get the idea. If I make the same assignment in the array form I end up changing a whole row
In [1347]: stack=np.zeros((4,3,3),int)
In [1348]: stack[0][2]=3
In [1349]: stack
Out[1349]:
array([[[0, 0, 0],
[0, 0, 0],
[3, 3, 3]],
[[0, 0, 0],
...
[0, 0, 0]]])
I should have used an expression like stack[0,2,:]=4.
It's probably possible to construct a triply next list like this where all initial values are independent. But this array approach is simpler.
I've been given the pseudo-code:
for i= 1 to 3
for j = 1 to 3
board [i] [j] = 0
next j
next i
How would I create this in python?
(The idea is to create a 3 by 3 array with all of the elements set to 0 using a for loop).
If you really want to use for-loops:
>>> board = []
>>> for i in range(3):
... board.append([])
... for j in range(3):
... board[i].append(0)
...
>>> board
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]
But Python makes this easier for you:
>>> board = [[0]*3 for _ in range(3)]
>>> board
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]
arr=[[0,0,0] for i in range(3)] # create a list with 3 sublists containing [0,0,0]
arr
Out[1]: [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
If you want an list with 5 sublists containing 4 0's:
In [29]: arr=[[0,0,0,0] for i in range(5)]
In [30]: arr
Out[30]:
[[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]]
The range specifies how many sublists you want, ranges start at 0, so ranges 4 is 0,1,2,3,4.
gives you five [0,0,0,0]
Using the list comprehension is the same as:
arr=[]
for i in range(5):
arr.append([0,0,0,0])
arr
[[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]]
If you want something closer to your pseudocode:
board = []
for i in range(3):
board.append([])
for j in range(3):
board[i].append(0)
numpy has something for this:
numpy.zeros((3,3))
You can use the style of pseudocode given or simply just use a python one liner
chess_board = [[x]*3 for _ in range(y)] --> list comprehension
or you can use the plain loop style of other languages like java. I prefer the one liner as it looks much nicer and cleaner.
I have this list:
row = [1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
I need to then shuffle or randomize the list:
shuffle(row)
And then I need to go through and find any adjacent 1's and move them so that they are separated by at least one 0. For example I need the result to look like this:
row = [0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0]
I am not sure of what the most efficient way to go about searching for adjacent 1's and then moving them so that they aren't adjacent is... I will also being doing this repeatedly to come up with multiple combinations of this row.
Originally when the list was shorter I did it this way:
row = [1, 1, 1, 0, 0, 0, 0, 0, 0, 0]
rowlist = set(list(permutations(row)))
rowschemes = [(0, 0) + x for x in rowlist if '1, 1' not in str(x)]
But now that my row is 20 elements long this takes forever to come up with all the possible permutations.
Is there an efficient way to go about this?
I had a moderately clever partition-based approach in mind, but since you said there are always 20 numbers and 6 1s, and 6 is a pretty small number, you can construct all the possible locations (38760) and toss the ones which are invalid. Then you can uniformly draw from those, and build the resulting row:
import random
from itertools import combinations
def is_valid(locs):
return all(y-x >= 2 for x,y in zip(locs, locs[1:]))
def fill_from(size, locs):
locs = set(locs)
return [int(i in locs) for i in range(size)]
and then
>>> size = 20
>>> num_on = 6
>>> on_locs = list(filter(is_valid, combinations(range(size), num_on)))
>>> len(on_locs)
5005
>>> fill_from(size, random.choice(on_locs))
[0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1]
>>> fill_from(size, random.choice(on_locs))
[0, 1, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1]
>>> fill_from(size, random.choice(on_locs))
[1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1]
Why not go directly for what you want? Something like:
row = ["0","0","0","0","0","0","0","0","0","01","01","01","01","01","01"]
random.shuffle(row)
print (map(int, list("".join(row)[1:])))
Since the number of 1's is fixed in a row and you don't want any 1's to be adjacent, let m be the number of 1's and let k be the number of 0's of the row. Then you want to place the m 1's in (k+1) locations randomly so that there is at most one 1 in each location. This amounts to choosing a random subset of size ((k+1) choose m) from the set (1,2,...,k+1). This is easy to do. Given the random choice of subset, you can construct your random arrangement of 0's and 1's so that no two 1's are adjacent. The random choice algorithm takes O(m) time.
Place the 6 1's and 5 of the 0's in a list giving
row = [1,0,1,0,1,0,1,0,1,0,1]
Then insert the remaining 0's one by one at random positions in the (growing) list.
for i in range(11,19):
row.insert(random.randint(0,i), 0)