Saw a question on another site about a piece of Python code that was driving someone nuts. It was a fairly small, straightforward-looking piece of code, so I looked at it, figured out what it was trying to do, then ran it on my local system, and discovered why it was driving the original questioner nuts. Hoping that someone here can help me understand what's going on.
The code seems to be a straightforward "ask the user for three values (x,y,z) and a sum (n); iterate all values to find tuples that sum to n, and add those tuples to a list." solution. But what it outputs is, instead of all tuples that sum to n, a list of tuples the count of which is equal to the count of tuples that sum to n, but the contents of which are all "[x,y,z]". Trying to wrap my head around this, I changed the append call to an extend call (knowing that this would un-list the added tuples), to see if the behavior changed at all. I expected to get the same output, just as "x,y,z,x,y,z..." repeatedly, instead of "[x,y,z],[x,y,z]" repeatedly, because as I read and understand the Python documentation, that's the difference between append and extend on lists. What I got instead when I used extend was the correct values of the tuples that summed to n, just broken out of their tuple form by extend.
Here's the problem code:
my = []
x = 3
y = 5
z = 7
n = 11
part = [0,0,0]
for i in range(x+1):
part[0] = i
for j in range(y+1):
part[1] = j
for k in range(z+1):
part[2] = k
if sum(part) == n:
my.append(part)
print(my)
and the output:
[[3, 5, 7], [3, 5, 7], [3, 5, 7], [3, 5, 7], [3, 5, 7], [3, 5, 7], [3, 5, 7], [3, 5, 7], [3, 5, 7], [3, 5, 7], [3, 5, 7], [3, 5, 7], [3, 5, 7], [3, 5, 7]]
And here's the extend output:
[0, 4, 7, 0, 5, 6, 1, 3, 7, 1, 4, 6, 1, 5, 5, 2, 2, 7, 2, 3, 6, 2, 4, 5, 2, 5, 4, 3, 1, 7, 3, 2, 6, 3, 3, 5, 3, 4, 4, 3, 5, 3]
And the extend code:
my = []
x = 3
y = 5
z = 7
n = 11
part = [0,0,0]
for i in range(x+1):
part[0] = i
for j in range(y+1):
part[1] = j
for k in range(z+1):
part[2] = k
if sum(part) == n:
my.extend(part)
print(my)
Any light that could be shed on this would be greatly appreciated. I've dug around for a while on Google and several Q&A sites, and the only things that I found regarding Python append/extend deltas are things that don't seem to have any relevance to this issue.
{edit: environment detail}
Also, ran this in both Python 2.7.10 and Python 3.4.3 (cygwin, under Windows 10 home) with the same results.
extend adds items from the parameter list to the list object making the call. More like, dump objects from one list to another without emptying the former.
append on the other hand, just appends; nothing more. Therefore, appending a list object to another list with an existing reference to the appended list could do some damage - as in this case. After the list has been appended, part still holds a reference to the list (since you're modifying in place), so you're essentially modifying and (re-)appending the same list object every time.
You can prevent this by either building a new list at the start of each parent iteration of the append case.
Or by simply appending a copy of the part list:
my.append(part[:])
my.append(list(part))
my.append(part.copy()) # Python 3 only
This will append a list that has no other existing reference outside its new parent list.
There are a couple of things going on - the difference between append and extend, and the mutability of a list.
Consider a simpler case:
In [320]: part=[0,0,0]
In [321]: alist=[]
In [322]: alist.append(part)
In [323]: alist
Out[323]: [[0, 0, 0]]
The append actually put a pointer to part in the list.
In [324]: alist.extend(part)
In [325]: alist
Out[325]: [[0, 0, 0], 0, 0, 0]
extend put the elements of part in the list, not part itself.
If we change an element in part, we can see the consequences of this difference:
In [326]: part[1]=1
In [327]: alist
Out[327]: [[0, 1, 0], 0, 0, 0]
The append part also changed, but the extended part did not.
That's why your append case consists of sublists, and the sublists all have the final value of part - because they all are part.
The extend puts the current values of part in the list. Not only aren't they sublists, but they don't change as part changes.
Here's a variation on that list pointer issue:
In [333]: alist = [part]*3
In [334]: alist
Out[334]: [[0, 1, 0], [0, 1, 0], [0, 1, 0]]
In [335]: alist[0][0]=2
In [336]: part
Out[336]: [2, 1, 0]
In [337]: alist
Out[337]: [[2, 1, 0], [2, 1, 0], [2, 1, 0]]
alist contains 3 pointers to part (not 3 copies). Change one of those sublists, and we change them all, including part.
Related
Hi everyone I am new to Python and have been trying out how to create lists within lists containing multiples from 1 to 4, with each of them containing the same number of multiples as the number they are being multiplied with. In particular, applying the use of lambda and map function.
[[0],
[0, 2],
[0, 3, 6],
[0, 4, 8, 12],
[0, 5, 10, 15, 20]]
I have tried several ways and this is the code that I am currently working on, but I have received NameError: name 'x' is not defined error. I will really be grateful if someone can point out the mistakes that I have overlooked!
list_1 = [[0], [0,1], [0,1,2]]
result = list(map(lambda x, i ,j ,k: x[i][j] * k, list_1, range(3),items in x[i], range(1,3)))
print(result)
You get the NameError because one of your parameters to the map is items in x[i], and at this point none of those things are defined. Keep in mind that a lambda is equivalent to a function definition; x and i are the function's names for the parameters that are passed in - they aren't available to be used in defining the other parameters, because you're doing this work outside that scope. (As for items, it doesn't appear anywhere else in your code, so of course it isn't defined.)
Fundamentally, none of this logic makes sense. In particular: since you are trying to produce a nested data structure, you should expect nested map calls as well.
Take this in smaller steps. Try first to write the code that produces e.g. [0, 4, 8, 12] given 4 as an input. Then you can write code that repeatedly calls that to make the list of lists that you want.
I don't know if using map and lambda is a constraint in your exercise, but you
don't need them, by the way
>>> [[i*j for j in range(i)] for i in range(1,6)]
[[0], [0, 2], [0, 3, 6], [0, 4, 8, 12], [0, 5, 10, 15, 20]]
Your current code is too complex for this problem. Instead, use a list comprehension and a single nested map:
new_result = [list(map(lambda x:i*x, range(i))) for i in range(1, 6)]
Output:
[[0], [0, 2], [0, 3, 6], [0, 4, 8, 12], [0, 5, 10, 15, 20]]
If you want in pure functional programming as you said then you can try this :
print(list(map(lambda x:list(map(lambda z:z*x,list(range(x)))),range(1,6))))
output:
[[0], [0, 2], [0, 3, 6], [0, 4, 8, 12], [0, 5, 10, 15, 20]]
Need to make a 6x8 matrix. In this matrix, need to assign values to various cells within the matrix and then simulate a heating/cooling system. Before I get there though, I need to make sure this is right. Is this how you make rows and columns? Does it matter that it does not display this way when printed? Like I said I need to assign values to each of theses cells, does it matter that some already have a value by the way I made the lists? Is there a way to make the list without any initial values?
matrix = [] # Create an empty list
numberOfRows = 6
numberOfColumns = 8
for row in range(0, numberOfRows):
matrix.append([]) # Add an empty new row
for column in range(0, numberOfColumns):
matrix[row].append(column)
print(matrix)
[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]
Yes, it is a good method. However, it is much easier and more pythonic to use list comprehension : matrix = [list(range(numberOfColumns)) for _ in range(numberOfRows)]
And yes, you can make a list with no value : [] or list()
And even a 2-dimensional list : [[]]
However, it has no use.
You can use this way to make 2D arrays, but a few comments:
You may consider initializing the elements to 0 (or -1) by changing the last line to matrix[row].append(0).
This is mostly a design choice. For example, if, in your program, later on you do not change the values at every position, then you will be left with old values from the initialization.
You can re-write range(0, numberOfRows) to range(numberOfRows); it starts from 0 by default.
List comprehension, as Labo has already mentioned
Suppose I have this list:
newlis = [[3, 6, 4, 10], [1, 9, 2, 5], [0, 7, 8]]
I want to sort it in a way that each list is sorted. For instance:
newlis = [[3, 4, 6, 10], [1, 2, 5, 9], [0, 7, 8]]
I tried to write this code:
for i in range(len(newlis)):
if j in newlis[i] < newlis[i+1]:
newlis[i],newlis[i+1]=newlis[i+1],newlis[i]
print newlis
It's not working though. Can someone please help me out? Built in function are not allowed.
There are many things wrong here (among which is that this sounds like a homework question and we aren't supposed to respond to those) but I will give you some helpful advice:
You are comparing element J in list I to list I + 1.
You would want to compare element J in list I to element J + 1 in list I.
Also, you appear to be attempting to sort backwards. You will end up with large left and small right.
Also this is not a sorting algorithm. What happens when you have an array like
[3,6,4,10] => [6,4,10,3]
which is still not ordered, at all. Sorting algorithms are simple, but not that simple. I recommend looking them up.
In if j in newlis[i] < newlis[i+1]:, you are comparing sublists and not the elements of the sublists itself. You need two loops, one for iterating on newlis, and one for sorting the elements of each sublist of newlis.
A sample using Bubble Sort:
You can test it here:
>>> newlis = [[3, 6, 4, 10], [1, 9, 2, 5], [0, 7, 8]]
>>> for sublist in newlis:
... for i in range(len(sublist) - 1):
... if sublist[i] > sublist[i + 1]:
... sublist[i], sublist[i + 1] = sublist[i + 1], sublist[i]
>>> print(newlis)
[[3, 4, 6, 10], [1, 2, 5, 9], [0, 7, 8]]
Links about Bubble Sort:
http://www-ee.eng.hawaii.edu/~tep/EE160/Book/chap10/subsection2.1.2.2.html
http://www.go4expert.com/articles/bubble-sort-algorithm-absolute-beginners-t27883/
I'm relatively new to programming, and I want to sort a 2-D array (lists as they're called in Python) by the value of all the items in each sub-array. For example:
pop = [[1,5,3],[1,1,1],[7,5,8],[2,5,4]]
The sum of the first element of pop would be 9, because 1 + 5 + 3 = 9. The sum of the second would be 3, because 1 + 1 + 1 = 3, and so on.
I want to rearrange this so the new order would be:
newPop = [pop[1], pop[0], pop[3], pop[2]]
How would I do this?
Note: I don't want to sort the elements each sub-array, but sort according to the sum of all the numbers in each sub-array.
You can use sorted():
>>> pop = [[1,5,3],[1,1,1],[7,5,8],[2,5,4]]
>>> newPop = sorted(pop, key=sum)
>>> newPop
[[1, 1, 1], [1, 5, 3], [2, 5, 4], [7, 5, 8]]
You can also sort in-place with pop.sort(key=sum). Unless you definitely want to preserve the original list, you should prefer in-pace sorting.
Try this:
sorted(pop, key=sum)
Explanation:
The sorted() procedure sorts an iterable (a list in this case) in ascending order
Optionally, a key parameter can be passed to determine what property of the elements in the list is going to be used for sorting
In this case, the property is the sum of each of the elements (which are sublists)
So essentially this is what's happening:
[[1,5,3], [1,1,1], [7,5,8], [2,5,4]] # original list
[sum([1,5,3]), sum([1,1,1]), sum([7,5,8]), sum([2,5,4])] # key=sum
[9, 3, 20, 11] # apply key
sorted([9, 3, 20, 11]) # sort
[3, 9, 11, 20] # sorted
[[1,1,1], [1,5,3], [2,5,4], [7,5,8]] # elements coresponding to keys
#arshajii beat me to the punch, and his answer is good. However, if you would prefer an in-place sort:
>>> pop = [[1,5,3],[1,1,1],[7,5,8],[2,5,4]]
>>> pop.sort(key=sum)
>>> pop
[[1, 1, 1], [1, 5, 3], [2, 5, 4], [7, 5, 8]]
I have to look up Python's sorting algorithm -- I think it's called Timsort, bit I'm pretty sure an in-place sort would be less memory intensive and about the same speed.
Edit: As per this answer, I would definitely recommend x.sort()
If you wanted to sort the lists in a less traditional way, you could write your own function (that takes one parameter.) At risk of starting a flame war, I would heavily advise against lambda.
For example, if you wanted the first number to be weighted more heavily than the second number more heavily than the third number, etc:
>>> def weightedSum(listToSum):
... ws = 0
... weight = len(listToSum)
... for i in listToSum:
... ws += i * weight
... weight -= 1
... return ws
...
>>> weightedSum([1, 2, 3])
10
>>> 1 * 3 + 2 * 2 + 3 * 1
10
>>> pop
[[1, 5, 3], [1, 1, 1], [7, 5, 8], [2, 5, 4]]
>>> pop.sort(key=weightedSum)
>>> pop
[[1, 1, 1], [1, 5, 3], [2, 5, 4], [7, 5, 8]]
>>> pop += [[1, 3, 8]]
>>> pop.sort(key=weightedSum)
>>> pop
[[1, 1, 1], [1, 5, 3], [1, 3, 8], [2, 5, 4], [7, 5, 8]]
I want to merge two arrays in python based on the first element in each column of each array.
For example,
A = ([[1, 2, 3],
[4, 5, 6],
[4, 6, 7],
[5, 7, 8],
[5, 9, 1]])
B = ([[1, .002],
[4, .005],
[5, .006]])
So that I get an array
C = ([[1, 2, 3, .002],
[4, 5, 6, .005],
[4, 6, 7, .005],
[5, 7, 8, .006],
[5, 9, 1, .006]])
For more clarity:
First column in A is 1, 4, 4, 5, 5 and
First column of B is 1, 4, 5
So that 1 in A matches up with 1 in B and gets .002
How would I do this in python? Any suggestions would be great.
Is it Ok to modify A in place?:
d = dict((x[0],x[1:]) for x in B)
Now d is a dictionary where the first column are keys and the subsequent columns are values.
for lst in A:
if lst[0] in d: #Is the first value something that we can extend?
lst.extend(d[lst[0]])
print A
To do it out of place (inspired by the answer by Ashwini):
d = dict((x[0],x[1:]) for x in B)
C = [lst + d.get(lst[0],[]) for lst in A]
However, with this approach, you need to have lists in both A and B. If you have some lists and some tuples it'll fail (although it could be worked around if you needed to), but it will complicate the code slightly.
with either of these answers, B can have an arbitrary number of columns
As a side note on style: I would write the lists as:
A = [[1, 2, 3],
[4, 5, 6],
[4, 6, 7],
[5, 7, 8],
[5, 9, 1]]
Where I've dropped the parenthesis ... They make it look too much like you're putting a list in a tuple. Python's automatic line continuation happens with parenthesis (), square brackets [] or braces {}.
(This answer assumes these are just regular lists. If they’re NumPy arrays, you have more options.)
It looks like you want to use B as a lookup table to find values to add to each row of A.
I would start by making a dictionary out of the data in B. As it happens, B is already in just the right form to be passed to the dict() builtin:
B_dict = dict(B)
Then you just need to build C row by row.
For each row in A, row[0] is the first element, so B_dict[row[0]] is the value you want to add to the end of the row. Therefore row + [B_dict[row[0]] is the row you want to add to C.
Here is a list comprehension that builds C from A and B_dict.
C = [row + [B_dict[row[0]]] for row in A]
You can convert B to a dictionary first, with the first element of each sublist as key and second one as value.
Then simply iterate over A and append the related value fetched from the dict.
In [114]: A = ([1, 2, 3],
[4, 5, 6],
[4, 6, 7],
[5, 7, 8],
[6, 9, 1])
In [115]: B = ([1, .002],
[4, .005],
[5, .006])
In [116]: [x + [dic[x[0]]] if x[0] in dic else [] for x in A]
Out[116]:
[[1, 2, 3, 0.002],
[4, 5, 6, 0.005],
[4, 6, 7, 0.005],
[5, 7, 8, 0.006],
[6, 9, 1]]
Here is a solution using itertools.product() that prevents having to create a dictionary for B:
In [1]: from itertools import product
In [2]: [lst_a + lst_b[1:] for (lst_a, lst_b) in product(A, B) if lst_a[0] == lst_b[0]]
Out[2]:
[[1, 2, 3, 0.002],
[4, 5, 6, 0.005],
[4, 6, 7, 0.005],
[5, 7, 8, 0.006],
[5, 9, 1, 0.006]]
The naive, simple way:
for alist in A:
for blist in B:
if blist[0] == alist[0]:
alist.extend(blist[1:])
# alist.append(blist[1]) if B will only ever contain 2-tuples.
break # Remove this if you want to append more than one.
The downside here is that it's O(N^2) complexity. For most small data sets, that should be ok. If you're looking for something more comprehensive, you'll probably want to look at #mgilson's answer. Some comparison:
His response converts everything in B to a dict and performs list slicing on each element. If you have a lot of values in B, that could be expensive. This uses the existing lists (you're only looking at the first value, anyway).
Because he's using dicts, he gets O(1) lookup times (his answer also assumes that you're never going to append multiple values to the end of the values in A). That means overall, his algorithm will achieve O(N). You'll need to weigh whether the overhead of creating a dict is going to outweight the iteration of the values in B.