Related
Why do these two operations (append() resp. +) give different results?
>>> c = [1, 2, 3]
>>> c
[1, 2, 3]
>>> c += c
>>> c
[1, 2, 3, 1, 2, 3]
>>> c = [1, 2, 3]
>>> c.append(c)
>>> c
[1, 2, 3, [...]]
>>>
In the last case there's actually an infinite recursion. c[-1] and c are the same. Why is it different with the + operation?
To explain "why":
The + operation adds the array elements to the original array. The array.append operation inserts the array (or any object) into the end of the original array, which results in a reference to self in that spot (hence the infinite recursion in your case with lists, though with arrays, you'd receive a type error).
The difference here is that the + operation acts specific when you add an array (it's overloaded like others, see this chapter on sequences) by concatenating the element. The append-method however does literally what you ask: append the object on the right-hand side that you give it (the array or any other object), instead of taking its elements.
An alternative
Use extend() if you want to use a function that acts similar to the + operator (as others have shown here as well). It's not wise to do the opposite: to try to mimic append with the + operator for lists (see my earlier link on why). More on lists below:
Lists
[edit] Several commenters have suggested that the question is about lists and not about arrays. The question has changed, though I should've included this earlier.
Most of the above about arrays also applies to lists:
The + operator concatenates two lists together. The operator will return a new list object.
List.append does not append one list with another, but appends a single object (which here is a list) at the end of your current list. Adding c to itself, therefore, leads to infinite recursion.
As with arrays, you can use List.extend to add extend a list with another list (or iterable). This will change your current list in situ, as opposed to +, which returns a new list.
Little history
For fun, a little history: the birth of the array module in Python in February 1993. it might surprise you, but arrays were added way after sequences and lists came into existence.
The concatenation operator + is a binary infix operator which, when applied to lists, returns a new list containing all the elements of each of its two operands. The list.append() method is a mutator on list which appends its single object argument (in your specific example the list c) to the subject list. In your example this results in c appending a reference to itself (hence the infinite recursion).
An alternative to '+' concatenation
The list.extend() method is also a mutator method which concatenates its sequence argument with the subject list. Specifically, it appends each of the elements of sequence in iteration order.
An aside
Being an operator, + returns the result of the expression as a new value. Being a non-chaining mutator method, list.extend() modifies the subject list in-place and returns nothing.
Arrays
I've added this due to the potential confusion which the Abel's answer above may cause by mixing the discussion of lists, sequences and arrays.
Arrays were added to Python after sequences and lists, as a more efficient way of storing arrays of integral data types. Do not confuse arrays with lists. They are not the same.
From the array docs:
Arrays are sequence types and behave very much like lists, except that the type of objects stored in them is constrained. The type is specified at object creation time by using a type code, which is a single character.
append is appending an element to a list. if you want to extend the list with the new list you need to use extend.
>>> c = [1, 2, 3]
>>> c.extend(c)
>>> c
[1, 2, 3, 1, 2, 3]
Python lists are heterogeneous that is the elements in the same list can be any type of object. The expression: c.append(c) appends the object c what ever it may be to the list. In the case it makes the list itself a member of the list.
The expression c += c adds two lists together and assigns the result to the variable c. The overloaded + operator is defined on lists to create a new list whose contents are the elements in the first list and the elements in the second list.
So these are really just different expressions used to do different things by design.
The method you're looking for is extend(). From the Python documentation:
list.append(x)
Add an item to the end of the list; equivalent to a[len(a):] = [x].
list.extend(L)
Extend the list by appending all the items in the given list; equivalent to a[len(a):] = L.
list.insert(i, x)
Insert an item at a given position. The first argument is the index of the element before which to insert, so a.insert(0, x) inserts at the front of the list, and a.insert(len(a), x) is equivalent to a.append(x).
you should use extend()
>>> c=[1,2,3]
>>> c.extend(c)
>>> c
[1, 2, 3, 1, 2, 3]
other info: append vs. extend
See the documentation:
list.append(x)
Add an item to the end of the list; equivalent to a[len(a):] = [x].
list.extend(L)
- Extend the list by appending all the items in the given list;
equivalent to a[len(a):] = L.
c.append(c) "appends" c to itself as an element. Since a list is a reference type, this creates a recursive data structure.
c += c is equivalent to extend(c), which appends the elements of c to c.
This question already has answers here:
How to change values in a tuple?
(17 answers)
Closed 1 year ago.
I want to subtract 1 from every digit in a tuple using for loop in Python.
I don't understand why my code is returning the original tuple instead of the desired one. Here is the code:
tuple = (3, 2)
for i in tuple:
i -= 1
print(tuple)
Any idea ?
Tuples are immutable objects, that is, you cannot change the elements of the tuple once it is created, if you do want to change them it would be better to use a list.
a = [3, 2]
for i in a:
i -= 1
print(a)
Which gives an output:
[3, 2]
Why didn't it work, even though lists are mutable? And also, why doesn't your original code produce some kind of error, if tuples are really immutable? Well, this becomes more obvious if you write your for each style loop as a simple for:
for index in range(len(a)):
i = a[index]
i -= 1
print(a)
This code is just a more verbose version of your original example, but now the problem is clear - the code never actually changes the elements of the original list. Instead, each iteration of the for loop creates a new variable, i, which takes its value from the correspondinglist element. i -= 1 changes i, not a[index].
Incidentally, that is why there is no error in your original code, even though tuples are supposed to be immutable - the code simply doesn't try to mutate the original tuple at all.
A working example with a list would be something like:
a = [3, 2]
for index in range(len(a)):
a[index] -= 1
print(a)
which can be made more concise in the end by writing:
a = [3, 2]
a = [i - 1 for i in a]
print(a)
As an aside, if you really must take tuples as input and produce tuples as output, you can convert between tuples and lists using the list and tuple functions. This is also why you shouldn't use tuple as your variable name! Doing so will mean that you can't use that tuple function in the same scope as your tuple variable, making things very difficult and potentially confusing for you.
tuple is immutable
my_tuple = (1,2,3)
my_tuple [0] = 3
Traceback (most recent call last):
File "<input>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
A couple of things going on. A tuple is immutable and you're assigning the value (3, 2) to the type tuple .
tup = (3, 2)
for i in tup:
i -= 1
print(tup)
Outputs:
(3, 2)
However you could do a list comprehension and cast it to tuple to get the expected result.
tup = (3,2)
tup = tuple([i - 1 for i in tup])
print(tup)
Outputs:
(2, 1)
Tuple is immutable.
try this, i think it will help you.
my_tuple = (3, 2)
new_list = []
for i in my_tuple:
i -= 1
new_list.append(i)
new_tuple = tuple(new_list)
print(new_tuple)
A brief introduction to python data structures should help you out here. Here's the gist
tuples are immutable i.e once created you cannot change the contents, you'll need to create a new one - you cannot modify the contents inplace similar to how strings work in python.
list makes sense but in your case it will still not work because you need to update the contents of the list when you do this:
tuple = (3, 2)
for i in tuple:
i -= 1
print(tuple)
you are basically extracting the contents from the tuple and modifying the value but not storing it anywhere. Even if you use a list like this:
list = [3,2]
for i in list:
i -= 1
print(list)
this will still not work as you are modifying a list item (a primitive type - int, string,etc.) which creates a new item but as it's not being stored it gets lost, when you print the list - it remains the same.
What we can do is modify the list as its mutable at runtime and update the contents like this:
list = [3, 2]
for idx, item in enumerate(list):
list[idx] = item - 1
print(list)
I think tuple is a bad variable name for a tuple because it shadows the tuple of Python
Issue is the values assigned to i are integers and integers themselves are not mutable. So even if i changes, the value in tuple does not. In other words, even though i points to the same object as the integer in tuple, when you do in place subtraction, a new integer is formed and i now points to that, different than what tuple has.
However, if you were to put a mutable object and do similar things:
tup = ([1, 2, 3], [4, -12])
for i in tup:
i[1] = 999
print(tup)
You get
([1, 999, 3], [4, 999])
python noob here and playing around with the limitations of Tuples and Lists.
I don't have a problem, just a general query about usage of the __methodname__ methods in the Tuple class and/or in general.
I am aware that you cannot modify Tuples and in order to do so, you have to convert it to a list, modify said list, then convert back to Tuple but I had a play around with the __add__ method and found that it works. What are issues and limitations with using this to create new Tuples with modifications to an existing one?
CODE:
myTuple = ('item1', 2, 'item3', ['list1', 'list2'])
tupleModification = myTuple.__add__(('newTupleItem1','newTupleItem2'))
This outputs the following:
('item1', 2, 'item3', ['list1', 'list2'], 'newTupleItem1', 'newTupleItem2')
which is correct but i'm wondering if i'm playing with fire because I haven't seen this solution posted anywhere in relation to modifying Tuples.
EDIT: I am aware that you cannot modify existing Tuples and that this will create a new instance of one. I think I may have confused people with my naming conventions.
__add__ is the method which is called when you do:
myTuple + ("newTupleItem1", "newTupleItem2")
So this does not modify myTuple but creates a new tuple whose content is the content of myTuple concatenated with ("newTupleItem1", "newTupleItem2").
You can print myTuple to see that it has not been modified:
>>> myTuple
('item1', 2, 'item3', ['list1', 'list2'])
And you can check that myTuple and tupleModification are not the same object:
>>> myTuple is tupleModification
False
You cannot modify tuples, that's right. However, you can concatenate two existing tuples into a new tuple. This is done with the + operator, which in turn calls the __add__ method. The resulting tuple will not be a "modification" of any of the original ones, but a new distinct tuple. This is what the code you posted does. More concisely, you can just do:
myTuple = ('item1', 2, 'item3', ['list1', 'list2'])
tupleModification = myTuple + ('newTupleItem1','newTupleItem2')
print(tupleModification)
# ('item1', 2, 'item3', ['list1', 'list2'], 'newTupleItem1', 'newTupleItem2')
EDIT: Just as a clarification, you cannot "edit" a tuple anyhow, that is, add or remove elements from it, or change its contents. However, if your tuple contains a mutable object, such as a list, then that inner object can be modified:
myTuple = (1, [2, 3])
myTuple[1].append(4)
print(myTuple)
# (1, [2, 3, 4])
I think fundamentally you're confused about the difference between creating a new object with modifications (__add__ in this case) and modifying an existing object (extend for example).
__add__
As other answers already mentioned, the __add__ method implements the + operator, and it returns a new object. Tuples and lists each have one. For example:
>>> tuple_0 = (1,)
>>> tuple_1 = tuple_0.__add__((2,))
>>> tuple_1 is tuple_0
False
>>>
>>> list_0 = [1]
>>> list_1 = list_0.__add__([2])
>>> list_1 is list_0
False
extend
Lists, which are mutable, have an extend method, which modifies the existing object and returns None. Tuples, which are immutable, don't. For example:
>>> list_2 = [4, 5, 6]
>>> id_save = id(list_2)
>>> list_2.extend([7])
>>> id(list_2) == id_save
True
>>>
>>> tuple_2 = (4, 5, 6)
>>> tuple_2.extend([7])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'tuple' object has no attribute 'extend'
Lists have other methods which change the existing object, like append, sort, pop, etc, but extend is the most similar to __add__.
There is a difference between 1) modifying an existing tuple (in-place), and 2) leaving the original alone, but creating a new modified copy. These two different paradigms can be seen throughout computer programming, not just python tuples. For example, consider increment a number by one. You could modify the original, or you could leave the original alone, create a copy and then modify the copy.
MODIFYING IN-PLACE
BEFORE:
x == 5
AFTER:
x == 6
CREATING A MODIFIED COPY
BEFORE:
x == 5
AFTER:
x == 5 (unchanged)
y == 6
tuple.__add__ concatenates tuples. For example, (x, y) + (a, b, c) returns (x, y, a, b, c)
tuple.__add__ does not modify the original tuples. It leaves the original tuples alone, and creates a new tuple which is the concatenation of the original two. This is contrast to something like list.append or list.extend which modifies the original list instead of returning a modified copy.
Tuple methods generally do the following:
copy the original
modify the copy in some way
leave the original alone.
This question already has answers here:
Why do these list operations (methods: clear / extend / reverse / append / sort / remove) return None, rather than the resulting list?
(6 answers)
Closed 6 years ago.
What is the actual difference between list1.append() and list1+list2 in python??
Along with this, why the following statement return NULL?
print(list1.append(list2))
{where list1 and list2 are 2 simple list}
list.append() modifies the object and returns None.
[] + [] creates and "returns" new list.
https://docs.python.org/2.7/tutorial/datastructures.html#more-on-lists
Recommended reading: http://docs.python-guide.org/en/latest/writing/gotchas/
Returning None is a way to communicate that an operation is side-effecting -- that is, that it's changing one of its operands, as opposed to leaving them unchanged and returning a new value.
list1.append(list2)
...changes list1, making it a member of this category.
Compare the following two chunks of code:
# in this case, it's obvious that list1 was changed
list1.append(list2)
print list1
...and:
# in this case, you as a reader don't know if list1 was changed,
# unless you already know the semantics of list.append.
print list1.append(list2)
Forbidding the latter (by making it useless) thus enhances the readability of the language.
>>> a = [1,2,3]
>>> b = [4,5,6]
>>> a.append(b) # append just appends the variable to the next index and returns None
>>> print a
[1,2,3,[4,5,6]]
>>> a.extend(b) # Extend takes a list as input and extends the main list
[1,2,3,4,5,6]
>>> a+b # + is exactly same as extend
[1,2,3,4,5,6]
When you print a function, you print what it returns and the append method does not return anything. However your list now has a new element. You can print the list to see the changes made.
list1 + list2 means that you combine 2 lists into one big list.
list1.append(element) adds one element to the end of the list.
Here's an example of append vs +
>>> a = [1,2,3]
>>> b = [4,5,6]
>>> a + b
[1, 2, 3, 4, 5, 6]
>>>
>>> a.append(b)
>>> a
[1, 2, 3, [4, 5, 6]]
I am learning concepts of functional programming and trying out problem exercises.
An exercise,
Reverse a list using map/reduce.
My solution:
lists = [ 1, 2, 3, 5 ,6]
def tree_reverse(lists):
return reduce(lambda a, x: a + [a.insert(0,x)], lists, [])
print tree_reverse(lists)
Output:
[6, 5, 3, 2, 1, None, None, None, None, None]
I don't understand why there are Nones equal to number of elements in list.
EDIT: Extending question for case of nested lists.
lists = [ 1 ,2 ,3 , [5,6,7], 8, [9, 0, 1 ,4 ]]
At first, i don't know Python good enough, but as you marked the question as "functional-programming" and you stated you want to learn in general about functional programming i want to give some general explanation of your problem.
At first, you should rethink what the map and reduce operation do and what they expect. Let's start with the map function.
A map function for a list (in functional programming you don't have only one map, you have one map for every generic type) executes a function on every single argument. That function can operate on that single argument and transform it. The result of this transformation is turned into a new list. One important note is that the function you pass to map only ever sees one element, so you cannot use map itself to transform a list as a whole. You only can transform every single element of a list. But changing the order needs the idea of having at least two elements of a list, and a map only ever gets a single element. This is the reason why the reverse logic cannot be in map and has to be part of the reduce operation.
The reduce function on the other hand expects a function that takes two things of the same type, and returns a single item of the same type. For example the reduce function could take the "+" function. "+" is a function that takes two int and returns a new int. The signature is basically int + int = int. The essence of reduce is to take two things and turn it into a single new item. If you for example had something more complex like a class "Foo", then the function you have to provide to reduce need to have the signature Foo * Foo -> Foo. Meaning take a Foo as first and second argument, and return a single new Foo.
It is also important to note how reduce will work with that function. Let's assume you have the List [1,2,3,4,5] on top let's assume you have an accumulator function that takes two int that just adds them together. What you can think of what will happen is the following.
The reduce function takes the first two arguments of your list (1 and 2 in this case)
Passes those to the accumulator function (1 + 2 = 3)
And replaces the two arguments in the list with the result. [3,3,4,5]
Go back to 1) and repeat that process until your List only have a single item
Return that single item
So what you can imagine what happens is the following
[1,2,3,4,5] -> 1 + 2 = 3
[3,3,4,5] -> 3 + 3 = 6
[6,4,5] -> 6 + 4 = 10
[10,5] -> 10 + 5 = 15
[15] -> Return just "15". Not a "[15]"
Actually the process internal usually works a little bit different. But this is the process you can imagine what happens. It is important to note that you never modify a list. You always create new lists in that process. Another important note. At the end of the reduce function you have a list of an single item. But the reduce function does not return the whole list. It returns just the single element. So you get 15 an int as a result. You don't get a list with a single item containing 15. Reduce will just return that single element. Whatever it is. One other way to think about it. You always get exactly the type back of your accumulator function. If you pass a accumulator function that takes two int adds them and returns a new int. Your reduce function will also return an int. If you pass an accumulator function that takes two Foo classes and returns a new Foo. Your reduce function will also return a Foo as its result. The return type of reduce is always the same as the types of your accumulator function.
Now let's take all those pieces and put them together. The goal is to reverse a list. The first important thing is. Your result type will be a list. That also means that the function you pass into reduce also have to return a list. But as the input are always the same as the output. You now have to provide a accumulator function that takes two lists, and returns a single new list.
But now let's step back. What happens if you use your list "[1,2,3,4,5]" directly as the input to reduce? The short answer is, it will not work. reduce will take two argument of your list. But what you have is a list of int. But your accumulator function expects two lists not two int. To solve that problem, what you now can do is try to convert every single element of the list into its own list. So how do we transform every single element of a list? Right! With map! So what you have to do is the following. You first map your list into a list of lists.
[1,2,3,4,5] -> [[1],[2],[3],[4],[5]]
Now your reduce function gets the first two elements of your first list. It means you now have an accumulator function that gets [1] and [2] as its argument. Two separate lists. But your accumulator function has to return a single new list. If unclear, just remind what the accumulator function takes and return.
int, int -> int
float,float -> float
Foo,Foo -> Foo
list,list -> list
So what you now have to do is to combine those two lists into a single new list. Or in other words you have to append/concat those two lists. I don't knew exactly how you concat two lists in Python let's assume here the operation would be "++". So if you concat just the first argument and the second argument, you will not get what you want.
[[1],[2],[3],[4],[5]] -> [1] ++ [2] = [1,2]
[[1,2],[3],[4],[5]] -> [1,2] ++ [3] = [1,2,3]
[[1,2,3],[4],[5]] -> [1,2,3] ++ [4] = [1,2,3,4]
[[1,2,3,4],[5]] -> [1,2,3,4] ++ [5] = [1,2,3,4,5]
[[1,2,3,4,5]] -> Return first element [1,2,3,4,5]
So what you have to do is to concat the second argument with the first one.
[[1],[2],[3],[4],[5]] -> [2] ++ [1] = [2,1]
[[2,1],[3],[4],[5]] -> [3] ++ [2,1] = [3,2,1]
[[3,2,1],[4],[5]] -> [4] ++ [3,2,1] = [4,3,2,1]
[[4,3,2,1],[5]] -> [5] ++ [4,3,2,1] = [5,4,3,2,1]
[[5,4,3,2,1]] -> Return first element [5,4,3,2,1]
And now what you get is your reversed list. So what you have todo
map every element to a list. So you get a list of list
use reduce to concat every list of list into a new list in reversed order.
For example, in F# the whole code would look like this.
let list = [1;2;3;4;5]
let reversed =
list
|> List.map (fun x -> [x]) // map every element to a list
|> List.reduce (fun xs ys -> List.append ys xs) // Note ys ++ xs - reversed order for appending
printfn "%A" reversed
// prints: [5;4;3;2;1]
I think you should be able to translate this to Python. As a further notice. Doing "map" and then a "concat" operation (without reversing) is also named "bind" in functional programming.
Your reduce operation is appending a None to the end of the list a because a.insert(0,x) will return None.
So you are modifying the list a in-place, but appending a None value at the same time.
If you rewrite your reduce operation to use a for loop, it'd look like:
def reverse_list(lists):
r = []
for x in lists:
val = r.insert(0,x)
r = r + [val] # val (the return value of r.insert) is None
return r
In case of nested list. I am posting answer for reference.
lists = [ 1 ,2 ,3 , [5,6,7], 8, [9, 0, 1 ,4 ]]
def tree_reverse(lists):
return reduce(lambda a, x: ( map(tree_reverse, [x]) if isinstance(x,list) else [x] ) + a , lists , [] )
print tree_reverse(lists)
Output :
[[4, 1, 0, 9], 8, [7, 6, 5], 3, 2, 1]
Functional programming languages will often use recursion to implement iteration.
A recursive tree_reverse function might look like
def tree_reverse(lists):
return tree_reverse(lists[1:]) + [lists[0]] if lists else []
If you look at Haskell it might be implemented like this
reverse' :: [a] -> [a]
reverse' [] = []
reverse' (x:xs) = reverse' xs ++ [x]
Where the x is the head of the list (first element) and xs is the tail (the list minus the head) and you can see that reverse' is called recursively with those elements reversed and the reversed list is built up iteratively.