What is the difference between the following two code snippets:
a1 = map(int,'222211112211')
and
a2 = int('222211112211')
Why can I iterate over a1 which is type int?
For example: I can do something like this with a1:
for i in a1:
print(i)
But not with a2?
In a1, you're iterating over a string and converting each of its characters into an int, the result is an iterator of ints, check the documentation for map:
list(a1) # use `list()` to consume the iterator
=> [2, 2, 2, 2, 1, 1, 1, 1, 2, 2, 1, 1]
In a2, you're converting a string into an int, the result is an int. They're very different things!
a2
=> 222211112211
That's why you can iterate over a1, because it's an iterator, not an int. And that's why you can't iterate over a2, it's just an int.
I think the best thing to do in situations like these is to check the actual values in the REPL.
>>> a1 = map(int, '2212')
>>> a1
<map object at 0x7f0036c2a4a8>
>>> list(a1)
[2, 2, 1, 2]
>>> a2 = int('2212')
>>> a2
2212
So a1 is a special map object which turns out to be iterable. It stores each character of '2212', individually converted to an integer. Meanwhile, a2 simply converts the whole string to a simple integer. It would be an error to iterate over a2, but it would also be an error to do integer arithmetic on a1.
in python 3.6 ,the map function returns a generator ,
Note:these are are not loaded into the memory as a whole
This is the implementation of map in python3
def map(func, iterable):
for i in iterable:
yield func(i)
when u call the map as iterable the value is produced onspot as opposed to loading the entire thing, which could crash the memory in some cases.
list(map()) does is load the entire list into the memory which is what python2 versions of map does
`a=int('2212')`
gives a an integer value thats it its not an iterable
a=map(int,'2212')
returns an iterable generator object
<map object at 0x7f97e30d7f98>
it is an iterable which converts each character in the string to int and yields the result one by one
so
a=map(int ,'2212')
for i in a:
print(i)
for i in a:
print(i)
would print
2
2
1
2
calling the stored map object second time yields no result as the function has run out of values in the first run
if you want to get the values in the second run convert it to a list so that it resides in the main memory list(a) or if it is really long store the result of the map object in a separate file and read from there
In Python 3, map return an iterator that you can use in a for loop.
map takes at least two parameters, a function and an iterable argument. Here int is the function and '222211112211', a string, is the iterable object. map applies the function to each value of the iterable. Here, int will be applied to "2", "2", "2", ... "2", "1", "1" individually and make them all to be integers. map will return an iterator that allows you to loop over the results yielding from the previous step: (2, 2, 2, ..., 1, 1)
For a2, you are creating an integer out of the int function, and an integer is not iterable. Thus, you cannot loop of it.
Below is the description of map cited from Python 3 documentation.
map(function, iterable, ...)
: Return an iterator that applies function to every item of iterable, yielding the results. If additional iterable arguments are passed, function must take that many arguments and is applied to the items from all iterables in parallel. With multiple iterables, the iterator stops when the shortest iterable is exhausted. For cases where the function inputs are already arranged into argument tuples, see itertools.starmap().
One thing worth noting is that in Python 2, map returns a list rather than an iterator. In Python 3, you have to explicitly make it a list like list(map(int, "222")).
Related
Why do these two operations (append() resp. +) give different results?
>>> c = [1, 2, 3]
>>> c
[1, 2, 3]
>>> c += c
>>> c
[1, 2, 3, 1, 2, 3]
>>> c = [1, 2, 3]
>>> c.append(c)
>>> c
[1, 2, 3, [...]]
>>>
In the last case there's actually an infinite recursion. c[-1] and c are the same. Why is it different with the + operation?
To explain "why":
The + operation adds the array elements to the original array. The array.append operation inserts the array (or any object) into the end of the original array, which results in a reference to self in that spot (hence the infinite recursion in your case with lists, though with arrays, you'd receive a type error).
The difference here is that the + operation acts specific when you add an array (it's overloaded like others, see this chapter on sequences) by concatenating the element. The append-method however does literally what you ask: append the object on the right-hand side that you give it (the array or any other object), instead of taking its elements.
An alternative
Use extend() if you want to use a function that acts similar to the + operator (as others have shown here as well). It's not wise to do the opposite: to try to mimic append with the + operator for lists (see my earlier link on why). More on lists below:
Lists
[edit] Several commenters have suggested that the question is about lists and not about arrays. The question has changed, though I should've included this earlier.
Most of the above about arrays also applies to lists:
The + operator concatenates two lists together. The operator will return a new list object.
List.append does not append one list with another, but appends a single object (which here is a list) at the end of your current list. Adding c to itself, therefore, leads to infinite recursion.
As with arrays, you can use List.extend to add extend a list with another list (or iterable). This will change your current list in situ, as opposed to +, which returns a new list.
Little history
For fun, a little history: the birth of the array module in Python in February 1993. it might surprise you, but arrays were added way after sequences and lists came into existence.
The concatenation operator + is a binary infix operator which, when applied to lists, returns a new list containing all the elements of each of its two operands. The list.append() method is a mutator on list which appends its single object argument (in your specific example the list c) to the subject list. In your example this results in c appending a reference to itself (hence the infinite recursion).
An alternative to '+' concatenation
The list.extend() method is also a mutator method which concatenates its sequence argument with the subject list. Specifically, it appends each of the elements of sequence in iteration order.
An aside
Being an operator, + returns the result of the expression as a new value. Being a non-chaining mutator method, list.extend() modifies the subject list in-place and returns nothing.
Arrays
I've added this due to the potential confusion which the Abel's answer above may cause by mixing the discussion of lists, sequences and arrays.
Arrays were added to Python after sequences and lists, as a more efficient way of storing arrays of integral data types. Do not confuse arrays with lists. They are not the same.
From the array docs:
Arrays are sequence types and behave very much like lists, except that the type of objects stored in them is constrained. The type is specified at object creation time by using a type code, which is a single character.
append is appending an element to a list. if you want to extend the list with the new list you need to use extend.
>>> c = [1, 2, 3]
>>> c.extend(c)
>>> c
[1, 2, 3, 1, 2, 3]
Python lists are heterogeneous that is the elements in the same list can be any type of object. The expression: c.append(c) appends the object c what ever it may be to the list. In the case it makes the list itself a member of the list.
The expression c += c adds two lists together and assigns the result to the variable c. The overloaded + operator is defined on lists to create a new list whose contents are the elements in the first list and the elements in the second list.
So these are really just different expressions used to do different things by design.
The method you're looking for is extend(). From the Python documentation:
list.append(x)
Add an item to the end of the list; equivalent to a[len(a):] = [x].
list.extend(L)
Extend the list by appending all the items in the given list; equivalent to a[len(a):] = L.
list.insert(i, x)
Insert an item at a given position. The first argument is the index of the element before which to insert, so a.insert(0, x) inserts at the front of the list, and a.insert(len(a), x) is equivalent to a.append(x).
you should use extend()
>>> c=[1,2,3]
>>> c.extend(c)
>>> c
[1, 2, 3, 1, 2, 3]
other info: append vs. extend
See the documentation:
list.append(x)
Add an item to the end of the list; equivalent to a[len(a):] = [x].
list.extend(L)
- Extend the list by appending all the items in the given list;
equivalent to a[len(a):] = L.
c.append(c) "appends" c to itself as an element. Since a list is a reference type, this creates a recursive data structure.
c += c is equivalent to extend(c), which appends the elements of c to c.
I have a tuple of numbers, and I want to choose a random number from my tuple in addition with a certain number. For example:
my_tuple = (1,2,3)
and I have the number 4. I want to choose a random number from the numbers 1,2,3,4 (without changing the tuple of course).
I tried:
my_tp = (1, 2, 3)
a = random.choice(list(my_tp).append(4))
print(a)
I'm new to python. I tried converting the tuple to a list, and then performing the random function.
The code above didn't work. Got the error
object of type 'NoneType' has no len()
Would love some help.
list.append returns none
Once converting to a list as you have done, appending will modify that list but return none so that is the source of your error.
To get round that you can either convert the tuple to a list then append 4 to it, then use random.choice, or in just one step, you can concatenate a list of [4] with the + operand.
This approach is much simpler:
import random
my_tuple = (1,2,3)
random.choice(list(my_tuple) + [4])
Hope this helps and clears some things up! :)
Update:
If you want to just randomly select from the tuple without the last item, then just slice the list with the normal syntax:
random.choice(list(my_tuple)[:-1])
The method list.append alters the provided list and returns None, which explains the exception you got. To get the desired result, you can implicitly use the tuple.__add__ method, which will return a new tuple.
values = (1, 2, 3)
random.choice(values + (4,))
If you want to remove values in a concise-ish fashion, using a set might be appropriate.
values = {1, 2, 3}
random.choice(list(values - {3}))
You can try:
my_tuple = (1,2,3)
random.choice((*my_tuple, 4))
Where (*my_tuple, 4) creates a new tuple with the unpacked content of my_tuple and 4.
Lets say, there is a nested list, like:
my_list = [[1, 2, 21], [1, 3], [1, 2]]
When the function min() is called on this:
min(my_list)
The output received is
[1, 2]
Why and How does it work? What are some use cases of it?
How are lists and other sequences compared in Python?
Lists (and other sequences) in Python are compared lexicographically and not based on any other parameter.
Sequence objects may be compared to other objects with the same sequence type. The comparison uses lexicographical ordering: first the first two items are compared, and if they differ this determines the outcome of the comparison; if they are equal, the next two items are compared, and so on, until either sequence is exhausted.
What is lexicographic sorting?
From the Wikipedia page on lexicographic sorting
lexicographic or lexicographical order (also known as lexical order, dictionary order, alphabetical order or lexicographic(al) product) is a generalization of the way the alphabetical order of words is based on the alphabetical order of their component letters.
The min function returns the smallest value in the iterable. So the lexicographic value of [1,2] is the least in that list. You can check by using [1,2,21]
>>> my_list=[[1,2,21],[1,3],[1,2]]
>>> min(my_list)
[1, 2]
What is happening in this case of min?
Going element wise on my_list, firstly [1,2,21] and [1,3]. Now from the docs
If two items to be compared are themselves sequences of the same type, the lexicographical comparison is carried out recursively.
Thus the value of [1,1,21] is less than [1,3], because the second element of [1,3], which is, 3 is lexicographically higher than the value of the second element of [1,1,21], which is, 1.
Now comparing [1,2] and [1,2,21], and adding another reference from the docs
If one sequence is an initial sub-sequence of the other, the shorter sequence is the smaller (lesser) one.
[1,2] is an initial sub-sequence of [1,2,21]. Therefore the value of [1,2] on the whole is smaller than that of [1,2,21]. Hence [1,2] is returned as the output.
This can be validated by using the sorted function
>>> sorted(my_list)
[[1, 2], [1, 2, 21], [1, 3]]
What if the list has multiple minimum elements?
If the list contains duplicate min elements the first is returned
>>> my_list=[[1,2],[1,2]]
>>> min(my_list)
[1, 2]
This can be confirmed using the id function call
>>> my_list=[[1,2],[1,2]]
>>> [id(i) for i in my_list]
[140297364849368, 140297364850160]
>>> id(min(my_list))
140297364849368
What do I need to do to prevent lexicographic comparison in min?
If the required comparison is not lexicographic then the key argument can be used (as mentioned by Padraic)
The min function has an additional optional argument called key. The key argument takes a function.
The optional key argument specifies a one-argument ordering function
like that used for list.sort(). The key argument, if supplied, must be
in keyword form (for example, min(a,b,c,key=func)).
For example, if we need the smallest element by length, we need to use the len function.
>>> my_list=[[1,2,21],[1,3],[1,2]]
>>> min(my_list,key=len) # Notice the key argument
[1, 3]
As we can see the first shortest element is returned here.
What if the list is heterogeneous?
Until Python2
If the list is heterogeneous type names are considered for ordering, check Comparisions,
Objects of different types except numbers are ordered by their type names
Hence if you put an int and a list there you will get the integer value as the smallest as i is of lower value than l. Similarly '1' would be of higher value than both of this.
>>> my_list=[[1,1,21],1,'1']
>>> min(my_list)
1
Python3 and onwards
However this confusing technique was removed in Python3. It now raises a TypeError. Read What's new in Python 3.0
The ordering comparison operators (<, <=, >=, >) raise a TypeError exception when the operands don’t have a meaningful natural ordering. Thus, expressions like 1 < '', 0 > None or len <= len are no longer valid, and e.g. None < None raises TypeError instead of returning False. A corollary is that sorting a heterogeneous list no longer makes sense – all the elements must be comparable to each other.
>>> my_list=[[1,1,21],1,'1']
>>> min(my_list)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unorderable types: int() < list()
But it works for Comparable types, For example
>>> my_list=[1,2.0]
>>> min(my_list)
1
Here we can see that the list contains float values and int values. But as float and int are comparable types, min function works in this case.
One simple use case for lexicographical sorting is with making a sortable namedtuple class.
from collections import namedtuple
Time = namedtuple('Time', ['hours', 'minutes', 'seconds'])
t1 = Time(hours=8, minutes=15, seconds=30)
t2 = Time(hours=8, minutes=15, seconds=0)
t3 = Time(hours=8, minutes=30, seconds=30)
t4 = Time(hours=7, minutes=15, seconds=30)
assert min(t1, t2, t3, t4) == t4
assert max(t1, t2, t3, t4) == t3
Two lists are compared element wise
Even if sizes of two lists are different the two lists are compared element wise starting the comparison from the first element.
Now suppose that every element of a list has been checked and they are the same and there is no next element in the shorter list. Then the shorter list is declared to be smaller than the longer one.
Examples:
>>> [1,2]<[1,3]
True
>>> [1,2]<[1,2,21]
True
>>> [1,3]<[1,2,21]
False
>>>[1,2,22]<[1,2,21]
False
>>>[1]<[1,2,21]
True
>>>
it compares the lists elementwise:
>>> [1,2]<[1,3]
True
>>> [1,2]<[1,2,21]
True
>>>
I am learning concepts of functional programming and trying out problem exercises.
An exercise,
Reverse a list using map/reduce.
My solution:
lists = [ 1, 2, 3, 5 ,6]
def tree_reverse(lists):
return reduce(lambda a, x: a + [a.insert(0,x)], lists, [])
print tree_reverse(lists)
Output:
[6, 5, 3, 2, 1, None, None, None, None, None]
I don't understand why there are Nones equal to number of elements in list.
EDIT: Extending question for case of nested lists.
lists = [ 1 ,2 ,3 , [5,6,7], 8, [9, 0, 1 ,4 ]]
At first, i don't know Python good enough, but as you marked the question as "functional-programming" and you stated you want to learn in general about functional programming i want to give some general explanation of your problem.
At first, you should rethink what the map and reduce operation do and what they expect. Let's start with the map function.
A map function for a list (in functional programming you don't have only one map, you have one map for every generic type) executes a function on every single argument. That function can operate on that single argument and transform it. The result of this transformation is turned into a new list. One important note is that the function you pass to map only ever sees one element, so you cannot use map itself to transform a list as a whole. You only can transform every single element of a list. But changing the order needs the idea of having at least two elements of a list, and a map only ever gets a single element. This is the reason why the reverse logic cannot be in map and has to be part of the reduce operation.
The reduce function on the other hand expects a function that takes two things of the same type, and returns a single item of the same type. For example the reduce function could take the "+" function. "+" is a function that takes two int and returns a new int. The signature is basically int + int = int. The essence of reduce is to take two things and turn it into a single new item. If you for example had something more complex like a class "Foo", then the function you have to provide to reduce need to have the signature Foo * Foo -> Foo. Meaning take a Foo as first and second argument, and return a single new Foo.
It is also important to note how reduce will work with that function. Let's assume you have the List [1,2,3,4,5] on top let's assume you have an accumulator function that takes two int that just adds them together. What you can think of what will happen is the following.
The reduce function takes the first two arguments of your list (1 and 2 in this case)
Passes those to the accumulator function (1 + 2 = 3)
And replaces the two arguments in the list with the result. [3,3,4,5]
Go back to 1) and repeat that process until your List only have a single item
Return that single item
So what you can imagine what happens is the following
[1,2,3,4,5] -> 1 + 2 = 3
[3,3,4,5] -> 3 + 3 = 6
[6,4,5] -> 6 + 4 = 10
[10,5] -> 10 + 5 = 15
[15] -> Return just "15". Not a "[15]"
Actually the process internal usually works a little bit different. But this is the process you can imagine what happens. It is important to note that you never modify a list. You always create new lists in that process. Another important note. At the end of the reduce function you have a list of an single item. But the reduce function does not return the whole list. It returns just the single element. So you get 15 an int as a result. You don't get a list with a single item containing 15. Reduce will just return that single element. Whatever it is. One other way to think about it. You always get exactly the type back of your accumulator function. If you pass a accumulator function that takes two int adds them and returns a new int. Your reduce function will also return an int. If you pass an accumulator function that takes two Foo classes and returns a new Foo. Your reduce function will also return a Foo as its result. The return type of reduce is always the same as the types of your accumulator function.
Now let's take all those pieces and put them together. The goal is to reverse a list. The first important thing is. Your result type will be a list. That also means that the function you pass into reduce also have to return a list. But as the input are always the same as the output. You now have to provide a accumulator function that takes two lists, and returns a single new list.
But now let's step back. What happens if you use your list "[1,2,3,4,5]" directly as the input to reduce? The short answer is, it will not work. reduce will take two argument of your list. But what you have is a list of int. But your accumulator function expects two lists not two int. To solve that problem, what you now can do is try to convert every single element of the list into its own list. So how do we transform every single element of a list? Right! With map! So what you have to do is the following. You first map your list into a list of lists.
[1,2,3,4,5] -> [[1],[2],[3],[4],[5]]
Now your reduce function gets the first two elements of your first list. It means you now have an accumulator function that gets [1] and [2] as its argument. Two separate lists. But your accumulator function has to return a single new list. If unclear, just remind what the accumulator function takes and return.
int, int -> int
float,float -> float
Foo,Foo -> Foo
list,list -> list
So what you now have to do is to combine those two lists into a single new list. Or in other words you have to append/concat those two lists. I don't knew exactly how you concat two lists in Python let's assume here the operation would be "++". So if you concat just the first argument and the second argument, you will not get what you want.
[[1],[2],[3],[4],[5]] -> [1] ++ [2] = [1,2]
[[1,2],[3],[4],[5]] -> [1,2] ++ [3] = [1,2,3]
[[1,2,3],[4],[5]] -> [1,2,3] ++ [4] = [1,2,3,4]
[[1,2,3,4],[5]] -> [1,2,3,4] ++ [5] = [1,2,3,4,5]
[[1,2,3,4,5]] -> Return first element [1,2,3,4,5]
So what you have to do is to concat the second argument with the first one.
[[1],[2],[3],[4],[5]] -> [2] ++ [1] = [2,1]
[[2,1],[3],[4],[5]] -> [3] ++ [2,1] = [3,2,1]
[[3,2,1],[4],[5]] -> [4] ++ [3,2,1] = [4,3,2,1]
[[4,3,2,1],[5]] -> [5] ++ [4,3,2,1] = [5,4,3,2,1]
[[5,4,3,2,1]] -> Return first element [5,4,3,2,1]
And now what you get is your reversed list. So what you have todo
map every element to a list. So you get a list of list
use reduce to concat every list of list into a new list in reversed order.
For example, in F# the whole code would look like this.
let list = [1;2;3;4;5]
let reversed =
list
|> List.map (fun x -> [x]) // map every element to a list
|> List.reduce (fun xs ys -> List.append ys xs) // Note ys ++ xs - reversed order for appending
printfn "%A" reversed
// prints: [5;4;3;2;1]
I think you should be able to translate this to Python. As a further notice. Doing "map" and then a "concat" operation (without reversing) is also named "bind" in functional programming.
Your reduce operation is appending a None to the end of the list a because a.insert(0,x) will return None.
So you are modifying the list a in-place, but appending a None value at the same time.
If you rewrite your reduce operation to use a for loop, it'd look like:
def reverse_list(lists):
r = []
for x in lists:
val = r.insert(0,x)
r = r + [val] # val (the return value of r.insert) is None
return r
In case of nested list. I am posting answer for reference.
lists = [ 1 ,2 ,3 , [5,6,7], 8, [9, 0, 1 ,4 ]]
def tree_reverse(lists):
return reduce(lambda a, x: ( map(tree_reverse, [x]) if isinstance(x,list) else [x] ) + a , lists , [] )
print tree_reverse(lists)
Output :
[[4, 1, 0, 9], 8, [7, 6, 5], 3, 2, 1]
Functional programming languages will often use recursion to implement iteration.
A recursive tree_reverse function might look like
def tree_reverse(lists):
return tree_reverse(lists[1:]) + [lists[0]] if lists else []
If you look at Haskell it might be implemented like this
reverse' :: [a] -> [a]
reverse' [] = []
reverse' (x:xs) = reverse' xs ++ [x]
Where the x is the head of the list (first element) and xs is the tail (the list minus the head) and you can see that reverse' is called recursively with those elements reversed and the reversed list is built up iteratively.
What is this called in python:
[('/', MainPage)]
Is that an array .. of ... erhm one dictionary?
Is that
()
A tuple? ( or whatever they call it? )
Its a list with a single tuple.
Since no one has answered this bit yet:
A tuple? ( or whatever they call it? )
The word "tuple" comes from maths. In maths, we might talk about (ordered) pairs, if we're doing 2d geometry. Moving to three dimensions means we need triples. In higher dimensions, we need quadruples, quintuples, and, uh, whatever the prefix is for six, and so on. This starts to get to be a pain, and mathematicians also love generalising ("let's work in n dimensions today!"), so they started using the term "n-tuple" for an ordered list of n things (usually numbers).
After that, a bit of natural laziness is all you need to drop the "n-" and we end up with tuples.
Note that this:
("is not a tuple")
A tuple is defined by the commas, except in the case of the zero-length tuple. This:
"is a tuple",
because of the comma at the end. The parentheses just enforce grouping (again, except in the case of a zero-length tuple.
That's a list of tuples.
This is a list of integers: [1, 2, 3, 4, 5]
This is also a list of integers: [1]
This is a (string, integer) tuple: ("hello world", 42)
This is a list of (string, integer) tuples: [("a", 1), ("b", 2), ("c", 3)]
And so is this: [("a", 1)]
In Python, there's not much difference between lists and tuples. However, they are conceptually different. An easy way to think of it is that a list contains lots of items of the same type (homogeneous) , and a tuple contains a fixed number of items of different types (heterogeneous). An easy way to remember this is that lists can be appended to, and tuples cannot, because appending to a list makes sense and appending to a tuple doesn't.
Python doesn't enforce these distinctions -- in Python, you can append to a tuple with +, or store heterogeneous types in a list.
Yes, it's a tuple.
They look like this:
()
(foo,)
(foo, bar)
(foo, bar, baz)
etc.
[('/', MainPage)]
That's a list consisting of a two element tuple.
()
That's a zero element tuple.
It is a list of tuple(s). You can verify that by
x=[('/', MainPage)]
print type(x) # You will find a <list> type here
print type(x[0]) # You will find a <tuple> type here
You can build a dictionary from this type of structure (may be more tuple inside the list) with this code
my_dict = dict(x) # x=[('/',MainPage)]
It is a list of tuples containing one tuple.
A tuple is just like a list except that it is immutable, meaning that it can't be changed once it's created. You can't add, remove, or change elements in a tuple. If you want your tuple to be different, you have to create a new tuple with the new data. This may sound like a pain but in reality tuples have many benefits both in code safety and speed.
It's a list of just one tuple. That tuple has two elements, a string and the object MainPage whatever it is.
Both lists and tuples are ordered groups of object, it doesn't matter what kind of object, they can be heterogeneous in both cases.
The main difference between lists and tuples is that tuples are immutable, just like strings.
For example we can define a list and a tuple:
>>> L = ['a', 1, 5, 'b']
>>> T = ('a', 1, 5, 'b')
we can modify elements of L simply by assigning them a new value
>>> print L
['a', 1, 5, 'b']
>>> L[1] = 'c'
>>> print L
['a', 'c', 5, 'b']
This is not true for tuples
>>> print T
('a', 1, 5, 'b')
>>> T[1] = 'c'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
This is because they are immutable.
Tuples' elements may be mutable, and you can modify them, for example:
>>> T = (3, ['a', 1, 2], 'lol')
>>> T[1]
['a', 1, 2]
>>> T[1][0] = 'b'
>>> T
(3, ['b', 1, 2], 'lol')
but the list we edited is still the same object, we didn't replaced the tuple's element.