Used a loop to add a bunch of elements to a list with
mylist = []
for x in otherlist:
mylist.append(x[0:5])
But instead of the expected result ['x1','x2',...], I got: [u'x1', u'x2',...]. Where did the u's come from and why? Also is there a better way to loop through the other list, inserting the first six characters of each element into a new list?
The u means unicode, you probably will not need to worry about it
mylist.extend(x[:5] for x in otherlist)
The u means unicode. It's Python's internal string representation (from version ... ?).
Most times you don't need to worry about it. (Until you do.)
The answers above me already answered the "u" part - that the string is encoded in Unicode. About whether there's a better way to extract the first 6 letters from the items in a list:
>>> a = ["abcdefgh", "012345678"]
>>> b = map(lambda n: n[0:5], a);
>>> for x in b:
print(x)
abcde
01234
So, map applies a function (lambda n: n[0:5]) to each element of a and returns a new list with the results of the function for every element. More precisely, in Python 3, it returns an iterator, so the function gets called only as many times as needed (i.e. if your list has 5000 items, but you only pull 10 from the result b, lambda n: n[0:5] gets called only 10 times). In Python2, you need to use itertools.imap instead.
>>> a = [1, 2, 3]
>>> def plusone(x):
print("called with {}".format(x))
return x + 1
>>> b = map(plusone, a)
>>> print("first item: {}".format(b.__next__()))
called with 1
first item: 2
Of course, you can apply the function "eagerly" to every element by calling list(b), which will give you a normal list with the function applied to each element on creation.
>>> b = map(plusone, a)
>>> list(b)
called with 1
called with 2
called with 3
[2, 3, 4]
Related
I'm going through some old code trying to understand what it does, and I came across this odd statement:
*x ,= p
p is a list in this context. I've been trying to figure out what this statement does. As far as I can tell, it just sets x to the value of p. For example:
p = [1,2]
*x ,= p
print(x)
Just gives
[1, 2]
So is this any different than x = p? Any idea what this syntax is doing?
*x ,= p is basically an obfuscated version of x = list(p) using extended iterable unpacking. The comma after x is required to make the assignment target a tuple (it could also be a list though).
*x, = p is different from x = p because the former creates a copy of p (i.e. a new list) while the latter creates a reference to the original list. To illustrate:
>>> p = [1, 2]
>>> *x, = p
>>> x == p
True
>>> x is p
False
>>> x = p
>>> x == p
True
>>> x is p
True
It's a feature that was introduced in Python 3.0 (PEP 3132). In Python 2, you could do something like this:
>>> p = [1, 2, 3]
>>> q, r, s = p
>>> q
1
>>> r
2
>>> s
3
Python 3 extended this so that one variable could hold multiple values:
>>> p = [1, 2, 3]
>>> q, *r = p
>>> q
1
>>> r
[2, 3]
This, therefore, is what is being used here. Instead of two variables to hold three values, however, it is just one variable that takes each value in the list. This is different from x = p because x = p just means that x is another name for p. In this case, however, it is a new list that just happens to have the same values in it. (You may be interested in "Least Astonishment" and the Mutable Default Argument)
Two other common ways of producing this effect are:
>>> x = list(p)
and
>>> x = p[:]
Since Python 3.3, the list object actually has a method intended for copying:
x = p.copy()
The slice is actually a very similar concept. As nneonneo pointed out, however, that works only with objects such as lists and tuples that support slices. The method you mention, however, works with any iterable: dictionaries, sets, generators, etc.
You should always throw these to dis and see what it throws back at you; you'll see how *x, = p is actually different from x = p:
dis('*x, = p')
1 0 LOAD_NAME 0 (p)
2 UNPACK_EX 0
4 STORE_NAME 1 (x)
While, the simple assignment statement:
dis('x = p')
1 0 LOAD_NAME 0 (p)
2 STORE_NAME 1 (x)
(Stripping off unrelated None returns)
As you can see UNPACK_EX is the different op-code between these; it's documented as:
Implements assignment with a starred target: Unpacks an iterable in TOS (top of stack) into individual values, where the total number of values can be smaller than the number of items in the iterable: one of the new values will be a list of all leftover items.
Which is why, as Eugene noted, you get a new object that's referred to by the name x and not a reference to an already existing object (as is the case with x = p).
*x, does seem very odd (the extra comma there and all) but it is required here. The left hand side must either be a tuple or a list and, due to the quirkiness of creating a single element tuple in Python, you need to use a trailing ,:
i = 1, # one element tuple
If you like confusing people, you can always use the list version of this:
[*x] = p
which does exactly the same thing but doesn't have that extra comma hanging around there.
You can clearly understand it from below example
L = [1, 2, 3, 4]
while L:
temp, *L = L
print(temp, L)
what it does is, the front variable will get the first item every time and the remaining list will be given to L.
The output will look shown below.
1 [2, 3, 4]
2 [3, 4]
3 [4]
4 []
Also look at below example
x, *y, z = "python"
print(x,y,z)
In this both x,z will get each one letter from the string meaning first letter is assigned to x and the last letter will be assigned to z and the remaining string will be assigned to variable y.
p ['y', 't', 'h', 'o'] n
One more example,
a, b, *c = [0,1,2,3]
print(a,b,c)
0 1 [2,3]
Boundary case: If there is nothing remaining for star variable then it will get an empty list.
Example:
a,b=[1]
print(a,b)
1 []
This question already has answers here:
What is the purpose of the single underscore "_" variable in Python?
(5 answers)
Closed 8 months ago.
Reading through Peter Norvig's Solving Every Sudoku Puzzle essay, I've encountered a few Python idioms that I've never seen before.
I'm aware that a function can return a tuple/list of values, in which case you can assign multiple variables to the results, such as
def f():
return 1,2
a, b = f()
But what is the meaning of each of the following?
d2, = values[s] ## values[s] is a string and at this point len(values[s]) is 1
If len(values[s]) == 1, then how is this statement different than d2 = values[s]?
Another question about using an underscore in the assignment here:
_,s = min((len(values[s]), s) for s in squares if len(values[s]) > 1)
Does the underscore have the effect of basically discarding the first value returned in the list?
d2, = values[s] is just like a,b=f(), except for unpacking 1 element tuples.
>>> T=(1,)
>>> a=T
>>> a
(1,)
>>> b,=T
>>> b
1
>>>
a is tuple, b is an integer.
_ is like any other variable name but usually it means "I don't care about this variable".
The second question: it is "value unpacking". When a function returns a tuple, you can unpack its elements.
>>> x=("v1", "v2")
>>> a,b = x
>>> print a,b
v1 v2
The _ in the Python shell also refers to the value of the last operation. Hence
>>> 1
1
>>> _
1
The commas refer to tuple unpacking. What happens is that the return value is a tuple, and so it is unpacked into the variables separated by commas, in the order of the tuple's elements.
You can use the trailing comma in a tuple like this:
>>> (2,)*2
(2, 2)
>>> (2)*2
4
This question already has answers here:
Why do these list operations (methods: clear / extend / reverse / append / sort / remove) return None, rather than the resulting list?
(6 answers)
Closed 6 years ago.
What is the actual difference between list1.append() and list1+list2 in python??
Along with this, why the following statement return NULL?
print(list1.append(list2))
{where list1 and list2 are 2 simple list}
list.append() modifies the object and returns None.
[] + [] creates and "returns" new list.
https://docs.python.org/2.7/tutorial/datastructures.html#more-on-lists
Recommended reading: http://docs.python-guide.org/en/latest/writing/gotchas/
Returning None is a way to communicate that an operation is side-effecting -- that is, that it's changing one of its operands, as opposed to leaving them unchanged and returning a new value.
list1.append(list2)
...changes list1, making it a member of this category.
Compare the following two chunks of code:
# in this case, it's obvious that list1 was changed
list1.append(list2)
print list1
...and:
# in this case, you as a reader don't know if list1 was changed,
# unless you already know the semantics of list.append.
print list1.append(list2)
Forbidding the latter (by making it useless) thus enhances the readability of the language.
>>> a = [1,2,3]
>>> b = [4,5,6]
>>> a.append(b) # append just appends the variable to the next index and returns None
>>> print a
[1,2,3,[4,5,6]]
>>> a.extend(b) # Extend takes a list as input and extends the main list
[1,2,3,4,5,6]
>>> a+b # + is exactly same as extend
[1,2,3,4,5,6]
When you print a function, you print what it returns and the append method does not return anything. However your list now has a new element. You can print the list to see the changes made.
list1 + list2 means that you combine 2 lists into one big list.
list1.append(element) adds one element to the end of the list.
Here's an example of append vs +
>>> a = [1,2,3]
>>> b = [4,5,6]
>>> a + b
[1, 2, 3, 4, 5, 6]
>>>
>>> a.append(b)
>>> a
[1, 2, 3, [4, 5, 6]]
Could some one please explain why this code:
A = [1,2,3,4]
B = ((element) for element in A)
print(B)
produces: <generator object <genexpr> at 0x0319B490>
while this code:
A = [1,2,3,4]
for element in A:
print(A)
produces:
1
2
3
4
They seem to be the same to me but they are obviously different. I can't figure out the difference between them.
Thanks.
The first code is a generator expression, hence it will create a generator object at a certain memory address. If you want to use list comprehension then use [] as per:
A = [1,2,3,4]
B = [element for element in A]
print(B)
# [1, 2, 3, 4]
This list comprehension is equivalent to:
A = [1,2,3,4]
B = []
for element in A:
B.append(element)
The first is not a loop but a generator expresion so will printing B it shows us the object ref.
The second one Is a loop, it iterates over the elements and print them all.
Try doing this, you can iterate over a generator:
A = [1,2,3,4]
B = ((element) for element in A)
for e in B:
print(e)
This will result in the same as your second expresion:
for e in A:
print(e)
Notice that you can only iterate once until the generator is exausted.
The fundamental difference between the two is that a generator expression defines an object that will generate values as you loop. In other words, the values will be generated on each iteration and consumed on demand. With a list comprehension, the values are created up-front and will consume as much memory as is required to hold all the values in memory at once.
It's easy to look at these two constructs as being the exact same thing but in the case of the generator, you are consuming the values on demand in a lazy way. This is very useful because you don't have pay the cost of memory to hold all of the data up-front.
I am trying to use print inside lambda. Something like that:
lambda x: print x
I understand, that in Python 2.7 print is not a function. So, basically, my question is: Is there a pretty way to use print as function in Python 2.7?
You can import print_function from the __future__ and use it as a function like this
from __future__ import print_function
map(print, [1, 2, 3])
# 1
# 2
# 3
The question is about Python 2, but I ended up here from Google trying to use the print function inside a lambda in Python 3. I'm adding this answer for context for others that come here for the same.
If you only want to see the code that works and not how I arrived there, skip to the last code sample at the bottom. I wanted to clearly document what didn't work for learning purposes.
Desired result
Let's suppose you want to define a lambda print_list that prints each item of a list with a newline in between.
lst = [1, 2, 3]
print_list = lambda lst: ...
The desired output is:
1
2
3
And there should be no unused return value.
Attempt 1 - A map doesn't evaluate the print function in Python 3
To start, here's what doesn't work well in Python 3:
map(print, lst)
However, the output is somewhat counterintuitively not printed lines, because the map call in Python 3 returns an iterator instead of an evaluated list.
Output:
n/a
Return value:
<map at 0x111b3a6a0>
Attempt 2 - Evaluate the map iterator
You can realize the printing by passing the map result to list(...), which produces the ideal output, but has the side effect of returning a list of nulls (as evaluated in the REPL).
list(map(print, lst))
Output:
1
2
3
Return value:
[None, None, None]
You could workaround this by using the underscore throwaway variable convention:
_ = list(map(print, lst))
A similar approach is calling print inside a list comprehension:
[print(i) for i in lst]
I don't love these approaches because they both still generate an unused return value.
Attempt 3 - Apply the unpacking operator to the map iterator
Like this:
[*map(print, [1, 2, 3])]
(This still returns a list of nulls which is non-ideal.)
In the comments above #thefourtheye suggests using a one-line for loop:
for item in [1, 2, 3]: print(item)
This works fine for most cases and avoids the side effect. Attempting to put this in a lambda throws a SyntaxError. I tried wrapping it in parens without success; though there is probably a way to achieve this, I haven't figured it out.
(SOLUTION!) Attempt 4 - Apply the unpacking operator inside of the print call
The answer I arrived at is to explode the list inside the print call alongside using the separator arg:
print(*lst, sep='\n')
Output:
1
2
3
This produces the intended result without a return value.
Finally, let's wrap it up in a lambda to use as desired:
print_list = lambda lst: print(*lst, sep='\n')
print_list([1, 2, 3])
This was the best solution for my use case in Python 3.
Related questions
Why map(print, a_list) doesn't work?
Print doesnt print when it's in map, Python
If you don't want to import from __future__ you can just make the lambda write to the standard output:
>>>import sys
>>>l = lambda x : sys.stdout.write(x)
>>>l('hi')
'hi'
I guess there is another scenario people may be interested in: "print out the intermediate step value of the lambda function variables"
For instance, say I want to find out the charset of a collection of char list:
In [5]: instances = [["C","O","c","1","c","c","c","c","c","1","O","C","C","N","C"],
...: ["C","C","O","C","(","=","O",")","C","C","(","=","O",")","c"],
...: ["C","N","1","C","C","N","(","C","c","2","c","c","c","(","N"],
...: ["C","l","c","1","c","c","c","2","c","(","N","C","C","C","["],
...: ["C","C","c","1","c","c","c","(","N","C","(","=","S",")","N"]]
one way of doing this is to use reduce:
def build_charset(instances):
return list(functools.reduce((lambda x, y: set(y) | x), instances, set()))
In this function, reduce takes a lambda function with two variables x, y, which at the beginning I thought it would be like x -> instance, and y -> set(). But its results give a different story, so I want to print their value on the fly. lambda function, however, only take a single expression, while the print would introduce another one.
Inspired by set(y) | x, I tried this one and it worked:
lambda x, y: print(x, y) or set(y) | x
Note that print() is of NoneType, so you cannot do and, xor these kinds of operation that would change the original value. But or works just fine in my case.
Hope this would be helpful to those who also want to see what's going on during the procedure.