I am using python in operator with for loop and with if statement. My question is how is in implemented, that it behaves differently in this two cases: it iterates when using for loop and it checks if some element exists when using with if statement? Does this depend on implementation of for and if?
for i in x:
#iterates
if i in x:
#checks weather i in x or not
Membership testing with in is implemented via the method __contains__ (see Documentation). The different behaviour comes from the keyword before, for and if in your case.
Iteration with for is implemented such, that the method next is called and its return value is written to the iteration variable as long as the condition after the key word for is true. Membership testing in general is just a condition.
Code
A in B
Execution
B.__contains__(A) # returns boolean
Code
for A in B :
# Body
Execution
A = B.next()
if B.__contains__(A) :
# Body
A = B.next()
if B.__contains__(A) :
# Body
# ...
A = B.next()
if B.__contains__(A) :
# B.next() throws "StopIteration" if there is no more element
The keyword "in" in python solves different purposes based on "for" and "if". please look at this related link in stack overflow for more clarity
In many languages you'll find keywords that have multiple uses. This is simply an example of that. It's probably more helpful to think in terms of statements than thinking about the in keyword like an operator.
The statement x in y is a boolean-valued statement taking (assuming y is some appropriate collection) True if and only if the value of x is in the collection y. It is implemented with the __contains__ member function of y.
The statement for x in y: starts a loop, where each iteration x takes a different value from the collection y. This is implemented using the __iter__ member function of y and __next__ on the resulting iterator object.
There are other statements where the in keyword can appear, such as list comprehension or generator comprehension.
The reason is that for...in is something different from just in.
for x in y iterates over y.
if x in y calls y.__contains__(x).
The in keyword is an operator usually:
print(2 in [1, 2, 3]) # True
if 3 in range(7, 20):
print('3 in range!')
It corresponds to the object.__contains__ special method. The expression a in b corresponds to type(b).__contains__(a). Note that both a and b are names that are looked up.
In a for statement, in is not an operator. It is part of the for .. in .. syntax and separates the loop variable name from the iterable.
for thing in range(20):
print(thing) # thing is something else on each step
Note that for a in b only b is a name that is looked up. a is a name to bind to, similar to an assignment statement.
Python syntax has several constructs where the leading keyword defines the meaning of following keywords. For example, the as keyword has a different meaning in import and with:
# as aliases ExitStack
from contextlib import ExitStack as ContextStack
# as holds the result of ContextStack().__enter__()
with ContextStack() as stack:
...
It helps to think about such keywords not by implementation but by meaning. For example, a in b always means that "a is contained by b".
Related
As with regards to python, when it comes to a for loop, will it re-evaluate the upper bound before re-iteration?
Say I have the following scenario:
def remove(self,list,element):
for x in range(0,len(list)):
if somecondition:
list.pop(x)
Will the len(list) condition be re-evaluated before executing the next iteration of the for loop? (As done in some languages such as Objective-C I believe) As otherwise if a number of elements is popped, an out of bounds error would arise if say 1 element was removed, and the last iteration would try to access list[len(list)-1].
I've tried to investigate this myself however the results are muddled each time.
Edit: I believe that my question is different to the one flagged as a duplicate, as my question is regarding the condition for the loop to continue to the next iteration, it could easily ba adding an element instead of removing an element.
For clarification, my question asks whether or not the for loop condition will recheck the conditions posed before the next iteration.
The documentation about The for statement is clear about this:
The for statement is used to iterate over the elements of a sequence (such as a string, tuple or list) or other
iterable object:
for_stmt ::= "for" target_list "in" expression_list ":" suite
["else" ":" suite]
The expression list is evaluated once; it should yield an iterable object. An iterator is created for
the result of the expression_list. The suite is then executed once for
each item provided by the iterator, in the order returned by the
iterator.
[emphasis mine]
So, in your case, range(0,len(list)) will only be evaluated once.
Yes, you can see an out of range error if you try the following:
my_list = [0,1,2,3,4,5]
for x in range(0, len(my_list)):
print("Length of my_list:", len(my_list))
my_list.pop(x)
(You should also avoid using a variable name like list as it will shadow Python's built in list.)
Consider the following code:
i = 3
def test():
global i ;
i = i + 1
return i
for x in range(0,test()):
print( x, i )
Every time test() is callediis incremented, so the loop woul be endless iftest()` were evaluated on each iteration. The output however is :
0 4
1 4
2 4
3 4
Clearly therefore test() is evaluated once.
I wrote a python program to print the ascii value of up to every 3 numbers in a string or "ERROR" if the length is not divisible by three. I was golf the code when I ran into a SyntaxError.
Code:
c=input()
p=len(c)
c=[int(c[i:i+3])for i in range(0,len(c),3)]
print("ERROR"if p%3else*map(chr,c),sep='')#SyntaxError here
But this works:
c=input()
p=len(c)
c=[int(c[i:i+3])for i in range(0,len(c),3)]
print(*map(chr,c),sep='')
Putting a space before the * or after the 3 doesn't work. I could just use ''.join but it's one character longer. My question is why can't I use a starred expression in a ternary operator?
Because the * has to apply to the whole expression that produces the set of arguments, not a part of it, and not conditionally. Internally, CPython uses different bytecodes for calling with unpacking vs. normal calls, so your conditional would require it to change the byte code to call print based on the arguments to print, essentially rewriting what you wrote into pseudocode like:
if p % 3:
call(print, "ERROR", sep='')
else:
call_varargs(print, *map(chr, c), sep='')
which is beyond the compiler's capability.
If you wanted to make this work, you could do the following:
print(*(("ERROR",) if p%3 else map(chr,c)), sep='')
which ensures the whole ternary evaluates to an unpackable sequence and unpacks whatever survives unconditionally, avoiding the confusion.
print(*(["ERROR"] if p%3 else map(chr,c)),sep="!")
keep it outside of the ternary
The * expander transforms a single enumerable variable into individual variables. E.g.
li = [1,2,3]
print(*li)
produces: 1 2 3 instead of [1, 2, 3].
One value vs. multiple values
It appears to remove the brackets and pass a single string to print, but this is only an appearance, it actually replaces the single list variable by 3 variables and is actually equivalent to:
print(li[0], li[1], li[2])
It works because print accepts a variable number of arguments, so in our case it can deal with the single list or with these three integers.
The conditional expression is a one-value operator
However in your code you use the star operator within a conditional expression:
c = '065066067068'
p = len(c)
c = [int(c[i:i+3]) for i in range(0, p, 3)]
print('ERROR' if p%3 else *map(chr, c), sep='!')
print would be able to accept both evaluations of the expression, a single string value ('ERROR') or multiple char values from map.
But the conditional expression prevents returning multiple values according to the condition (p%3). The expression output has to be a single value. So you have no other choice than to return the list from map as an enumerable, and un-nest it only outside of the ternary operator, e.g. in the print statement.
A string is an enumerable, not a scalar value
However this solution now introduces another problem: Un-nesting will also convert the constant string ERROR into single chars, as a string is considered by Python an enumerable of single chars (as you know since you use this property for your input string). When the condition is true, the output would be:
E!R!R!O!R
Therefore the string must be first converted to an enumerable of strings, e.g. a tuple
Final solution
if p%3: s = ('ERROR',)
else: s = map(chr, c)
print(*s, sep='!')
The outputs will be:
A!B!C!D
ERROR
hey guys am new to python app development..i have been trying to fetch only numbers from a list using a for loop..But am confused with the correct syntax..The code i have been used.is like below.
babe = [10,11,13,'vv']
int(honey) [for honey in babe]:
print honey
When i run this i got syntax error.i have tried many situations.But it didnt helped me at all.Sorry for the silly question..
do i wanna add square brackets or something on the second line ??
Am really stuck.Hope you guys can help me out..Thanks in advance
You seem to be conflating the syntax for for loops (a statement followed by a suite of statements ... otherwise known as a "block of code") and a list comprehension (an expression).
Here's a list comprehension:
#!/usr/bin/python
# Given:
b = [1,2,3,'vv']
a = [int(x) for x in b]
... that's syntactically valid. However, the semantics of that example will raise an exception because 'vv' is not a valid literal (string). It cannot be interpreted as a decimal integer.
Here's a for loop:
#!/usr/bin/python
# Given:
b = [1,2,3,'vv']
a = list()
for x in b:
try:
a.append(int(x))
except ValueError:
pass
In this case we explicitly loop over the given list (b) and ignore any ValueError exceptions raised when we try to convert each of those entries into an integer.
There is no reasonable way to handle exceptions from within a list comprehension. You could write a function which returned some sentinel value (from the expression) for any invalid input value. That would look something like this:
#/usr/bin/python
# Given:
b = [1, 2, 3, 'vv']
def mk_integer_if_possible(n):
'''Returns an integer or the sentinel value None
'''
results = None
try:
results = int(n)
except ValueError:
pass
return results
# Use that function:
a = [mk_integer_if_possible(x) for x in b if mk_integer_if_possible(x) is not None]
Note: the absurd function name is deliberate. This is an ugly way to do this and the awkwardness of having to call this putative function TWICE for each element of b is an indication that you should NOT use a list comprehension for this situation. (You have to call it once to make the conversion but again for the conditional. Saving the results from one call would, of course, be a STATEMENT, which we can't have embedded within an EXPRESSION).
Statements contain one or more expressions. Expressions are components of statements. Python strictly delineates between statements and expressions. Assignments are statements in Python. These distinctions can be nuanced and there are other programming languages where assignments are expressions rather than being strictly defined, by the language's syntax, as statements.
So, use the for loop whenever you have to handle possible exceptions while iterating over any sort of data set and usually when you need to filter on the results generated by mapping a function over a list comprehension.
Incidentally the explicit use of the expression is not None is necessary in this example. If I attempted to shorten that test to simply be if mk_integer_if_possible(x) using Python's implicit boolean handling then we'd be inadvertently filtering out any entries from b that evaluated to integer 0 as well as any that were returned as the None sentinel by my ill-advised function.
In Python it's often fine to use implicit boolean values as conditions. None and False as well as any numerically zero value, any empty string or any sort of empty list, tuple or dictionary, are all treated as "false" in a boolean context. However, when dealing with sentinel values it's best to use the is operator and explicitly test for object identity. Otherwise you'll have corner cases where your condition might be matched by values other than your sentinel.
(Handy trick: if you ever come across the need to allow None through some sort of filter or pass it along, but you need some other sentinel ... just use sentinel = object() ... you can create (instantiate) a generic Pythonobject and use is to match it for your sentinel handling. That will be unique to your code and no other Python object or type will match it. Guaranteed).
By the way ... I should note that this code it technically not "fetching only numbers from a list." It is returning integers for all entries in the list which can be converted thereto. This is a nitpick; but it's a distinction that any good engineer will notice. Do you want to return all integers from the input list? Or do you want to return all entries as integers if that can be so converted? Your code suggested that you're trying to accomplish the latter; so that's how I implemented my working examples for you. However, to implement the later semantics you'd probably want to use either the (mathematical) additive or multiplicative identity property like so:
# ... from within some function:
try:
results = x == x + 0 # Additive identity
except (TypeError, ValueError):
results = None
return results
babe = [10,11,13,'vv']
a = [honey for honey in babe if isinstance(honey, int)]
print a
See more here about list comprehension: https://docs.python.org/2/tutorial/datastructures.html#list-comprehensions
I know that reduce and all/any can do the trick, but its performance is bad when the list is large.
For example:
Define a function including print to check whether the function has been executed
In [33]: def func(x):
....: print x
....: return bool(x)
....:
Pass operator.or_ as the reduce function
In [34]: import operator
In [35]: reduce(operator.or_, [func(1), func(0)])
1
0
Out[35]: True
Then we found that the second function has been executed even that the first function returns True.
If I use or operation directly, it will return immediately once it find that one of them return True.
In [36]: func(1) or func(0)
1
Out[36]: True
However, I can't do so if I have a large list.
Is there any elegant way to do that? Or I should what a for loop to check?
Update
The origin way I use for any is
In [26]: any([func(1), func(0)])
1
0
Out[26]: True
It did evaluate all the function.
Via #Martijn Pieters' answer, I now know that I might use it in the wrong way. Sorry for the unclearing.
any() is exactly what you need here, combined with a generator expression:
any(func(i) for i in big_list)
This will stop iterating for the first value where func(i) returns a true value. As soon as a True value is found, you've proven that there is a value that is true in the input sequence ("is there any value that is true?" -> yup, we found at least one).
For and, you'd use all() instead:
all(func(i) for i in big_list)
which will return False the moment a func(i) falsey value is found. If one false value is found, then you have proven that there is at least one value that is not true, so they cannot all be true.
Note that these two functions are given a generator expression:
(func(i) for i in big_list)
This is evaluated lazily; every time you ask for the next value of a generator expression, it'll evaluate the loop and execute the func(i) expression once. It will not produce the whole list at once, it'll produce items one by one.
Your reduce(operator.or_, [func(1), func(0)]) expression has to build the whole input list before it can call reduce(). The reduce() method will process the whole input list, it will not short-circuit, because it has no knowledge of what operation is being applied to the input values. You could give reduce() a generator expression as well, but it won't stop iterating once the outcome is set (on the first true value for or or the first false value for and), again because reduce() has no specialist knowledge of the operation being performed.
Adding to the other answer, the problem with this:
reduce(operator.or_, [func(1), func(0)])
Is that the arguments are always evaluated before a function gets called, since Python doesn't do lazy evaluation. Using an iterator (like in Martijn's answer) avoids this, since it generates the list as needed instead of all at once.
This question already has answers here:
Python: Is the split function evaluated multiple times in a list comprehension?
(3 answers)
Closed 7 years ago.
Which one's a better way of doing list comprehension in python (in terms of computation time & cpu cycles).
In example (1) is the value f(r) evaluated in each iteration or is it evaluated once and cached ?
y = [x*f(r) for x in xlist]
c = f(r)
y = [x*c for x in xlist]
where
def f(r):
... some arbitrary function ...
It evaluates for every iteration. Look at this:
>>> def f():
... print("func")
...
>>> [f() for i in range(4)]
func
func
func
func
[None, None, None, None]
As you say, if f() has no side effects, storing the return value on a variable and using that variable instead is a lot faster solution.
I would probably choose the latter because the Python compiler doesn't know if the function has side-effects so it is called for each element.
Here is an easy way to find out:
>>> def f():
... print "called"
... return 1
...
>>> [1+f() for x in xrange(5)]
called
called
called
called
called
[2, 2, 2, 2, 2]
so yes, if the function is going to be the same each time then it is better to call it once outside the list comprehension.
The function f will be called for every element.
It is very complex for the compiler/interpreter to determine that the function need not to be called many times. It is then very probable that the function is called many times. Thus, using the second solution is always the best solution.
Functions have a non-trivial execution time compared to a name lookup, and caching the value is considered acceptable if the function is called many times and the same value is expected each time.
Python is probably free to do it once or many times, I'm not sure I would rely on any observed behavior. It might change in the next version.
If it's important to you to make sure the function is only called once, call it yourself and save the results.