We Keep Coding

Python 'in' operator with 'for' loop and with 'if' statement

I am using python in operator with for loop and with if statement. My question is how is in implemented, that it behaves differently in this two cases: it iterates when using for loop and it checks if some element exists when using with if statement? Does this depend on implementation of for and if?
for i in x:
#iterates
if i in x:
#checks weather i in x or not

Membership testing with in is implemented via the method __contains__ (see Documentation). The different behaviour comes from the keyword before, for and if in your case.
Iteration with for is implemented such, that the method next is called and its return value is written to the iteration variable as long as the condition after the key word for is true. Membership testing in general is just a condition.
Code
A in B
Execution
B.__contains__(A) # returns boolean
Code
for A in B :
# Body
Execution
A = B.next()
if B.__contains__(A) :
# Body
A = B.next()
if B.__contains__(A) :
# Body
# ...
A = B.next()
if B.__contains__(A) :
# B.next() throws "StopIteration" if there is no more element

The keyword "in" in python solves different purposes based on "for" and "if". please look at this related link in stack overflow for more clarity

In many languages you'll find keywords that have multiple uses. This is simply an example of that. It's probably more helpful to think in terms of statements than thinking about the in keyword like an operator.
The statement x in y is a boolean-valued statement taking (assuming y is some appropriate collection) True if and only if the value of x is in the collection y. It is implemented with the __contains__ member function of y.
The statement for x in y: starts a loop, where each iteration x takes a different value from the collection y. This is implemented using the __iter__ member function of y and __next__ on the resulting iterator object.
There are other statements where the in keyword can appear, such as list comprehension or generator comprehension.

The reason is that for...in is something different from just in.
for x in y iterates over y.
if x in y calls y.__contains__(x).

The in keyword is an operator usually:
print(2 in [1, 2, 3]) # True
if 3 in range(7, 20):
print('3 in range!')
It corresponds to the object.__contains__ special method. The expression a in b corresponds to type(b).__contains__(a). Note that both a and b are names that are looked up.
In a for statement, in is not an operator. It is part of the for .. in .. syntax and separates the loop variable name from the iterable.
for thing in range(20):
print(thing) # thing is something else on each step
Note that for a in b only b is a name that is looked up. a is a name to bind to, similar to an assignment statement.
Python syntax has several constructs where the leading keyword defines the meaning of following keywords. For example, the as keyword has a different meaning in import and with:
# as aliases ExitStack
from contextlib import ExitStack as ContextStack
# as holds the result of ContextStack().__enter__()
with ContextStack() as stack:
...
It helps to think about such keywords not by implementation but by meaning. For example, a in b always means that "a is contained by b".

Order of evaluation of logical AND (python)

In python, which is more efficient:
if a:
if b:
# do something
or
if a and b:
# do something
The latter would be more efficient if b isn't calculated when a is false. But I can't seem to pinpoint whether that's the case in Python docs. Maybe someone can point me to it?

Short-Circuiting
Python doesn't run Y in X and Y if X is false. You can try it out yourself:
if True and print("Hello1"):
pass
if False and print("Hello2"):
pass # "Hello2" won't be printed
The reason Python does this is it has something called short-circuiting which optimising logic expressions like this one. Python realises that if if X is false then there is no point in checking Y because the whole expression will be false anyway.
How to Avoid Short-Circuiting
You can bypass this in Python by using the bitwise versions of the logic operators:
and -> &
or -> |
For example:
if True & bool(print("Hello1")):
pass
if False & bool(print("Hello2")):
pass # "Hello2" will be printed this time
Note however that you need to wrap print in bool because bitwise only works on the same data types.
Performance
I would imagine performance wise the one with only one if statement would be faster as other wise python might have to go through 2 if statements if it finds the conditions are true.

This is called short-circuiting, and it is mentioned in the docs.
And the answer is, Python will not evaluate b if a is False.

It evaluates from left to right, simple experiment
def iftrue1():
print("iftrue1")
return True
def iftrue2():
print("iftrue2")
return True
if iftrue1() and iftrue2():
pass
This outputs
iftrue1
iftrue2

python or on operator module [duplicate]

This question already has answers here:
Why doesn't the operator module have a function for logical or?
(3 answers)
Closed 6 years ago.
On the operator module, we have the or_ function, which is the bitwise or (|).
However I can't seem to find the logical or (or).
The documentation doesn't seem to list it.
I'm wondering why isn't it included? Is it not considered a operator?
Is there a builtin function that provides its behaviour?

The or operator short circuits; the right-hand expression is not evaluated when the left-hand returns a true value. This applies to the and operator as well; when the left-hand side expression returns a false value, the right-hand expression is not evaluated.
You could not do this with a function; all operands have to be evaluated before the function can be called. As such, there is no equivalent function for it in the operator module.
Compare:
foo = None
result = foo and foo(bar)
with
foo = None
result = operator.boolean_and(foo, foo(bar)) # hypothetical and implementation
The latter expression will fail, because you cannot use None as a callable. The first version works, because the and operator won't evaluate the foo(bar) expression.

The closest thing to a built-in or function is any:
>>> any((1, 2))
True
If you wanted to duplicate or's functionality of returning non-boolean operands, you could use next with a filter:
>>> next(operand for operand in (1, 2) if operand)
1
But like Martijn said, neither are true drop-in replacements for or because it short-circuits. A true or function would have to accept functions to be able to avoid evaluating all the results:
logical_or(lambda: 1, lambda: 2)
This is somewhat unwieldy, and would be inconsistent with the rest of the operator module, so it's probably best that it's left out and you use other explicit methods instead.

It's not possible:
This can explicitly be found in the docs:
The expression x or y first evaluates x; if x is true, its value is
returned; otherwise, y is evaluated and the resulting value is
returned.
It does not exist as an operator function because due to the language specification, it is impossible to implement because you cannot delay execution of a called argument when calling the function. Here is an example of or in action:
def foo():
return 'foo'
def bar():
raise RuntimeError
If bar is called, we get a Runtime error. And looking at the following line, we see that Python shortcuts the evaluation of the line, since foo returns a True-ish value.
>>> foo() or bar()
'foo'
We can approximate:
We can simulate this behavior by passing in uncalled functions, and then calling them inside our or function:
def my_or(*funcs):
for func in funcs:
call = func()
if call:
return call
return call
>>> my_or(foo, bar)
'foo'
But you cannot shortcut execution of called callables that are passed to a function:
>>> my_or(foo, bar())
Traceback (most recent call last):
File "<pyshell#28>", line 1, in <module>
like_or(foo, bar())
File "<pyshell#24>", line 2, in bar
raise RuntimeError
RuntimeError
So it would be improper to include such a function in the built-ins or standard library because users would expect an or function to work just as a boolean or test, which again, is impossible.

The reason you are getting 1 after executing >>> 1 or 2 is because 1 is true so the expression has been satisfied.
It might make more sense if you try executing >>> 0 or 2. It will return 2 because the first statement is 0 or false so the second statement gets evaluated. In this case 2 evaluates to a true boolean value.
The and and or operators evaluate the first value and then plug the result into the "AND and OR" truth tables. In the case of and the second value is only considered if the first evaluates to true. In the case of the or operator, if the first value evaluates to true the expression is true and can return, if it isn't the second value is evaluated.

How can I apply a or/and operation to all element in a python list with efficiency

I know that reduce and all/any can do the trick, but its performance is bad when the list is large.
For example:
Define a function including print to check whether the function has been executed
In [33]: def func(x):
....: print x
....: return bool(x)
....:
Pass operator.or_ as the reduce function
In [34]: import operator
In [35]: reduce(operator.or_, [func(1), func(0)])
1
0
Out[35]: True
Then we found that the second function has been executed even that the first function returns True.
If I use or operation directly, it will return immediately once it find that one of them return True.
In [36]: func(1) or func(0)
1
Out[36]: True
However, I can't do so if I have a large list.
Is there any elegant way to do that? Or I should what a for loop to check?
Update
The origin way I use for any is
In [26]: any([func(1), func(0)])
1
0
Out[26]: True
It did evaluate all the function.
Via #Martijn Pieters' answer, I now know that I might use it in the wrong way. Sorry for the unclearing.

any() is exactly what you need here, combined with a generator expression:
any(func(i) for i in big_list)
This will stop iterating for the first value where func(i) returns a true value. As soon as a True value is found, you've proven that there is a value that is true in the input sequence ("is there any value that is true?" -> yup, we found at least one).
For and, you'd use all() instead:
all(func(i) for i in big_list)
which will return False the moment a func(i) falsey value is found. If one false value is found, then you have proven that there is at least one value that is not true, so they cannot all be true.
Note that these two functions are given a generator expression:
(func(i) for i in big_list)
This is evaluated lazily; every time you ask for the next value of a generator expression, it'll evaluate the loop and execute the func(i) expression once. It will not produce the whole list at once, it'll produce items one by one.
Your reduce(operator.or_, [func(1), func(0)]) expression has to build the whole input list before it can call reduce(). The reduce() method will process the whole input list, it will not short-circuit, because it has no knowledge of what operation is being applied to the input values. You could give reduce() a generator expression as well, but it won't stop iterating once the outcome is set (on the first true value for or or the first false value for and), again because reduce() has no specialist knowledge of the operation being performed.

Adding to the other answer, the problem with this:
reduce(operator.or_, [func(1), func(0)])
Is that the arguments are always evaluated before a function gets called, since Python doesn't do lazy evaluation. Using an iterator (like in Martijn's answer) avoids this, since it generates the list as needed instead of all at once.

Using "and" in return

I'm taking Web Application Engineering course on Udacity. I noticed that the instructor use and operator in return statement in his validation method. And I didn't understand how it is possible to return 2 arguments. I think, it may be something like if statement. Could anyone explain what it actually is?
Here is the validation method:
USER_RE = re.compile(r"^[a-zA-Z0-9_-]{3,20}$")
def valid_username(username):
return username and USER_RE.match(username)
Thanks in advance.

The and operator evaluates whether both of its arguments are tru-ish, but in a slightly surprising way: First it examines its left argument. If it is truish, then it returns its right argument. If the left argument is falsish, then it returns the left argument.
So the last line in your code:
return username and USER_RE.match(username)
is the same as:
if username:
return USER_RE.match(username)
else:
return username
Strings like username are truish if they are not empty. The regex match function returns a truish match object if the pattern matches, and returns None, a falsish value, if it doesn't match.
The net result is that valid_username will return a truish value if username is not an empty string, and the username matches the given pattern.
Note the "and" here has nothing to do with returning two values, it's computing one value.

When you use a logical operator, it continues according to the rules, so with and, it evaluates the truthiness of the first statement and if it isn't truthy, it returns a non-truthy value (in the first case, '').
print repr("" and "THIS IS POST AND")
""
print "" or "THIS IS POST AND"
THIS IS POST AND
print None or "Something else"
Something else
Where this comes in handy is when you don't want to call a non-existent method on something like None (e.g. the length trait):
r = None
s = [1,2,3,4]
print r and len(r)
None
print s and len(s)
4
In the case you posted, the point is that you only want to check the username against the regular expression if the username is truthy.
It's important to note here that and, and or both short-circuit. So if you get something non-truthy, the function won't even evaluate the regular expression.

and is just a binary operator. return a and b is structurally the same as return a + b or return a * b, and works the same way. The expression is evaluated, then its result is given to return, which returns it to the caller. There is nothing at all special about using and in a return statement.
As explained very well by Ned's answer, the result of a and b is true-ish if both of a and b are true-ish, and false-ish otherwise. It also short-circuits if the left argument is false-ish, since that is sufficient to determine the result of the whole a and b expression.
In this case, an empty string would not match that regular expression, so the operation is redundant considered purely as logic. I strongly suspect though that it's being used here because username might be given the value None, which would cause the regular expression match to throw an exception. Using username and USER_RE.match(username) rather than just USER_RE.match(username) means that any false-ish value (including None) for username causes the function to return something false-ish without even attempting the regular-expression match.
But again, this has nothing to do with return, it's just how and works.