Why does list(next(iter(())) for _ in range(1)) return an empty list rather than raising StopIteration?
>>> next(iter(()))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
>>> [next(iter(())) for _ in range(1)]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
>>> list(next(iter(())) for _ in range(1)) # ?!
[]
The same thing happens with a custom function that explicitly raises StopIteration:
>>> def x():
... raise StopIteration
...
>>> x()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in x
StopIteration
>>> [x() for _ in range(1)]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in x
StopIteration
>>> list(x() for _ in range(1)) # ?!
[]
assuming all goes well, the generator comprehension x() for _ in range(1) should raise StopIteration when it is finished iterating over range(1) to indicate that there are no more items to pack into the list.
However because x() raises StopIteration it ends up exiting early meaning this behaviour is a bug in python that is being addressed with PEP 479
In python 3.6 or using from __future__ import generator_stop in python 3.5 when a StopIteration propagates out farther it is converted into a RuntimeError so that list doesn't register it as the end of the comprehension. When this is in effect the error looks like this:
Traceback (most recent call last):
File "/Users/Tadhg/Documents/codes/test.py", line 6, in <genexpr>
stuff = list(x() for _ in range(1))
File "/Users/Tadhg/Documents/codes/test.py", line 4, in x
raise StopIteration
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/Tadhg/Documents/codes/test.py", line 6, in <module>
stuff = list(x() for _ in range(1))
RuntimeError: generator raised StopIteration
The StopIteration exception is used to tell the underlying mechanism of the list function when to actually stop iterating on the iterable that has been passed to it. In your case, you're telling Python that the thing that has been passed into list() is a generator. So when the generator throws a StopIteration before generating any items, it outputs an empty list because nothing has been accumulated.
Related
def prefixes(s):
if s:
yield from prefixes(s[:-1])
yield s
t = prefixes('both')
next(t)
The next(t) returns 'b'. I'm just confused as to why this is because if we follow down the yield from statement, we will eventually end at yield from prefixes('') which would return None. In all my other tests yield from None raises a TypeError. Instead, this seems to just be ignored and prefixes('b') moves onto the next yield statement (? why does it do that?) to yield 'b'...
Any ideas as to why? Would greatly appreciate an explanation.
prefixes is wrapped in a generator that raises StopIteration when the function returns. When passed an empty string, prefixes skips any yields, reaches the end of its code block and returns, causing StopIteration. The return value doesn't matter, it is discarded
>>> next(prefixes(""))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
yield from suppresses the inner generator's StopIteration and lets the outer generator continue.
generators are lazy(on-demand) objects, you didn't exhaust your generator t, to exhaust your generator you can use:
list(t)
# ['b', 'bo', 'bot', 'both']
now if you use next(t) you will get the expected StopIteration
StopIteration Traceback (most recent call last)
<ipython-input-25-680ef78477e2> in <module>
6 t = prefixes('both')
7 list(t)
----> 8 next(t)
StopIteration:
the if statement is "guaranteeing" that you have an end and you will never do None[:-1] to get the TypeError
I know what are complex numbers and how they mathematically work, but how is it done for python to know it's complex just by putting a j after a digit ?
>>> j
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'j' is not defined
>>> 1*j
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'j' is not defined
>>> 1j
1j
>>> 1j**2
(-1+0j)
Can I make my own suffix, let's say p (for strictly positive) ?
Could I do something working like this ?
>>> ... some weird stuff here ...
>>> p
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'p' is not defined
>>> 1*p
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'p' is not defined
>>> -1p
1p
>>> 0p
1p
>>>
This is built into Python's grammar, just like the decimal point is, or the e in scientific notation (1e10 etc.). The j makes a numeric literal imaginary.
Python does not allow you to change this. That doesn't mean you can't--you could amend the grammar--but then the language is no longer Python.
The closest approximation allowed in Python would be by implementing an operator.
>>> class StrictlyPositive:
def __rmatmul__(self, number):
return abs(number)
>>> p = StrictlyPositive()
>>> -1#p
1
But you have to be careful of operator precedence when doing stuff like this. Why not just use the builtin abs directly?
I have a function with returns a tuple:
def gradiant(params, data):
Q1 = params[0]
Q2 = params[1]
dQ1 = sum(2*(y_i - Q1*x_i - Q2)*(x_i) for x_i, y_i in data)
dQ2 = sum(-2*(y-i - Q1*x_i - Q2) for x_i, y_i in data)
return (dQ1, dQ2)
I'm trying to print the returned values:
grad = gradiant(params, Data)
for x in grad: print(x)
Output:
<generator object <genexpr> at 0x7fdfe009adc0>
<generator object <genexpr> at 0x7fdfe009a5f0>
How do I get to print the integer values?
You have a y-i instead of y_i. It produced a NameError for me. I fixed the typo and it worked fine (I assume, anyway - it printed two numbers).
However, the more interesting issue is how errors are handled within generator expressions (as the typo was part of the generator expression you sent to sum()). I got a NameError, but you got generator expression objects. Look at this function call with an unpacked generator that should contain a TypeError:
>>> print(*(''.join(item2) for item2 in range(3)))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: print() argument after * must be a sequence, not generator
Now let's see what we get by printing the actual object:
>>> print(''.join(item2) for item2 in range(3))
<generator object <genexpr> at 0x00000000028945E8>
That makes some sense, as generators are lazily evaluated and it was never actually called. But let's see what happens when we send it to list(), which should force evaluation:
>>> print(list(''.join(item2) for item2 in range(3)))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 1, in <genexpr>
TypeError: can only join an iterable
There's the expected error.
How about a valid generator that contains an invalid generator?
>>> print(*(list(''.join(item2) for item2 in range(3)) for i in range(1)))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: print() argument after * must be a sequence, not generator
The same uninformative error.
What if we use a list comprehension instead of a generator expression?
>>> print(*[''.join(item2) for item2 in range(3)])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 1, in <listcomp>
TypeError: can only join an iterable
We get the informative error we were expecting.
Unfortunately, I don't know why this happens, but be extra-careful with generator expressions.
I see from other discussions that reload is considered an unnecessary operation and a bad way to develop programmes. People say to use doctest and unittest. I must be missing something. To develop a programme I write it in a module, load the module into an eclipse console, experiment in the console by running little samples of code, then I spot an error in the module or decide to change something. Surely the quickest thing to do is save the module, reload it and carry on working in the console.
Is there a better way?
Basically instead of trying things out in the console, try things out by writing tests. It's almost the same amount of typing, but repeatable (you can check if things still work after you make changes) and it works as a rudimentary form of documentation.
Here's a contrived example of how you could use doctest and test driven development in comparision with testing by entering code at the console, with a recursive factorial function being used as an example:
Console:
First attempt:
def factorial(x)
pass
Console:
>>> factorial(7)
SyntaxError: invalid syntax
Second attempt:
def factorial(x):
return x*x-1
Console:
>>> factorial(7)
42
Third attempt:
def factorial(x):
return x * factorial(x-1)
Console:
>>> factorial(7)
RuntimeError: maximum recursion depth reached # StackOverflow! (basically)
Fourth attempt:
def factorial(x):
if x == 0:
return 1
else:
return x * factorial(x-1)
Console:
>>> factorial(5)
120
In the end, we get the right answer, but at each stage we have to go back to the console and write the same thing. This is alright for a short program like this, but for a program with many functions, and more complex interactions and possibilities to be tested it would take a very long time. Programming is about automating repetitive tasks, and testing is a repetitive task, so it makes sense to automate it. And in Python, tools to do just this are provided for you.
Behold - the doctest module: (example taken from the docs)
First attempt:
def factorial(n):
"""Return the factorial of n, an exact integer >= 0.
>>> [factorial(n) for n in range(6)]
[1, 1, 2, 6, 24, 120]
>>> factorial(30)
265252859812191058636308480000000
>>> factorial(-1)
Traceback (most recent call last):
...
ValueError: n must be >= 0
Factorials of floats are OK, but the float must be an exact integer:
>>> factorial(30.1)
Traceback (most recent call last):
...
ValueError: n must be exact integer
>>> factorial(30.0)
265252859812191058636308480000000
It must also not be ridiculously large:
>>> factorial(1e100)
Traceback (most recent call last):
...
OverflowError: n too large
"""
if __name__ == "__main__":
import doctest
doctest.testmod(verbose=True)
First test (just run the program):
Trying:
[factorial(n) for n in range(6)]
Expecting:
[1, 1, 2, 6, 24, 120]
ok
Trying:
factorial(30)
Expecting:
265252859812191058636308480000000
ok
Trying:
factorial(-1)
Expecting:
Traceback (most recent call last):
...
ValueError: n must be >= 0
ok
Trying:
factorial(30.1)
Expecting:
Traceqwrqaeqrback (most recent call last):
...
ValueError: n must be exact integer
**********************************************************************
File "C:/Python33/doctestex.py", line 14, in __main__.factorial
Failed example:
factorial(30.1)
Exception raised:
Traceback (most recent call last):
File "C:\Python33\lib\doctest.py", line 1287, in __run
compileflags, 1), test.globs)
File "<doctest __main__.factorial[3]>", line 1, in <module>
factorial(30.1)
File "C:/Python33/doctestex.py", line 32, in factorial
raise ValueError("n must be exact integer")
ValueError: n must be exact integer
Trying:
factorial(30.0)
Expecting:
265252859812191058636308480000000
ok
Trying:
factorial(1e100)
Expecting:
Traceback (most recent call last):
...
OverflowError: n too large
ok
1 items had no tests:
__main__
**********************************************************************
1 items had failures:
1 of 6 in __main__.factorial
6 tests in 2 items.
5 passed and 1 failed.
***Test Failed*** 1 failures.
>>> ================================ RESTART ================================
>>>
Trying:
[factorial(n) for n in range(6)]
Expecting:
[1, 1, 2, 6, 24, 120]
**********************************************************************
File "C:/Python33/doctestex.py", line 4, in __main__.factorial
Failed example:
[factorial(n) for n in range(6)]
Expected:
[1, 1, 2, 6, 24, 120]
Got:
[None, None, None, None, None, None]
Trying:
factorial(30)
Expecting:
265252859812191058636308480000000
**********************************************************************
File "C:/Python33/doctestex.py", line 6, in __main__.factorial
Failed example:
factorial(30)
Expected:
265252859812191058636308480000000
Got nothing
Trying:
factorial(-1)
Expecting:
Traceback (most recent call last):
...
ValueError: n must be >= 0
**********************************************************************
File "C:/Python33/doctestex.py", line 8, in __main__.factorial
Failed example:
factorial(-1)
Expected:
Traceback (most recent call last):
...
ValueError: n must be >= 0
Got nothing
Trying:
factorial(30.1)
Expecting:
Traceback (most recent call last):
...
ValueError: n must be exact integer
**********************************************************************
File "C:/Python33/doctestex.py", line 14, in __main__.factorial
Failed example:
factorial(30.1)
Expected:
Traceback (most recent call last):
...
ValueError: n must be exact integer
Got nothing
Trying:
factorial(30.0)
Expecting:
265252859812191058636308480000000
**********************************************************************
File "C:/Python33/doctestex.py", line 18, in __main__.factorial
Failed example:
factorial(30.0)
Expected:
265252859812191058636308480000000
Got nothing
Trying:
factorial(1e100)
Expecting:
Traceback (most recent call last):
...
OverflowError: n too large
**********************************************************************
File "C:/Python33/doctestex.py", line 22, in __main__.factorial
Failed example:
factorial(1e100)
Expected:
Traceback (most recent call last):
...
OverflowError: n too large
Got nothing
1 items had no tests:
__main__
**********************************************************************
1 items had failures:
6 of 6 in __main__.factorial
6 tests in 2 items.
0 passed and 6 failed.
***Test Failed*** 6 failures.
Then when we finish the program (and at all stages between) we just press one button and that does all our testing for us. If you were working with a team on a bigger project, you could, for instance, write the test data to file and use the failures to see where you need to focus on.
Trying:
[factorial(n) for n in range(6)]
Expecting:
[1, 1, 2, 6, 24, 120]
ok
Trying:
factorial(30)
Expecting:
265252859812191058636308480000000
ok
Trying:
factorial(-1)
Expecting:
Traceback (most recent call last):
...
ValueError: n must be >= 0
ok
Trying:
factorial(30.1)
Expecting:
Traceback (most recent call last):
...
ValueError: n must be exact integer
ok
Trying:
factorial(30.0)
Expecting:
265252859812191058636308480000000
ok
Trying:
factorial(1e100)
Expecting:
Traceback (most recent call last):
...
OverflowError: n too large
ok
1 items had no tests:
__main__
1 items passed all tests:
6 tests in __main__.factorial
6 tests in 2 items.
6 passed and 0 failed.
Test passed.
And (provided you added all the test cases to the docstring) now you can ship or integrate or whatever the program and it much fewer bugs than it would've if you just tested for the things you thought mattered each time you changed something, rather than the things you thought could possibly ever matter at the start of the development process.
Not to mention you now have the basis for your documentation! Running the program in the console, and then entering help(factorial) will now give you this:
Help on function factorial in module __main__:
factorial(n)
Return the factorial of n, an exact integer >= 0.
>>> [factorial(n) for n in range(6)]
[1, 1, 2, 6, 24, 120]
>>> factorial(30)
265252859812191058636308480000000
>>> factorial(-1)
Traceback (most recent call last):
...
ValueError: n must be >= 0
Factorials of floats are OK, but the float must be an exact integer:
>>> factorial(30.1)
Traceback (most recent call last):
...
ValueError: n must be exact integer
>>> factorial(30.0)
265252859812191058636308480000000
It must also not be ridiculously large:
>>> factorial(1e100)
Traceback (most recent call last):
...
OverflowError: n too large
You can then use a number of tools (pydoc is one in the standard library) to turn the docstrings into formatted HTML helpfiles.
Of course, this is only one of many testing tools you can use with Python. Others include the more powerful unittest module, and the less powerful technique of adding assert statements into your code.
I am a beginning python programmer, but have written several scripts including ones in which I define my own functions and use them. I cannot seem to get any user defined functions to work within the IDLE. Wondering if I am crazy/dumb. Can somebody please explain the following results?Thanks:
def f(x,y):
solution = x+y
return solution
f(2,2)
SyntaxError: invalid syntax
>>> a = f(2,2)
Traceback (most recent call last):
File "<pyshell#4>", line 1, in <module>
a = f(2,2)
NameError: name 'f' is not defined
def g(x):
solution = x + 2
return solution
g(2)
SyntaxError: invalid syntax
>>> a = g(2)
Traceback (most recent call last):
File "<pyshell#11>", line 1, in <module>
a = g(2)
NameError: name 'g' is not defined
Add a blank line after the definition of the function to make the interpreter understand that it's finished.
>>> def f(x,y):
solution = x+y
return solution
>>> f(2,2)
4