I am trying to make a function that yields a value. Before anything could be done, I want to make sure function input is valid. Below code creates generator after execution. It raises exception only after next. Is there an elegant structure of the function which throws exception before next?
def foo(value):
if validate(value):
raise ValueError
yield 1
you can't check the value before using next, that is the whole point of using gnerators, from the docs:
Each yield temporarily suspends processing, remembering the location
execution state (including local variables and pending
try-statements). When the generator iterator resumes, it picks up
where it left off (in contrast to functions which start fresh on every
invocation).
what you can do it is to check the value before using the generator
Related
I am creating a generator in python 3 which may yield a single value or more.
The condition that I wanted is, I want to loop with this iterator starting at the second value and so on, running an API request function with that value. If the generator yield only a single value, the for loop and corresponding code is not needed to be executed. If the generator yield more than one value, the function inside the for-loop will be executed starting from the second value of generator and so on.
The reason why I want to start at the second value is because the first value is already accessed for the API request and its result has been stored.
My question is related to a generator that produce a single value.
I give the code example below: (I simplified API Request with print() function):
def iterexample(): # creating a simple iterator that return a single value
yield 0
print(0)
iter = iterexample()
next(iter) #generator is consumed once here
for i in iter: #1 generator is exhausted
print(i, ' inside loop') #2 this is skipped because the generator is exhausted
#3 rest of the code outside the loop will be executed
It returns what I expected: only 0 is printed, not "0 inside loop"
0
My question is:
Is it the safest and the most pythonic way to do that? will it raise
any error?
Will it produce infinite loop? I am very afraid if it will result as
infinite loop of API request.
Please review my #1 ~ #3 comment in above codes, are my
understanding correct?
Thanks for the response and the help. Cheers!
1 Is it the safest and the most pythonic way to do that? will it raise any error?
Once a generator is exhausted, it will continually raise StopIteration exceptions when asked for new values. For loops can handle this case by terminating the loop when this exception is raised, which makes it safe to pass an exhausted generator to a for loop constructor.
However, your code calls next directly, and is therefore only safe only if it also handle StopIteration exceptions. In this case you would need to document that the generator provided must produce 1 or more values or be tolerant of the empty case. If the generator returned no values, then you would get an error. e.g.
def iterexample():
while False:
yield 0
print(next(iterexample()))
Traceback (most recent call last):
File "test.py", line 5, in <module>
print(next(iterexample()))
StopIteration
To prevent against empty generators you can use the second optional default argument to next.
print(next(iterexample(), "default"))
default
2 Will it produce infinite loop? I am very afraid if it will result as infinite loop of API request.
Again this depends on the generator. Generators do not need to have an end value. You can easily define non-ending generators like this:
def iterexample():
i = 0
while True:
yield i
i += 1
for i in iterexample(): #This never ends.
print(i)
If this is a concern for you, one way to prevent never ending outputs would be to use an islice that cuts off your generator after so many values are consumed:
from itertools import islice
for i in islice(iterexample(), 5):
print(i)
0
1
2
3
4
If I understand correctly your issue: you have a first value that you need for a case, and the rest for another case.
I would recommend building a structure that fits your needs, something like this:
class MyStructrue:
def __init__(self, initial_data):
if not initial_data:
# Make sure your data structure is valid before using it
raise ValueErro("initial_data is empty")
self.initial_data = initial_data
#property
def cached_value(self):
return self.initial_data[0]
#property
def all_but_first(self):
return self.initial_data[1:]
In this case, you make sure your data is valid, and you can give your accessors names that reflects what you those value are representing. In this example, I gave them dummy names, but you should try to make something that is relevant to your business.
Such a class could be used this way (changed names just to illustrate how method naming can document your code):
tasks = TaskQueue(get_input_in_some_way())
advance_task_status(tasks.current_task)
for pending_task in tasks.pending_tasks:
log_remaining_time(pending_tasks)
You should first try to understand what your datastructure represents and build a useful api that hide the implementation to better reflect your business.
Question
Please help pin-point the Python source code that implements the generator send part. I suppose somewhere in github This is Python version 3.8.7rc1 but not familiar with how the repository is organized.
Background
Having difficulty with what PEP 342 and documentation regarding the generator send(value). Hence trying to find out how it is implemented to understand.
there is no yield expression to receive a value when the generator has just been created
The value argument becomes the result of the **current yield expression**. The send() method returns the **next value yielded by the generator
Specification: Sending Values into Generators
Because generator-iterators begin execution at the top of the
generator's function body, there is no yield expression to receive a
value when the generator has just been created. Therefore, calling
send() with a non-None argument is prohibited when the generator
iterator has just started, and a TypeError is raised if this occurs
(presumably due to a logic error of some kind). Thus, before you can
communicate with a coroutine you must first call next() or send(None)
to advance its execution to the first yield expression.
generator.send(value)
Resumes the execution and “sends” a value into the generator function.
The value argument becomes the result of the current yield expression.
The send() method returns the next value yielded by the generator, or
raises StopIteration if the generator exits without yielding another
value. When send() is called to start the generator, it must be called
with None as the argument, because there is no yield expression that
could receive the value.
I suppose yield would be like a UNIX system call moving into a routine, inside which stack frame and execution pointer are saved and the generator co-routine is suspended. I think when save(value) is called, some tricks happen there and those are regarding the cryptic parts in the documents.
Although sent_value = (yield value) is one line statement, the blocking and resuming both happens in the same line, I think. The execution does not resume after the yield but within it, hence would like to know how block/resume are implemented. Also I believe next(generator) is the same with generator.send(None) and would like to verify.
Look here for class Generator, also look for this file, is a complete implementation of generators on C inside Python
Do I have to write
def count10():
for i in range(10):
yield i
gen = count10()
for j in gen:
print(j)
gen.close()
to save memory, or just
def count10():
for i in range(10):
yield i
for j in count10():
print(j)
In fact I would like to learn details of lifecycle of Python generator but failed to find relevant resources.
You don't need to close that generator.
close-ing a generator isn't about saving memory. (close-ing things is almost never about saving memory.) The idea behind the close method on a generator is that you might stop iterating over a generator while it's still in the middle of a try or with:
def gen():
with something_important():
yield from range(10)
for i in gen():
if i == 5:
break
close-ing a suspended generator throws a GeneratorExit exception into the generator, with the intent of triggering finally blocks and context manager __exit__ methods. Here, close would cause the generator to run the __exit__ method of something_important(). If you don't abandon a generator in the middle like this (or if your generator doesn't have any finally or with blocks, including in generators it delegates to with yield from), then close is unnecessary (and does nothing).
The memory management system usually runs close for you, but to really ensure prompt closure across Python implementations, you'd have to replace code like
for thing in gen():
...
with
with contextlib.closing(gen()) as generator:
for thing in generator:
...
I've never seen anyone do this.
The close method for generators came about in PEP 342:
Add a close() method for generator-iterators, which raises GeneratorExit at the point where the generator was paused. If the generator then raises StopIteration (by exiting normally, or due to already being closed) or GeneratorExit (by not catching the exception), close() returns to its caller. If the generator yields a value, a RuntimeError is raised. If the generator raises any other exception, it is propagated to the caller. close() does nothing if the generator has already exited due to an exception or normal exit.
Note the last sentence: close() does nothing if the generator has already exited due to an exception or normal exit.
I am facing a strange behavior with nested generators.
def empty_generator():
for i in []:
yield
def gen():
next(empty_generator())
print("This is not printed, why?")
yield
list(gen()) # No Error
next(empty_generator()) # Error
I would expect the gen() function to raises an error, as I am calling next() around an empty generator. But this is not the case, the functions is leaving from nowhere, without raising or printing anything.
That seems to violate the principle of least astonishment, isn't it?
Technically, you don't have an error; you have an uncaught StopIteration exception, which is used for flow control. The call to list, which takes an arbitrary iterable as its argument, catches the exception raised by gen for you.
for loops work similarly; every iterator raises StopIteration at the end, but the for loop catches it and ends in response.
Put another way, the consumer of an iterable is responsible for catching StopIteration. When gen calls next, it lets the exception bubble up. The call to list catches it, but you don't when you call next explicitly.
Note that PEP-479 changes this behavior. Python 3.5 provides the new semantics via __future__, Python 3.6 makes provides a deprecation warning, and Python 3.7 (due out Summer 2018) completes the transition. I refer the reader to the PEP itself for further details.
Once an iterator reaches its end, it raises StopIteration which... stops the iteration, so list(gen()) constructs an empty list.
In python is there a way to not return to the caller function if a certain event happened in the called function. For example,...
def acquire_image(sdkobject):
ret = sdkobject.PrepareAcquisition()
error_check(ret)
ret = sdkobject.StartAcquisition()
error_check(ret)
error_check is a function that checks the return code to see if the sdk call had an error. If it is an error message then I would like to not go back to acquire and image but go to another function to reinitalise the sdk and start from the beginning again. Is there a pythonic way of doing this?
Have your error_check function raise an exception (like SDKError) if there is a problem, then run all the commands in a while loop.
class SDKError(Exception):
pass
# Perhaps define a separate exception for each possible
# error code, and make a dict that maps error codes to the
# appropriate exception.
class SDKType1Error(SDKError):
pass
class SDKType5Error(SDKError):
pass
sdk_errors = {
1: SDKType1Error,
5: SDKType5Error,
}
# Either return, if there was no error, or raise
# the appropriate exception
def error_check(return_code):
if return_code == 0:
return # No error
else:
raise sdk_errors[return_code]
# Example of how to deal with specific SDKErrors subclasses, or a generic
# catch-all SDKError
def acquire_image(sdkobject):
while True:
try:
# initialize sdk here
error_check(sdkobject.PrepareAcquisition())
error_check(sdkobject.StartAcquisition())
except SDKType1Error:
# Special handling for this error
except SDKError:
pass
else:
break
Return the error and use an if condition to check if the returned value has error, and if it has, call the reinitialization code from the calling function.
Use return for happy scenario
Returning to calling function is done by simple return or return response.
Use it for solving typical run of your code, when all goes well.
Throw exception, when something goes wrong
If something goes wrong, call raise Exception(). In many situations, your code does not has to do it explicitly, it throws the exception on its own.
You may even you your own Exception instances and use them to pass to the caller more information about what went wrong.
It took me a while to learn this approach and it made my coding much simpler and shorter then.
Do not care about what will your calling code do with it
Let your function do the task or fail, if there are problems.
Trying to think for client responsibility in your function will mess up your code and will not be complete solution anyway.
Things to avoid
Ignore who is calling you
In OOP this is principle of client anonymity. Just serve the request and do not care, who is calling.
Do not attempt using Exceptions as replacement for returning a value
Sometime, people use the fact, Exception can pass some information to to caller. But this is rather antipattern (there are always exception.)