I have tried with this code to implement iterator protocol, and every thing works fine.
Ideally it shouldn't as I have not implemeted iter().
class PowerofTwo():
def __init__(self,maxi):
self.max = maxi
self.num = 0
def __next__(self):
if self.num < self.max:
result = 2 ** self.num
self.num += 1
return result
else:
raise StopIteration
myObj = PowerofTwo(5)
print(type(myObj)) #print <class '__main__.PowerofTwo'>
print(next(myObj)) #print 1
Even if assume that __iter__() method is available by default like __init__().
But not able to understand how can I directly call next() and skip calling iter(obj) at first place.
If I try same thing with list or other builtin iterator, I gets error that object is not an iterator.
Am I missing something basic here ?
Currently, PowerofTwo is an iterator, but it is not an iterable. To make it iterable, you need to define __iter__ to return an object that has a __next__ method. In this case, the instance of PowerofTwo itself will do.
class PowerofTwo:
def __init__(self,maxi):
self.max = maxi
self.num = 0
def __iter__(self):
return self
def __next__(self):
if self.num < self.max:
result = 2 ** self.num
self.num += 1
return result
else:
raise StopIteration
for x in PowerofTwo(5):
print(x)
outputs
1
2
4
8
16
It is considered good practice to always define
def __iter__(self):
return self
in a class that defines __next__.
list is an example of a class whose __iter__ method returns an instance of another class, namely list_iterator.
Related
I have got the following class:
class RandomNumbers:
def __init__(self, length, *, range_min=0, range_max=10):
self.length = length
self.range_min = range_min
self.range_max = range_max
self.i = 0
def __len__(self):
return self.length
def __next__(self):
if self.i >= self.length:
raise StopIteration
number = random.randint(self.range_min, self.range_max)
self.i += 1
return number
def __iter__(self):
print("I was called")
return self
This allows me to use a for-loop:
for number in RandomNumbers(10):
print(number)
This works. When I comment out __iter__, I get the following error:
TypeError: 'RandomNumbers' object is not iterable
Ok, so far so good. I understand that I need the __iter__ method, but why is it needed when it actually only returns self?
Your for loop could be rewritten as this somewhat equivalent code
it = iter(RandomNumbers(10))
while True:
try:
number = next(it)
except StopIteration:
break
else:
# Body of the for loop
print(number)
Your class needs to implement the __iter__ method to respond to the iter() function with an iterator, in your case self since your class implements __next__ making it an iterator
Can you create an iterable in python which runs clean up code when for loop exits? Something like:
from random import randint
class Iterable:
def __iter__(self):
return self
def __next__(self):
return randint(1, 10)
def __iterclose__(self):
print("Clean up code")
for x in Iterable():
if x < 5:
break
# Prints "Clean up code"
I think in order to benefit from the iterable protocol, you should let your Iterable gain control about the stop condition. How about this:
from random import randint
class Iterable:
def __init__(self, max):
self.i = 0
self.max = max
def __iter__(self):
return self
def __next__(self):
self.i += 1
if self.i > self.max:
self.cleanup()
raise StopIteration
return randint(1, 10)
def cleanup(self):
print("Clean up code")
for x in Iterable(5):
print(x)
As I said in the comment :
When finishing iteration, you should raise the StopIteration exception, at this point you should already know that you have to perform the clean up, else this iterator will run forever.
Also iterator and iterable are different things, from fluent python book:
iterator
Any object that implements the next no-argument method that
returns the next item in a series or raises StopIteration when there
are no more items. Python iterators also implement the iter method
so they are iterable as well
Where:
iterable Any object from which the iter built-in function can obtain
an iterator. Objects implementing an iter method returning an
iterator are iterable.
So iterator should know when it will end, Hopefully this code will clear things up:
from random import randint
class MyIterator:
def __init__(self, n):
self.n = n
self.i = 0
def __iter__(self):
return self
def __next__(self):
if self.i < self.n:
self.i += 1
return randint(1, 10)
else:
print("doing cleanup")
#do the clean up you want
raise StopIteration()
class MyIterable:
def __init__(self, n):
self.n = n
def __iter__(self):
return MyIterator(self.n)
for x in MyIterable(5):
print(x)
Output:
4
9
7
5
1
doing cleanup
Note: It is better to do things here using generator function or expression, but this will make things more distinguishable.
I am using yield to return the next value in the __next__() function in my class. However it does not return the next value, it returns the generator object.
I am trying to better understand iterators and yield. I might be doing it in the wrong way.
Have a look.
class MyString:
def __init__(self,s):
self.s=s
def __iter__(self):
return self
def __next__(self):
for i in range(len(self.s)):
yield(self.s[i])
r=MyString("abc")
i=iter(r)
print(next(i))
This returns:
generator object __next__ at 0x032C05A0
next pretty much just calls __next__() in this case. Calling __next__ on your object will start the generator and return it (no magic is done at this point).
In this case, you might be able to get away with not defining __next__ at all:
class MyString:
def __init__(self,s):
self.s=s
def __iter__(self):
for i in range(len(self.s)):
yield(self.s[i])
# Or...
# for item in self.s:
# yield item
If you wanted to use __iter__ and __next__ (to define an iterator rather than simply making an iterable), you'd probably want to do something like this:
class MyString:
def __init__(self,s):
self.s = s
self._ix = None
def __iter__(self):
return self
def __next__(self):
if self._ix is None:
self._ix = 0
try:
item = self.s[self._ix]
except IndexError:
# Possibly reset `self._ix`?
raise StopIteration
self._ix += 1
return item
Let's take a look at the purpose of the __next__ method. From the docs:
iterator.__next__()
Return the next item from the container. If there are no further items, raise the StopIteration exception.
Now let's see what the yield statement does. Another excerpt from the docs:
Using a yield expression in a function’s body causes that function to
be a generator
And
When a generator function is called, it returns an iterator known as a
generator.
Now compare __next__ and yield: __next__ returns the next item from the container. But a function containing the yield keyword returns an iterator. Consequently, using yield in a __next__ method results in an iterator that yields iterators.
If you want to use yield to make your class iterable, do it in the __iter__ method:
class MyString:
def __init__(self, s):
self.s = s
def __iter__(self):
for s in self.s:
yield s
The __iter__ method is supposed to return an iterator - and the yield keyword makes it do exactly that.
For completeness, here is how you would implement an iterator with a __next__ method. You have to keep track of the state of the iteration, and return the corresponding value. The easiest solution is probably to increment an index every time __next__ is called:
class MyString:
def __init__(self,s):
self.s = s
self.index = -1
def __iter__(self):
return self
def __next__(self):
self.index += 1
if self.index >= len(self.s):
raise StopIteration
return self.s[self.index]
As far as I can tell, generator functions are just syntactic sugar for classes with a next function. Example:
>>> def f():
i = 0
while True:
i += 1
yield i
>>> x = f()
>>> x
<generator object f at 0x0000000000659938>
>>> next(x)
1
>>> next(x)
2
>>> next(x)
3
>>> class g(object):
def __init__(self):
self.i = 0
def __next__(self):
self.i += 1
return self.i
>>> y = g()
>>> y
<__main__.g object at 0x000000000345D908>
>>> next(y)
1
>>> next(y)
2
>>> next(y)
3
In fact, I came here looking to see if there is any significant difference. Please shout if there is.
So, to answer the question, what you have is a class with a __next__ method that returns an object that also has a __next__ method. So the simplest thing to do would be to replace your yield with a return and to keep track of how far along you are, and to remember to raise a StopIteration when you reach the end of the array. So something like:
class MyString:
def __init__(self,s):
self.s=s
self._i = -1
def __iter__(self):
return self
def __next__(self):
self._i += 1
if self._i >= len(self.s):
raise StopIteration
return self.s[self._i]
That's probably the simplest way to achieve what I think you're looking for.
OBSERVATION
If next() function calls __next__() method, what's happening in the following example.
Code:
class T:
def __init__(self):
self.s = 10
def __iter__(self):
for i in range(self.s):
yield i
def __next__(self):
print('__next__ method is called.')
if __name__== '__main__':
obj = T()
k = iter(obj)
print(next(k)) #0
print(next(k)) #1
print(next(k)) #2
print(next(k)) #3
print(next(k)) #4
print(next(k)) #5
print(next(k)) #6
print(next(k)) #7
print(next(k)) #8
print(next(k)) #9
print(next(k))
print(next(k))
Terminal:
C:...>python test.py
0
1
2
3
4
5
6
7
8
9
Traceback (most recent call last):
File "test.py", line 25, in <module>
print(next(k))
StopIteration
WHAT IS HAPPENING?
It seams that next() function does not calling __next__ method. I cannot understand why python docs states that "next(iterator, default) Retrieve the next item from the iterator by calling its __next__() method." If someonw knows, let us help!
Case: __iter__ with __next__ in custom class with yield
So, if you want to use yield (in order to create a generator) with __iter__ and __next__ methods in a custom class, do not put just the yield into the __next__ method, but use it with __iter__(self) and return self.__next__() instead return self.
Code:
class T:
def __init__(self):
self.s = 10
def __iter__(self):
return self.__next__()
def __next__(self):
for i in range(self.s):
yield i
if __name__== '__main__':
obj = T()
for i in obj:
print(i)
Terminal:
C:\...>python test.py
0
1
2
3
4
5
6
7
8
9
C:...>
Also, you can call from __iter__ any other method instead __next__().
Code:
class T:
def __init__(self):
self.s = 10
def __iter__(self):
return self.foo()
def foo(self):
for i in range(self.s):
yield i
if __name__== '__main__':
obj = T()
for i in obj:
print(i)
You will have exactly the same results.
Case: yield in __iter__ method without __next__ method
I don't think it is a good idea to use yield in __iter__. Ok, it works, but I think that destroys the class API.
Case: __iter__ with __next__ in custom class without yield
Use these methods (__iter__ and __next__). In the __iter__ return self and do not forget to raise StopIteration in __next__ method.
Code:
class T:
def __init__(self):
self.s = 10
def __iter__(self):
self.__i = -1
return self
def __next__(self):
while self.__i < self.s-1:
self.__i+=1
return self.__i
raise StopIteration
if __name__== '__main__':
obj = T()
for i in obj:
print(i)
I am using yield to return the next value in the __next__() function in my class. However it does not return the next value, it returns the generator object.
I am trying to better understand iterators and yield. I might be doing it in the wrong way.
Have a look.
class MyString:
def __init__(self,s):
self.s=s
def __iter__(self):
return self
def __next__(self):
for i in range(len(self.s)):
yield(self.s[i])
r=MyString("abc")
i=iter(r)
print(next(i))
This returns:
generator object __next__ at 0x032C05A0
next pretty much just calls __next__() in this case. Calling __next__ on your object will start the generator and return it (no magic is done at this point).
In this case, you might be able to get away with not defining __next__ at all:
class MyString:
def __init__(self,s):
self.s=s
def __iter__(self):
for i in range(len(self.s)):
yield(self.s[i])
# Or...
# for item in self.s:
# yield item
If you wanted to use __iter__ and __next__ (to define an iterator rather than simply making an iterable), you'd probably want to do something like this:
class MyString:
def __init__(self,s):
self.s = s
self._ix = None
def __iter__(self):
return self
def __next__(self):
if self._ix is None:
self._ix = 0
try:
item = self.s[self._ix]
except IndexError:
# Possibly reset `self._ix`?
raise StopIteration
self._ix += 1
return item
Let's take a look at the purpose of the __next__ method. From the docs:
iterator.__next__()
Return the next item from the container. If there are no further items, raise the StopIteration exception.
Now let's see what the yield statement does. Another excerpt from the docs:
Using a yield expression in a function’s body causes that function to
be a generator
And
When a generator function is called, it returns an iterator known as a
generator.
Now compare __next__ and yield: __next__ returns the next item from the container. But a function containing the yield keyword returns an iterator. Consequently, using yield in a __next__ method results in an iterator that yields iterators.
If you want to use yield to make your class iterable, do it in the __iter__ method:
class MyString:
def __init__(self, s):
self.s = s
def __iter__(self):
for s in self.s:
yield s
The __iter__ method is supposed to return an iterator - and the yield keyword makes it do exactly that.
For completeness, here is how you would implement an iterator with a __next__ method. You have to keep track of the state of the iteration, and return the corresponding value. The easiest solution is probably to increment an index every time __next__ is called:
class MyString:
def __init__(self,s):
self.s = s
self.index = -1
def __iter__(self):
return self
def __next__(self):
self.index += 1
if self.index >= len(self.s):
raise StopIteration
return self.s[self.index]
As far as I can tell, generator functions are just syntactic sugar for classes with a next function. Example:
>>> def f():
i = 0
while True:
i += 1
yield i
>>> x = f()
>>> x
<generator object f at 0x0000000000659938>
>>> next(x)
1
>>> next(x)
2
>>> next(x)
3
>>> class g(object):
def __init__(self):
self.i = 0
def __next__(self):
self.i += 1
return self.i
>>> y = g()
>>> y
<__main__.g object at 0x000000000345D908>
>>> next(y)
1
>>> next(y)
2
>>> next(y)
3
In fact, I came here looking to see if there is any significant difference. Please shout if there is.
So, to answer the question, what you have is a class with a __next__ method that returns an object that also has a __next__ method. So the simplest thing to do would be to replace your yield with a return and to keep track of how far along you are, and to remember to raise a StopIteration when you reach the end of the array. So something like:
class MyString:
def __init__(self,s):
self.s=s
self._i = -1
def __iter__(self):
return self
def __next__(self):
self._i += 1
if self._i >= len(self.s):
raise StopIteration
return self.s[self._i]
That's probably the simplest way to achieve what I think you're looking for.
OBSERVATION
If next() function calls __next__() method, what's happening in the following example.
Code:
class T:
def __init__(self):
self.s = 10
def __iter__(self):
for i in range(self.s):
yield i
def __next__(self):
print('__next__ method is called.')
if __name__== '__main__':
obj = T()
k = iter(obj)
print(next(k)) #0
print(next(k)) #1
print(next(k)) #2
print(next(k)) #3
print(next(k)) #4
print(next(k)) #5
print(next(k)) #6
print(next(k)) #7
print(next(k)) #8
print(next(k)) #9
print(next(k))
print(next(k))
Terminal:
C:...>python test.py
0
1
2
3
4
5
6
7
8
9
Traceback (most recent call last):
File "test.py", line 25, in <module>
print(next(k))
StopIteration
WHAT IS HAPPENING?
It seams that next() function does not calling __next__ method. I cannot understand why python docs states that "next(iterator, default) Retrieve the next item from the iterator by calling its __next__() method." If someonw knows, let us help!
Case: __iter__ with __next__ in custom class with yield
So, if you want to use yield (in order to create a generator) with __iter__ and __next__ methods in a custom class, do not put just the yield into the __next__ method, but use it with __iter__(self) and return self.__next__() instead return self.
Code:
class T:
def __init__(self):
self.s = 10
def __iter__(self):
return self.__next__()
def __next__(self):
for i in range(self.s):
yield i
if __name__== '__main__':
obj = T()
for i in obj:
print(i)
Terminal:
C:\...>python test.py
0
1
2
3
4
5
6
7
8
9
C:...>
Also, you can call from __iter__ any other method instead __next__().
Code:
class T:
def __init__(self):
self.s = 10
def __iter__(self):
return self.foo()
def foo(self):
for i in range(self.s):
yield i
if __name__== '__main__':
obj = T()
for i in obj:
print(i)
You will have exactly the same results.
Case: yield in __iter__ method without __next__ method
I don't think it is a good idea to use yield in __iter__. Ok, it works, but I think that destroys the class API.
Case: __iter__ with __next__ in custom class without yield
Use these methods (__iter__ and __next__). In the __iter__ return self and do not forget to raise StopIteration in __next__ method.
Code:
class T:
def __init__(self):
self.s = 10
def __iter__(self):
self.__i = -1
return self
def __next__(self):
while self.__i < self.s-1:
self.__i+=1
return self.__i
raise StopIteration
if __name__== '__main__':
obj = T()
for i in obj:
print(i)
I was trying to write a simple countdown iterator of my own, I implemented a __iter__() function and the corresponding __next__() to support the iterator.I have used a yield function inside the __next__() function to return a new value everytime I iterate over the object.
When I use yield, the code goes over in an infinite loop as compared to using return statement.
Following is my code:
class MyIterator():
def __init__(self,value):
self.value = value
def __iter__(self):
return self
def __next__(self):
print("In the next function")
if self.value > 0:
yield self.value
self.value -= 1
else:
raise StopIteration("Failed to proceed to the next step")
if __name__ == '__main__':
myIt = MyIterator(10)
for i in myIt:
print(i)
And the O/P is as follows:
<generator object __next__ at 0x101181990>
<generator object __next__ at 0x1011818e0>
<generator object __next__ at 0x101181990>
<generator object __next__ at 0x1011818e0>
and so on for infinite times....
Your __next__ method should not itself be a generator. Replace yield with return:
def __next__(self):
print("In the next function")
if self.value > 0:
return_value = self.value
self.value -= 1
return return_value
else:
raise StopIteration("Failed to proceed to the next step")
Note that you still need to decrease self.value after determining what is to be returned, hence the use of a separate return_value variable.
Any function (or method) with yield in it will produce a generator object when called, and that generator is then the iterable. Such an object then has an __iter__ method that returns self and a __next__ method that will produce the next value when called. This is why you see the <generator object __next__ at 0x1011818e0> object being printed each time __next__ is called.
However, for your object to be an iterable itself, your __next__ method should instead return the next value in the sequence. It'll be called repeatedly until it raises StopIteration. This is different from using yield, it should return immediately, not defer until later!
Demo:
>>> class MyIterator():
... def __init__(self,value):
... self.value = value
... def __iter__(self):
... return self
... def __next__(self):
... print("In the next function")
... if self.value > 0:
... return_value = self.value
... self.value -= 1
... return return_value
... else:
... raise StopIteration("Failed to proceed to the next step")
...
>>> myIt = MyIterator(10)
>>> for i in myIt:
... print(i)
...
In the next function
10
In the next function
9
In the next function
8
In the next function
7
In the next function
6
In the next function
5
In the next function
4
In the next function
3
In the next function
2
In the next function
1
In the next function
If you wanted to use a generator function, make __iter__ the generator, and use a loop:
class MyIterator():
def __init__(self,value):
self.value = value
def __iter__(self):
value = self.value
while value > 0:
yield value
value -= 1
However, this makes your MyIterator class an iterable, not an iterator. Instead, each time you use a for loop a new iterator is created (the __iter__ generator object) that is then iterated over. Using __next__ makes your object an iterator that can only be iterated over once.
There is a bit of confusion here. When you using yield, you don't need to create an iterator class. The generator already is an iterator.
So, change the yield to a return and it will do what you want :-)
Also, you will need to update self.value before the return. Here is what the fixed-up code looks like:
class MyIterator():
def __init__(self,value):
self.value = value
def __iter__(self):
return self
def __next__(self):
print("In the next function")
value = self.value
if value > 0:
self.value -= 1
return value
else:
raise StopIteration("Failed to proceed to the next step")
And here is how to do the same thing with a generator:
def my_iterator(value):
while value > 0:
yield value
value -= 1
As you can see, generators make a programmer's life much easier :-)