We Keep Coding

How to get Python iterators not to communicate with each other?

Here's a simple iterator through the characters of a string.
class MyString:
def __init__(self,s):
self.s = s
self._ix = 0
def __iter__(self):
return self
def __next__(self):
try:
item = self.s[self._ix]
except IndexError:
self._ix = 0
raise StopIteration
self._ix += 1
return item
string = MyString('abcd')
iter1 = iter(string)
iter2 = iter(string)
print(next(iter1))
print(next(iter2))
Trying to get this iterator to function like it should. There are a few requirements. First, the __next__ method MUST raise StopIteration and multiple iterators running at the same time must not interact with each other.
I accomplished objective 1, but need help on objective 2. As of right now the output is:
'a'
'b'
When it should be:
'a'
'a'
Any advice would be appreciated.
Thank you!

MyString acts as its own iterator much like a file object
>>> f = open('deleteme', 'w')
>>> iter(f) is f
True
You use this pattern when you want all iterators to affect each other - in this case advancing through the lines of a file.
The other pattern is to use a separate class to iterate much like a list whose iterators are independent.
>>> l = [1, 2, 3]
>>> iter(l) is l
False
To do this, move the _ix indexer to a separate class that references MyString. Have MyString.__iter__ create an instance of the class. Now you have a separate indexer per iterator.
class MyString:
def __init__(self,s):
self.s = s
def __iter__(self):
return MyStringIter(self)
class MyStringIter:
def __init__(self, my_string):
self._ix = 0
self.my_string = my_string
def __iter__(self):
return self
def __next__(self):
try:
item = self.my_string.s[self._ix]
except IndexError:
raise StopIteration
self._ix += 1
return item
string = MyString('abcd')
iter1 = iter(string)
iter2 = iter(string)
print(next(iter1))
print(next(iter2))

Your question title asks how to get iterators, plural, to not communicate with each other, but you don't have multiple iterators, you only have one. If you want to be able to get distinct iterators from MyString, you can add a copy method:
class MyString:
def __init__(self,s):
self.s = s
self._ix = 0
def __iter__(self):
return self
def __next__(self):
try:
item = self.s[self._ix]
except IndexError:
self._ix = 0
raise StopIteration
self._ix += 1
return item
def copy(self):
return MyString(self.s)
string = MyString('abcd')
iter1 = string.copy()
iter2 = string.copy()
print(next(iter1))
print(next(iter2))

Doubly-linked list in Python

It turned out to be an error as a time limit was exceeded,
but I've already raised the StopIteration...
I think I did something wrong for my iteration part, but it's really hard to find the error. The test output keeps running and even printed out the None value. How does it happen?
class LinkedListIterator:
def __init__(self, head):
self.__current = head.get_next()
def __iter__(self):
return self
def __next__(self):
if self.__current == None:
raise StopIteration
else:
item = self.__current.get_data()
self.__current = self.__current.get_next()
return item
These were the inputs I used to run the program:
my_list = LinkedListDLL()
my_list.add_to_head(1)
print("Contents:", end=" ")
for node in my_list:
print(node, end=" ")
print()

This code is meant to stop iteration when it reaches the head of the list.
if self.__current == None:
raise StopIteration
However, you represent the head with a NodeDLL object which is different from None.
You could keep a reference to the head and check against that instead:
class LinkedListIterator:
def __init__(self, head):
self._head = head
self._current = head.get_next()
def __iter__(self):
return self
def __next__(self):
if self._current is self._head:
raise StopIteration
else:
item = self._current.get_data()
self._current = self._current.get_next()
return item

What you want to implement is the API of a MutableSequence with the implementation of a doubly-linked-list.
To do that in Python, you should rely on collections.abc which can guide you through the process of implementing all required methods.
By example, a linked-list is actually a class inheriting from MutableSequence.
from collections.abc import MutableSequence
class LinkedList(MutableSequence):
pass
ll = LinkedList()
On instantiation of a class which has some abstract methods not yet written, you will get a TypeError which will guide you through which methods need to be implemented.
TypeError: Can't instantiate abstract class LinkedList with abstract methods __delitem__, __getitem__, __len__, __setitem__, insert
In particular, note that a list or a linked-list is not an iterator, it is an iterable. What this means is the __iter__ method should not return self and rely on __next__, it should instead return a brand new iterator on the content of the linked-list.
In other words, you can iterate only once through an iterator and multiple times through and iterable.
Full implementation
It turns out I have a full implementation of a doubly-linked-list implemented that way. You can have a look.
from collections.abc import MutableSequence
class LinkedList(MutableSequence):
class _Node:
def __init__(self, value, _next=None, _last=None):
self.value, self._next, self._last = value, _next, _last
def __str__(self):
return f'Node({self.value})'
def __init__(self, iterable=()):
self.start = None
self.last = None
empty = object()
iterable = iter(iterable)
first = next(iterable, empty)
if first is empty:
return
current = self._Node(first)
self.start, self.last = current, current
for value in iterable:
new_node = self._Node(value, _last=self.last)
self.last._next = new_node
self.last = new_node
def __len__(self):
if self.start is None:
return 0
else:
return sum(1 for _ in self)
def __iter_nodes(self):
current = self.start
while current is not None:
yield current
current = current._next
def __reversed_iter_nodes(self):
current = self.last
while current is not None:
yield current
current = current._last
def __iter__(self):
for node in self.__iter_nodes():
yield node.value
def __reversed__(self):
for node in self.__reversed_iter_nodes():
yield node.value
def __get_node(self, index):
if index >= 0:
for item in self.__iter_nodes():
if index == 0:
return item
index -= 1
else:
for item in self.__reversed_iter_nodes():
if index == 0:
return item
index += 1
raise IndexError
def __getitem__(self, index):
if index >= 0:
for item in self:
if index == 0:
return item.value
index -= 1
else:
for item in reversed(self):
if index == 0:
return item.value
index += 1
raise IndexError
def __setitem__(self, key, value):
self[key].value = value
def __delitem__(self, key):
node = self[key]
if node._last:
node._last._next = node._next
if node._next:
node._next._last = node._last
def insert(self, index, value):
if index > len(self):
self.last = self._Node(value, _last=self.last)
else:
where = self.__get_node(index)
_last = where._last
new_node = self._Node(value, _next=where, _last=_last)
if _last:
_last._next = new_node
else:
self.start = new_node
where._last = new_node
Example
ll = LinkedList(range(1, 5))
print(*ll)
print(*reversed(ll))
ll.insert(2, 'foo')
print(*ll)
Output
1 2 3 4
4 3 2 1
1 2 foo 3 4

Single Linked List search in Python

I want to search a value/character in a linked list and return the number of times the value/character is in the linked list. Also would it be easier if I just used recursion instead of tail recursion?
class MyList():
__slots__=('head','size')
class Empty():
__slots__=()
class NonEmpty():
__slots__=('data','next')
def mkMyList():
lst = MyList()
lst.head = mkEmpty()
lst.size = 0
return lst
def mkEmpty():
return Empty()
def mkNonEmpty(data,lst):
node = NonEmpty()
node.data = data
node.next = lst
return node
def count(l, value, c = 0):
l = mkMyList()
if l.head != value:
l.head = l.head.next
if l.head == value:
return count(l.head.next, value, c + 1)
if l.size == 0:
return c
When I try to test it, I get this:
count(s,'s',c= 0)
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
count(s,'s',c= 0)
File "C:\Users\Qasim\Desktop\Linked Lists.py", line 30, in count
l.head = l.head.next
AttributeError: 'Empty' object has no attribute 'next'
\

Rather than use recursion, I would use the iterator pattern. Here is one way to do it in the context of your problem:
class LinkedList(object):
class Node(object):
__slots__ = ('prev', 'next', 'value')
def __init__(self, prev=None, next=None, value=None):
self.prev = prev
self.next = next
self.value = value
def __init__(self, iterable=[]):
self.head = LinkedList.Node() # dummy node
self.tail = self.head
self.size = 0
for item in iterable:
self.append(item)
def __iter__(self):
current = self.head
while True:
if current.next is not None:
current = current.next
yield current.value
else:
raise StopIteration
def append(self, value):
self.tail.next = LinkedList.Node(prev=self.tail, value=value)
self.tail = self.tail.next
self.size += 1
def pop(self):
if self.size > 0:
value = self.tail.value
self.tail = self.tail.prev
self.tail.next = None
self.size -= 1
return value
else:
raise IndexError('pop from empty list')
def count(self, value):
cumsum = 0
for item in self:
if item == value:
cumsum += 1
return cumsum
By my defining a Python special method __iter__, one can sequentially access the elements of a LinkedList in the following manner:
l = LinkedList([1, 2, 3, 3, 3, 4, 5])
for value in l:
print(value)
which then makes the desired method count straight-forward to implement.
Note that I have used the Python generator syntax to implement __iter__, you can read about generators and the yield statement here.

Tracing your code:
l = mkMyList() # => head = Empty()
if l.head != value: # True since head is Empty()
l.head = l.head.next # Empty does not have a ".next" attribute
This is what the Traceback is telling you.
EDIT: Two more things: (1) I'm not sure why count is even calling mkMyList when it seems your intent is to pass it the list, l, in the function args. (2) I'm guessing you want to put the size-check if statement at the top of this function:
if l.size == 0:
return c

The issue I see is that the list in count is never initialized properly. In mkMyList(), the head element is set to and Empty, which has no next attribute. In count(), you only use mkMyList(). This means that l.head is an Empty, and there's no way it could have a next attribute. To fix this, I would recommend instantiating the list l using the given input.
With regards to the question about recursion: no, there is very little difference in terms of composing a tail recursive function versus a regularly recursive function.

updating a python generator after it has been created

Is there any way to do something like this in python 2.7?
def scaleit(g, k):
for item in g:
yield item*k
promise = ??????
# defines a generator for reference but not use:
# other functions can make use of it,
# but it requires a call to promise.fulfill() to
# define what the generator is going to yield;
# promise raises an error if next() is called
# before the promise is fulfilled
f = scaleit(promise, 3)
promise.fulfill(range(10))
for item in f:
print item

Yes; generators don't run until they're actually iterated, so you can just defer iterating the fulfilled promise's value until requested:
class Promise(object):
def fulfill(self, result):
self.result = result
def __iter__(self):
return iter(self.result)
def scaleit(g, k):
for item in g:
yield item*k
promise = Promise()
f = scaleit(promise, 3)
promise.fulfill(range(10))
print list(f)

Is this what you want?
def scaleit(g, k):
for item in g:
yield item * k
class Promise(object):
def __init__(self):
self.g = None
def fulfill(self, g):
self.g = iter(g)
def __iter__(self):
return self
def next(self):
return next(self.g)
promise = Promise()
f = scaleit(promise, 3)
promise.fulfill(range(10))
for item in f:
print item

I think you want the send() method on generators:
def gen():
reply = yield None
if not reply: # reply will be None if send() wasn't called
raise ValueError("promise not fulfilled")
yield 5
g1 = gen()
next(g1) # advance to the first yield
g1.send(True)
print(next(g1)) # prints 5
g2 = gen()
next(g2)
# forget to send
print(next(g2)) # raises ValueError

Making a python iterator go backwards?

Is there anyway to make a python list iterator to go backwards?
Basically i have this
class IterTest(object):
def __init__(self, data):
self.data = data
self.__iter = None
def all(self):
self.__iter = iter(self.data)
for each in self.__iter:
mtd = getattr(self, type(each).__name__)
mtd(each)
def str(self, item):
print item
next = self.__iter.next()
while isinstance(next, int):
print next
next = self.__iter.next()
def int(self, item):
print "Crap i skipped C"
if __name__ == '__main__':
test = IterTest(['a', 1, 2,3,'c', 17])
test.all()
Running this code results in the output:
a
1
2
3
Crap i skipped C
I know why it gives me the output, however is there a way i can step backwards in the str() method, by one step?
EDIT
Okay maybe to make this more clear. I don't want to do a full reverse, basically what i want to know if there is an easy way to do the equivalent of a bidirectional iterator in python?

No, in general you cannot make a Python iterator go backwards. However, if you only want to step back once, you can try something like this:
def str(self, item):
print item
prev, current = None, self.__iter.next()
while isinstance(current, int):
print current
prev, current = current, self.__iter.next()
You can then access the previous element any time in prev.
If you really need a bidirectional iterator, you can implement one yourself, but it's likely to introduce even more overhead than the solution above:
class bidirectional_iterator(object):
def __init__(self, collection):
self.collection = collection
self.index = 0
def next(self):
try:
result = self.collection[self.index]
self.index += 1
except IndexError:
raise StopIteration
return result
def prev(self):
self.index -= 1
if self.index < 0:
raise StopIteration
return self.collection[self.index]
def __iter__(self):
return self

Am I missing something or couldn't you use the technique described in the Iterator section in the Python tutorial?
>>> class reverse_iterator:
... def __init__(self, collection):
... self.data = collection
... self.index = len(self.data)
... def __iter__(self):
... return self
... def next(self):
... if self.index == 0:
... raise StopIteration
... self.index = self.index - 1
... return self.data[self.index]
...
>>> for each in reverse_iterator(['a', 1, 2, 3, 'c', 17]):
... print each
...
17
c
3
2
1
a
I know that this doesn't walk the iterator backwards, but I'm pretty sure that there is no way to do that in general. Instead, write an iterator that walks a discrete collection in reverse order.
Edit you can also use the reversed() function to get a reversed iterator for any collection so that you don't have to write your own:
>>> it = reversed(['a', 1, 2, 3, 'c', 17])
>>> type(it)
<type 'listreverseiterator'>
>>> for each in it:
... print each
...
17
c
3
2
1
a

An iterator is by definition an object with the next() method -- no mention of prev() whatsoever. Thus, you either have to cache your results so you can revisit them or reimplement your iterator so it returns results in the sequence you want them to be.

Based on your question, it sounds like you want something like this:
class buffered:
def __init__(self,it):
self.it = iter(it)
self.buf = []
def __iter__(self): return self
def __next__(self):
if self.buf:
return self.buf.pop()
return next(self.it)
def push(self,item): self.buf.append(item)
if __name__=="__main__":
b = buffered([0,1,2,3,4,5,6,7])
print(next(b)) # 0
print(next(b)) # 1
b.push(42)
print(next(b)) # 42
print(next(b)) # 2

You can enable an iterator to move backwards by following code.
class EnableBackwardIterator:
def __init__(self, iterator):
self.iterator = iterator
self.history = [None, ]
self.i = 0
def next(self):
self.i += 1
if self.i < len(self.history):
return self.history[self.i]
else:
elem = next(self.iterator)
self.history.append(elem)
return elem
def prev(self):
self.i -= 1
if self.i == 0:
raise StopIteration
else:
return self.history[self.i]
Usage:
>>> prev = lambda obj: obj.prev() # A syntactic sugar.
>>>
>>> a = EnableBackwardIterator(iter([1,2,3,4,5,6]))
>>>
>>> next(a)
1
>>> next(a)
2
>>> a.next() # The same as `next(a)`.
3
>>> prev(a)
2
>>> a.prev() # The same as `prev(a)`.
1
>>> next(a)
2
>>> next(a)
3
>>> next(a)
4
>>> next(a)
5
>>> next(a)
6
>>> prev(a)
5
>>> prev(a)
4
>>> next(a)
5
>>> next(a)
6
>>> next(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration

You can wrap your iterator in an iterator helper to enable it to go backward. It will store the iterated values in a collection and reuse them when going backwards.
class MemoryIterator:
def __init__(self, iterator : Iterator):
self._iterator : Iterator = iterator
self._array = []
self._isComplete = False
self._pointer = 0
def __next__(self):
if self._isComplete or self._pointer < len(self._array):
if self._isComplete and self._pointer >= len(self._array):
raise StopIteration
value = self._array[self._pointer]
self._pointer = self._pointer + 1
return value
try:
value = next(self._iterator)
self._pointer = self._pointer + 1
self._array.append(value)
return value
except StopIteration:
self._isComplete = True
def prev(self):
if self._pointer - 2 < 0:
raise StopIteration
self._pointer = self._pointer - 1
return self._array[self._pointer - 1]
The usage can be similar to this one:
my_iter = iter(my_iterable_source)
memory_iterator = MemoryIterator(my_iter)
try:
if forward:
print(next(memory_iterator))
else:
print(memory_iterator.prev())
except StopIteration:
pass

I came here looking for a bi-directional iterator. Not sure if this is what the OP was looking for but it is one way to make a bi-directional iterator—by giving it an attribute to indicate which direction to go in next:
class BidirectionalCounter:
"""An iterator that can count in two directions (up
and down).
"""
def __init__(self, start):
self.forward = True
# Code to initialize the sequence
self.x = start
def __iter__(self):
return self
def __next__(self):
if self.forward:
return self.next()
else:
return self.prev()
def reverse(self):
self.forward = not self.forward
def next(self):
"""Compute and return next value in sequence.
"""
# Code to go forward
self.x += 1
return self.x
def prev(self):
"""Compute and return previous value in sequence.
"""
# Code to go backward
self.x -= 1
return self.x
Demo:
my_counter = BidirectionalCounter(10)
print(next(my_counter))
print(next(my_counter))
my_counter.reverse()
print(next(my_counter))
print(next(my_counter))
Output:
11
12
11
10

i think thi will help you to solve your problem
class TestIterator():
def __init__(self):`
self.data = ["MyData", "is", "here","done"]
self.index = -1
#self.index=len(self.data)-1
def __iter__(self):
return self
def next(self):
self.index += 1
if self.index >= len(self.data):
raise StopIteration
return self.data[self.index]
def __reversed__(self):
self.index = -1
if self.index >= len(self.data):
raise StopIteration
return self.data[self.index]
r = TestIterator()
itr=iter(r)
print (next(itr))
print (reversed(itr))

ls = [' a', 5, ' d', 7, 'bc',9, ' c', 17, '43', 55, 'ab',22, 'ac']
direct = -1
l = ls[::direct]
for el in l:
print el
Where direct is -1 for reverse or 1 for ordinary.

Python you can use a list and indexing to simulate an iterator:
a = [1,2,3]
current = 1
def get_next(a):
current = a[a.index(current)+1%len(a)]
return current
def get_last(a):
current = a[a.index(current)-1]
return current # a[-1] >>> 3 (negative safe)
if your list contains duplicates then you would have to track your index separately:
a =[1,2,3]
index = 0
def get_next(a):
index = index+1 % len(a)
current = a[index]
return current
def get_last(a):
index = index-1 % len(a)
current = a[index-1]
return current # a[-1] >>> 3 (negative safe)

An iterator that visits the elements of a list in reverse order:
class ReverseIterator:
def __init__(self,ls):
self.ls=ls
self.index=len(ls)-1
def __iter__(self):
return self
def __next__(self):
if self.index<0:
raise StopIteration
result = self.ls[self.index]
self.index -= 1
return result

I edited the python code from dilshad (thank you) and used the following Python 3 based code to step between list item's back and forth or let say bidirectional:
# bidirectional class
class bidirectional_iterator:
def __init__(self):
self.data = ["MyData", "is", "here", "done"]
self.index = -1
def __iter__(self):
return self
def __next__(self):
self.index += 1
if self.index >= len(self.data):
raise StopIteration
return self.data[self.index]
def __reversed__(self):
self.index -= 1
if self.index == -1:
raise StopIteration
return self.data[self.index]
Example:
>>> r = bidirectional_iterator()
>>> itr=iter(r)
>>> print (next(itr))
MyData
>>> print (next(itr))
is
>>> print (next(itr))
here
>>> print (reversed(itr))
is
>>> print (reversed(itr))
MyData
>>> print (next(itr))
is

This is a common situation when we need to make an iterator go back one step. Because we should get the item and then check if we should break the loop. When breaking the loop, the last item may be requied in later usage.
Except of implementing an iteration class, here is a handy way make use of builtin itertools.chain :
from itertools import chain
>>> iterator = iter(range(10))
>>> for i in iterator:
... if i <= 5:
... print(i)
... else:
... iterator = chain([i], iterator) # push last value back
... break
...
0
1
2
3
4
5
>>> for i in iterator:
... print(i)
...
6
7
8
9

please see this function made by Morten Piibeleht. It yields a (previous, current, next) tuple for every element of an iterable.
https://gist.github.com/mortenpi/9604377