I'm trying to port a custom class from Python 2 to Python 3. I can't find the right syntax to port the iterator for the class. Here is a MVCE of the real class and my attempts to solve this so far:
Working Python 2 code:
class Temp:
def __init__(self):
self.d = dict()
def __iter__(self):
return self.d.iteritems()
temp = Temp()
for thing in temp:
print(thing)
In the above code iteritems() breaks in Python 3. According to this highly voted answer, "dict.items now does the thing dict.iteritems did in python 2". So I tried that next:
class Temp:
def __init__(self):
self.d = dict()
def __iter__(self):
return self.d.items()
The above code yields "TypeError: iter() returned non-iterator of type 'dict_items'"
According to this answer, Python 3 requires iterable objects to provide a next() method in addition to the iter method. Well, a dictionary is also iterable, so in my use case I should be able to just pass dictionary's next and iter methods, right?
class Temp:
def __init__(self):
self.d = dict()
def __iter__(self):
return self.d.__iter__
def next(self):
return self.d.next
This time it's giving me "TypeError: iter() returned non-iterator of type 'method-wrapper'".
What am I missing here?
As the error message suggests, your __iter__ function does not return an iterator, which you can easily fix using the built-in iter function
class Temp:
def __init__(self):
self.d = {}
def __iter__(self):
return iter(self.d.items())
This will make your class iterable.
Alternatively, you may write a generator yourself, like so:
def __iter__(self):
for key,item in self.d.items():
yield key,item
If you want to be able to iterate over keys and items separately, i.e. in the form that the usual python3 dictionary can, you can provide additional functions, for example
class Temp:
def __init__(self, dic):
self.d = dic
def __iter__(self):
return iter(self.d)
def keys(self):
return self.d.keys()
def items(self):
return self.d.items()
def values(self):
return self.d.values()
I'm guessing from the way you phrased it that you don't actually want the next() method to be implemented if not needed. If you would, you would have to somehow turn your whole class into an iterator and somehow keep track of where you are momentarily in this iterator, because dictionaries themselves are not iterators. See also this answer.
I don't know what works in Python 2. But on Python 3 iterators can be most easily created using something called a generator. I am providing the name and the link so that you can research further.
class Temp:
def __init__(self):
self.d = {}
def __iter__(self):
for thing in self.d.items():
yield thing
Related
I tried to solve an easy problem in leetcode. Here is the source: https://leetcode.com/problems/remove-duplicates-from-sorted-list/
I almost solved it with for loop, but i get an error Python object of type ListNode has no len(). I have tried to use call() or len(), but i have no knowledge or understanding how does these built-in methods work. I read in many places, but the mess got bigger. If anyone could help me it would be great.
P.S. I know the solution with while loop is better, but i want this one to work somehow if possible. Or at least get some output.
`
**# Definition for singly-linked list.
# class ListNode:
# def __init__(self, sequence, val=0, next=None):
# self.val = val
# self.next = next
# self.sequence = sequence
class Solution:
def deleteDuplicates(self, head: Optional[ListNode]) -> Optional[ListNode]:**
j=0
for i in range(0, len(head) - 1):
if head[i-1-j] == head[i-j]:
head.remove(head[i-j])
j += 1
head.remove(head[-1])
return head
`
I can't really decipher how your ListNode class is designed to work, but the answer to how to be able to call len on an instance of your own class is to define the __len__ dunder method for the class. As others have pointed out in the comments, you'll also need to define the __getitem__ method in order to be able to do stuff like head[i-1-j]. It might look something like this (just guessing as to the actual definitions as it's not clear to me how you want the class to work):
class ListNode:
def __init__(self, sequence, val=0, next=None):
self.val = val
self.next = next
self.sequence = sequence
def __len__(self):
return len(self.sequence) # Maybe? Or whatever you want len to return
def __getitem__(self, i):
return self.sequence[i] # Maybe? Or whatever you want [ ] to return
More on dunder methods here.
I realised I still don't quite understand how to implement an iterator class. So a class where one can "for-loop" over some of its contents.
I have looked at these answers (I still don't get it):
What exactly are Python's iterator, iterable, and iteration protocols?
Build a Basic Python Iterator
As far as I understand, an iterable is one that implements __iter__ and returns an iterator which is something that has __next__ implemented.
From this I somehow understood that if I want my class to be an iterator, and iterable. I must define __iter__ to return self, and have __next__ defined. Am I wrong so far?
Here is a scaffold for my class:
class Wrapper:
def __init__(self, low, high):
self.store = OrderedDict() # it is imported (ommited)
def __getitem__(self, key):
return self.store[key]
def __iter__(self):
return self
# Option 3
return self.store
def __next__(self):
# Option 1
for key in self.store:
return key
# Option 2
for key in self.store.keys():
return key
Tried the above options, none worked. :(
What happens is I have some py.tests prepared to test the iteration if it works correctly, just a simple for loop, nothing fancy. And the tests just run forever (or longer then my patience < 5 min), but the mock class has 5 items, so it should not take long.
What am I doing wrong?
You've got your concepts mixed up. You say
I realised I still don't quite understand how to implement an iterator class. So a class where one can "for-loop" over some of its contents.
but that's not how things work. If you just want to be able to perform for loops over instances of your class, you almost certainly shouldn't be making your instances iterators directly. You should write an __iter__ method that returns an iterator, either manually or by yielding, and you should not write a __next__:
# Returning an iterator manually:
class Wrapper:
...
def __iter__(self):
return iter(self.store)
# or with yield:
class Wrapper:
...
def __iter__(self):
for thing in self.store:
yield thing
Iterators are technically iterable, but only once. To write an iterator, you would have __iter__ return self and have __next__ return the next item on each call, or raise StopIteration when the items are exhausted (and continue to raise StopIteration on any further calls). I'll write an iterator over the half-open interval [0, 10), since writing one for an OrderedDict would just delegate to the OrderedDict's iterator for everything too directly to be instructive:
class Iterator:
def __init__(self):
self.state = 0
def __iter__(self):
return self
def __next__(self):
if self.state == 10:
raise StopIteration
self.state += 1
return self.state - 1
Writing iterators manually is unusual even when you do want an iterator, though; it's much more common to use yield:
def iterator():
for i in range(10):
yield i
# or managing the index manually, to show what it would be like without range:
def iterator():
i = 0
while i < 10:
yield i
i += 1
You can make the following changes:
class Wrapper:
def __init__(self, low, high):
self.store = OrderedDict()
self.__iter = None # maintain state of self as iterator
def __iter__(self):
if self.__iter is None:
self.__iter = iter(self.store)
return self
def __next__(self):
try:
return next(self.__iter)
except StopIteration: # support repeated iteration
self.__iter = None
raise
__next__ should return the next element of an iterator that is being iterated. And your __iter__ method should return self to properly implement the protocol. This ensures for instance, that a call to next during a loop drives forward the same iterator as the loop:
for x in wrapper:
# do stuff with x
next(wrapper) # skip one
For most purposes, impementing __iter__ should suffice:
class Wrapper:
def __init__(self, low, high):
self.store = OrderedDict()
def __iter__(self):
return iter(self.store)
Here's is my simple class:
class row(object):
__slots__ = ("tan_id", "proc_name")
def __init__(
self,
tan_id = "INDETERMINATE",
proc_name = "INDETERMINATE"
):
self.tan_id = tan_id
self.proc_name = proc_name
This class used to be a dictionary, but I ran into serious memory issues. Later on in my code I use a generalized function to loop like this:
for key, val in row.items()
I don't want to change that... so my question is, how do I simulate that item() function in the class so that it works seamlessly in the existing for loop.
You can simply implement an items method that mimicks the previous dict.items() behaviour by producing the same key-value pairs:
class row(object):
# ...
def items(self):
# this is the iterator behaviour of 'items' (Py3) or 'iteritems' (Py2)
yield 'tan_id', self.tan_id
yield 'proc_name', self.proc_name
# for the Py2 items behaviour, put these pairs into a list
# and return the list
# Or, more generally:
def items(self):
for slot in self.__slots__:
yield slot, getattr(self, slot)
I'am trying to implement a 'heapq' or a 'deque' dynamically (according to user's input)
class MyClass():
def __init__(self, choose = True ):
self.Q = []
self.add = self.genAdd(choose)
self.get = self.genGet(choose)
def genAdd(self, ch):
if(ch == True):
def f(Q, elem):
return Q.append
else:
def f(Q):
return heappush
return f
and same for 'genGet'
the execution is correct on one side (x)or the other (but not both at the same time). I get things like
TypeError: f() takes exactly 1 argument (2 given)
tried multiple inhreitance but got
TypeError: Error when calling the metaclass bases
metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases
the problem is that heapq is called with
heappush(Q, elem)
and queue with
Q.append(elem)
I hope the point is clear. I think there should be a way to fix that (maybe using lambda)
Thanks
Inheritance isn't going to help here.
First, heapq isn't even a class, so you can't inherit from it. You can write a class that wraps up its functionality (or find one on the ActiveState recipes or in a PyPI package), but you have to have a class to inherit.
But, more importantly, the whole point of inheritance is to give you an "is-a" relationship. This thing you're building isn't-a deque, or a heapq-wrapping object, it's a thing with an interface that you've defined (add and get) that happens to use either a deque or a list with heapq for implementation.
So, just do that explicitly. You're trying to define a function that either calls append on a deque, or calls heapq.heappush on a list. You're not trying to write a curried function that returns a function that does the thing, just a function that does the thing.
def genAdd(self, ch):
# As a side note, you don't need to compare == True, nor
# do you need to wrap if conditions in parens.
if ch:
def f(elem):
self.Q.append(elem)
else:
def f(elem):
heappush(self.Q, elem)
return f
There are a few other problems here. First, you definitely need to set self.Q = deque() instead of self.Q = [] if you wanted a deque. And you probably want to wrap these functions up as a types.MethodType instead of using self as a closure variable (this will work, it's just less readable, because it may not be clear to many people why it works). And so on. But this is the fundamental problem.
For example:
from collections import deque
from heapq import heappush
class MyClass(object):
def __init__(self, choose=True):
self.Q = deque() if choose else []
self.add = self.genAdd(choose)
def genAdd(self, ch):
if ch:
def f(elem):
self.Q.append(elem)
else:
def f(elem):
heappush(self.Q, elem)
return f
d = MyClass(True)
d.add(3)
d.add(2)
print(d.Q)
h = MyClass(False)
h.add(3)
h.add(2)
print(h.Q)
This will print:
deque([3, 2])
[2, 3]
That being said, there's probably a much better design: Create a class that wraps a deque in your interface. Create another class that wraps a list with heapq in your interface. Create a factory function that returns one or the other:
class _MyClassDeque(object):
def __init__(self):
self.Q = deque()
def add(self, elem):
self.Q.append(elem)
class _MyClassHeap(object):
def __init__(self):
self.Q = []
def add(self, elem):
heappush(self.Q, elem)
def MyClass(choose=True):
return _MyClassDeque() if choose else _MyClassHeap()
Now you get the same results, but the code is a lot easier to understand (and slightly more efficient, if you care…).
I have written a custom container object.
According to this page, I need to implement this method on my object:
__iter__(self)
However, upon following up the link to Iterator Types in the Python reference manual, there are no examples given of how to implement your own.
Can someone post a snippet (or link to a resource), that shows how to do this?
The container I am writing, is a map (i.e. stores values by unique keys).
dicts can be iterated like this:
for k, v in mydict.items()
In this case I need to be able to return two elements (a tuple?) in the iterator.
It is still not clear how to implement such an iterator (despite the several answers that have been kindly provided). Could someone please shed some more light on how to implement an iterator for a map-like container object? (i.e. a custom class that acts like a dict)?
I normally would use a generator function. Each time you use a yield statement, it will add an item to the sequence.
The following will create an iterator that yields five, and then every item in some_list.
def __iter__(self):
yield 5
yield from some_list
Pre-3.3, yield from didn't exist, so you would have to do:
def __iter__(self):
yield 5
for x in some_list:
yield x
Another option is to inherit from the appropriate abstract base class from the `collections module as documented here.
In case the container is its own iterator, you can inherit from
collections.Iterator. You only need to implement the next method then.
An example is:
>>> from collections import Iterator
>>> class MyContainer(Iterator):
... def __init__(self, *data):
... self.data = list(data)
... def next(self):
... if not self.data:
... raise StopIteration
... return self.data.pop()
...
...
...
>>> c = MyContainer(1, "two", 3, 4.0)
>>> for i in c:
... print i
...
...
4.0
3
two
1
While you are looking at the collections module, consider inheriting from Sequence, Mapping or another abstract base class if that is more appropriate. Here is an example for a Sequence subclass:
>>> from collections import Sequence
>>> class MyContainer(Sequence):
... def __init__(self, *data):
... self.data = list(data)
... def __getitem__(self, index):
... return self.data[index]
... def __len__(self):
... return len(self.data)
...
...
...
>>> c = MyContainer(1, "two", 3, 4.0)
>>> for i in c:
... print i
...
...
1
two
3
4.0
NB: Thanks to Glenn Maynard for drawing my attention to the need to clarify the difference between iterators on the one hand and containers that are iterables rather than iterators on the other.
usually __iter__() just return self if you have already define the next() method (generator object):
here is a Dummy example of a generator :
class Test(object):
def __init__(self, data):
self.data = data
def next(self):
if not self.data:
raise StopIteration
return self.data.pop()
def __iter__(self):
return self
but __iter__() can also be used like this:
http://mail.python.org/pipermail/tutor/2006-January/044455.html
The "iterable interface" in python consists of two methods __next__() and __iter__(). The __next__ function is the most important, as it defines the iterator behavior - that is, the function determines what value should be returned next. The __iter__() method is used to reset the starting point of the iteration. Often, you will find that __iter__() can just return self when __init__() is used to set the starting point.
See the following code for defining a Class Reverse which implements the "iterable interface" and defines an iterator over any instance from any sequence class. The __next__() method starts at the end of the sequence and returns values in reverse order of the sequence. Note that instances from a class implementing the "sequence interface" must define a __len__() and a __getitem__() method.
class Reverse:
"""Iterator for looping over a sequence backwards."""
def __init__(self, seq):
self.data = seq
self.index = len(seq)
def __iter__(self):
return self
def __next__(self):
if self.index == 0:
raise StopIteration
self.index = self.index - 1
return self.data[self.index]
>>> rev = Reverse('spam')
>>> next(rev) # note no need to call iter()
'm'
>>> nums = Reverse(range(1,10))
>>> next(nums)
9
If your object contains a set of data you want to bind your object's iter to, you can cheat and do this:
>>> class foo:
def __init__(self, *params):
self.data = params
def __iter__(self):
if hasattr(self.data[0], "__iter__"):
return self.data[0].__iter__()
return self.data.__iter__()
>>> d=foo(6,7,3,8, "ads", 6)
>>> for i in d:
print i
6
7
3
8
ads
6
To answer the question about mappings: your provided __iter__ should iterate over the keys of the mapping. The following is a simple example that creates a mapping x -> x * x and works on Python3 extending the ABC mapping.
import collections.abc
class MyMap(collections.abc.Mapping):
def __init__(self, n):
self.n = n
def __getitem__(self, key): # given a key, return it's value
if 0 <= key < self.n:
return key * key
else:
raise KeyError('Invalid key')
def __iter__(self): # iterate over all keys
for x in range(self.n):
yield x
def __len__(self):
return self.n
m = MyMap(5)
for k, v in m.items():
print(k, '->', v)
# 0 -> 0
# 1 -> 1
# 2 -> 4
# 3 -> 9
# 4 -> 16
In case you don't want to inherit from dict as others have suggested, here is direct answer to the question on how to implement __iter__ for a crude example of a custom dict:
class Attribute:
def __init__(self, key, value):
self.key = key
self.value = value
class Node(collections.Mapping):
def __init__(self):
self.type = ""
self.attrs = [] # List of Attributes
def __iter__(self):
for attr in self.attrs:
yield attr.key
That uses a generator, which is well described here.
Since we're inheriting from Mapping, you need to also implement __getitem__ and __len__:
def __getitem__(self, key):
for attr in self.attrs:
if key == attr.key:
return attr.value
raise KeyError
def __len__(self):
return len(self.attrs)
One option that might work for some cases is to make your custom class inherit from dict. This seems like a logical choice if it acts like a dict; maybe it should be a dict. This way, you get dict-like iteration for free.
class MyDict(dict):
def __init__(self, custom_attribute):
self.bar = custom_attribute
mydict = MyDict('Some name')
mydict['a'] = 1
mydict['b'] = 2
print mydict.bar
for k, v in mydict.items():
print k, '=>', v
Output:
Some name
a => 1
b => 2
example for inhert from dict, modify its iter, for example, skip key 2 when in for loop
# method 1
class Dict(dict):
def __iter__(self):
keys = self.keys()
for i in keys:
if i == 2:
continue
yield i
# method 2
class Dict(dict):
def __iter__(self):
for i in super(Dict, self).__iter__():
if i == 2:
continue
yield i