I am currently working on LinkedList and I have the following code and I don't understand what the __iter__ and __repr__ are doing exactly?
class Node:
def __init__(self, value):
self.value = value
self.next = None
class LinkedList:
def __init__(self):
self.head = None
def append(self, value):
if self.head is None:
self.head = Node(value)
return
node = self.head
while node.next:
node = node.next
node.next = Node(value)
def __iter__(self):
node = self.head
while node:
yield node.value
node = node.next
def __repr__(self):
return str([v for v in self])
Here I am creating the LinkedList and append the values at the end of my list.
llist = LinkedList()
for value in [4,2,5,1,-3,0]:
llist.append(value)
If I print the list print(llist) then I am getting [4, 2, 5, 1, -3, 0]
I guess this is coming from __iter__ and __repr__. What I don't understand is when my __iter__ and __repr__ starts and which is running first? How can I print objects outside my class?
From my limited understanding of Python internals, when you print() an object, the str() function is called on it. This function probably knows how to format some kinds of objects, such as strings and numbers, for others, it calls their __repr__() method which is supposed to return a string. So __repr__() is called first.
The __repr__() method was written by the developer to return a normal list representation which is created by the list comprehension [v for v in self]. The iteration ultimately calls the __iter__() method which is a generator function (as indicated by the use of yield). This function iterates over the elements of the list, and every yield makes one element available to the for ... in ... construct.
Read the docs.
iter: defines on what attribute/how iterating on a class' object is supposed to work - e.g. what is "next". If you wanted say to iterate on animals, and you wanted the iteration to be defined by each animal's weight, then you could use self.weight in your animal class much in the way that you're using self.value right now....
_ repr _: When you do print(my_Object), Python by default has a representation (repr) defined. You can re-define using this function.
when you are implementing the iter method, you are changing the behaviour of looping throught your list.
what I mean is in the normal for loop when you call:
for l in list:
print(l)
this what will happen behind the scene :
# create an iterator object from that iterable
iter_obj = iter(list)
# infinite loop
while True:
try:
# get the next item
element = next(iter_obj)
# do something with element
except StopIteration:
# if StopIteration is raised, break from loop
break
so if you redefine the iter function inside your linked list class you are redefining the iteration function and what should iterating through your iterator returns.
same basically for repr you redefine the represantation of your object. for example when you call print(obj) you get the obj printed but you can change it to the format you want to if you implement repr correctly as you wish.
take a look here for more explaination:
https://www.programiz.com/python-programming/iterator
https://www.pythonforbeginners.com/basics/str-vs-__repr
__iter__ represents the iterator method of the class. Let's understand through code:
llist = LinkedList()
for value in [4,2,5,1,-3,0]:
llist.append(value)
for value in llist:
print(value)
Output
4
2
5
1
-3
0
Here, when we take the object as an iterator, __iter__() method would be into action which iterates and traverse linked list and yields node values.
print(llist)
Output
[4, 2, 5, 1, -3, 0]
Here, __repr__() method would be into action and would print node values as specified in the code.
Related
I was working on the sorting but I'm not able to call the function with the specific way.
Basically, what I want to do is to create a function that takes a list of object Node with attribute Value and returns a list with the items from the original list stored into sublists. Items of the same value should be in the same sublist and sorted in descending order.
For continuing the code I want to know what should be the parameter of this.
def advanced_sort(<What will come here according to the call>):
Function call:
advanced_sort([Node(1), Node(2), Node(1),Node(2)])
Can anyone please help me out with the code? Thanks in advance.
advanced_sort takes a single argument: a list (or possibly an arbitrary iterable). As such, the signature only has one argument:
def advanced_sort(nodes):
Ignoring type hints, the signature does not and cannot reflect the internal structure of the single argument; it's just a name to refer to the passed value inside the body of the function.
Inside the body, you can write code that assumes that nodes is a list, and that further each element of the list is a Node instance, so that you can do things like assume each value as a Value attribute.
def advanced_sort(nodes):
# If nodes is iterable, then x refers to a different
# element of the iterable each time through the loop.
for x in nodes:
# If nodes is a list of Node instances, then
# x is a Node instance, and thus you can access
# its Value attribute in the normal fashion.
print("Found value {}".format(x.Value))
Assuming a definition of Node like
class Node:
def __init__(self, v):
self.Value = v
the above definition of advanced_sort will produce the following output:
>>> advanced_sort([Node(3), Node(2), Node(1),Node(2)])
Found value 1
Found value 2
Found value 3
Found value 4
The argument is a single iterable object such as a list, a tuple, a set, ...
Then you iterate on the items as in chepner's response.
For exemple you can use a dictionary to group the Nodes by value:
def advanced_sort(node_list):
ret = dict()
for node in node_list:
if node.value not in ret.keys():
ret[node.value] = list()
ret[node.value].append(node)
return [ret[value] for value in sorted(ret.keys(), reverse=True)] #descending order
advanced_sort([Node(3), Node(2), Node(1),Node(1)])
>>> [[Node(3)], [Node(2)], [Node(1),Node(1)]]
Are you able to make changes to the Node class? In that case, you could do something like this:
from functools import total_ordering
#total_ordering
class Node:
def __init__(self, value):
self.value = value
def __eq__(self, other):
if not isinstance(other, Node):
return NotImplemented
return self.value == other.value
def __lt__(self, other):
if not isinstance(other, Node):
return NotImplemented
return self.value < other.value
def __str__(self):
return f"({self.value})"
def main():
from itertools import groupby
nodes = [Node(1), Node(2), Node(1), Node(2)]
nodes_sorted = sorted(nodes, reverse=True)
nodes_sublists = [list(group) for key, group in groupby(nodes_sorted)]
for sublist in nodes_sublists:
print(*map(str, sublist))
return 0
if __name__ == "__main__":
import sys
sys.exit(main())
Output:
(2) (2)
(1) (1)
How could I recursively iterate through nodes with reference to a previous node? Expecting output 4,3,2,1 in the example below:
class Node:
def __init__(self, parent, value):
self.parent = parent
self.value = value
def append(self, value):
return Node(self, value)
def list(l):
print(l.value)
while l.parent is not None:
list(l.parent)
l = Node(None, 1)
l = l.append(2)
l = l.append(3)
l = l.append(4)
list(l)
Your class structure already succesfully passes the node's self value to its child node. The problem is your list function. while l.parent is not None: never ends, because nothing in the loop is changing the value of l. Calling list recursively will create a new context where another variable named l has a different value from the first context's l, but this has no effect on the the first l or the first loop. Recursive functions generally do not require an actual loop in order to iterate over the elements of a data structure. Try:
def list(l):
print(l.value)
if l.parent is not None:
list(l.parent)
Or:
def list(l):
while l is not None:
print(l.value)
l = l.parent
(I recommend the latter because the first one will crash with "maximum recursion depth exceeded" if the chain has more than 999 elements)
Result:
4
3
2
1
Bonus style tip: consider naming your function something other than list. In general you should avoid overwriting the names of built-in functions and types.
I should vote to close your question for the lack of a clear problem statement, but anyway...
Within an object in Python, how can I pass a reference of my current object
The same way as you'd do with just any object.
to object b of the same class
This is actually irrelevant but anyway...
such that when I call b.parent, I can get back to object a?
class Foo(object):
def __init__(self, parent=None):
self.parent = parent
a = Foo()
b = Foo(a)
print(b.parent is a)
Now for the answer to the question you didn't ask, see (and accept) Kevin's answer ;-)
class Node:
def __init__(self, value):
self._value = value
self._children = []
def __repr__(self):
return 'Node({!r})'.format(self._value)
def add_child(self, node):
self._children.append(node)
def __iter__(self):
return iter(self._children)
def depth_first(self):
yield self
for c in self:
yield from c.depth_first()
if __name__ == '__main__':
root = Node(0)
child1 = Node(1)
child2 = Node(2)
root.add_child(child1)
root.add_child(child2)
child1.add_child(Node(3))
child1.add_child(Node(4))
child2.add_child(Node(5))
for a in root.depth_first():
print(a)
# Outputs Node(0), Node(1), Node(3), Node(4), Node(2), Node(5)
I thought that a list is an object that we can iterate over it so why use iter() ? I am new in python so this look to me so weird.
Because returning self._children returns a list object, which doesn't work as an iterator, remember, iterators implement the __next__ method in order to supply items during iteration:
>>> next(list()) # doesn't implement a __next__ method
TypeError: 'list' object is not an iterator
lists are iterable, they can be iterated through because calling iter on the will return an iterator, but, lists themselves are not iterators -- a good breakdown of these can be found in the top answer of this Question.
A lists __iter__ method returns a custom new list_iterator object each time __iter__ is called:
list().__iter__()
Out[93]: <list_iterator at 0x7efe7802d748>
and, by doing that, supports iteration multiple times, the list_iterator object implements the __next__ method that's required.
Calling and returning iter on it will just do that, return the lists iterator and save you the trouble of having to implement __next__.
Addressing your comment as for why for c in self._children works, well, because it is essentially doing the same thing. What basically happens with the for loop is:
it = iter(self._children) # returns the list iterator
while True:
try:
i = next(it)
<loop body>
except StopIteration:
break
meaning, iter is called again on the list object and is used by the for loop for next calls.
Every iterable object implements an iter() function that returns itself.
the iter(obj) calls the obj.iter(), which is same as returning the list object in your example.
I've got the following wrapper for a dictionary:
class MyDict:
def __init__(self):
self.container = {}
def __setitem__(self, key, value):
self.container[key] = value
def __getitem__(self, key):
return self.container[key]
def __iter__(self):
return self
def next(self):
pass
dic = MyDict()
dic['a'] = 1
dic['b'] = 2
for key in dic:
print key
My problem is that I don't know how to implement the next method to make MyDict iterable. Any advice would be appreciated.
Dictionaries are themselves not an iterator (which can only be iterated over once). You usually make them an iterable, an object for which you can produce multiple iterators instead.
Drop the next method altogether, and have __iter__ return an iterable object each time it is called. That can be as simple as just returning an iterator for self.container:
def __iter__(self):
return iter(self.container)
If you must make your class an iterator, you'll have to somehow track a current iteration position and raise StopIteration once you reach the 'end'. A naive implementation could be to store the iter(self.container) object on self the first time __iter__ is called:
def __iter__(self):
return self
def next(self):
if not hasattr(self, '_iter'):
self._iter = iter(self.container)
return next(self._iter)
at which point the iter(self.container) object takes care of tracking iteration position for you, and will raise StopIteration when the end is reached. It'll also raise an exception if the underlying dictionary was altered (had keys added or deleted) and iteration order has been broken.
Another way to do this would be to just store in integer position and index into list(self.container) each time, and simply ignore the fact that insertion or deletion can alter the iteration order of a dictionary:
_iter_index = 0
def __iter__(self):
return self
def next(self):
idx = self._iter_index
if idx is None or idx >= len(self.container):
# once we reach the end, all iteration is done, end of.
self._iter_index = None
raise StopIteration()
value = list(self.container)[idx]
self._iter_index = idx + 1
return value
In both cases your object is then an iterator that can only be iterated over once. Once you reach the end, you can't restart it again.
If you want to be able to use your dict-like object inside nested loops, for example, or any other application that requires multiple iterations over the same object, then you need to implement an __iter__ method that returns a newly-created iterator object.
Python's iterable objects all do this:
>>> [1, 2, 3].__iter__()
<listiterator object at 0x7f67146e53d0>
>>> iter([1, 2, 3]) # A simpler equivalent
<listiterator object at 0x7f67146e5390>
The simplest thing for your objects' __iter__ method to do would be to return an iterator on the underlying dict, like this:
def __iter__(self):
return iter(self.container)
For more detail than you probably will ever require, see this Github repository.
When using list comprehension or the in keyword in a for loop context, i.e:
for o in X:
do_something_with(o)
or
l=[o for o in X]
How does the mechanism behind in works?
Which functions\methods within X does it call?
If X can comply to more than one method, what's the precedence?
How to write an efficient X, so that list comprehension will be quick?
The, afaik, complete and correct answer.
for, both in for loops and list comprehensions, calls iter() on X. iter() will return an iterable if X either has an __iter__ method or a __getitem__ method. If it implements both, __iter__ is used. If it has neither you get TypeError: 'Nothing' object is not iterable.
This implements a __getitem__:
class GetItem(object):
def __init__(self, data):
self.data = data
def __getitem__(self, x):
return self.data[x]
Usage:
>>> data = range(10)
>>> print [x*x for x in GetItem(data)]
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
This is an example of implementing __iter__:
class TheIterator(object):
def __init__(self, data):
self.data = data
self.index = -1
# Note: In Python 3 this is called __next__
def next(self):
self.index += 1
try:
return self.data[self.index]
except IndexError:
raise StopIteration
def __iter__(self):
return self
class Iter(object):
def __init__(self, data):
self.data = data
def __iter__(self):
return TheIterator(data)
Usage:
>>> data = range(10)
>>> print [x*x for x in Iter(data)]
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
As you see you need both to implement an iterator, and __iter__ that returns the iterator.
You can combine them:
class CombinedIter(object):
def __init__(self, data):
self.data = data
def __iter__(self):
self.index = -1
return self
def next(self):
self.index += 1
try:
return self.data[self.index]
except IndexError:
raise StopIteration
Usage:
>>> well, you get it, it's all the same...
But then you can only have one iterator going at once.
OK, in this case you could just do this:
class CheatIter(object):
def __init__(self, data):
self.data = data
def __iter__(self):
return iter(self.data)
But that's cheating because you are just reusing the __iter__ method of list.
An easier way is to use yield, and make __iter__ into a generator:
class Generator(object):
def __init__(self, data):
self.data = data
def __iter__(self):
for x in self.data:
yield x
This last is the way I would recommend. Easy and efficient.
X must be iterable. It must implement __iter__() which returns an iterator object; the iterator object must implement next(), which returns next item every time it is called or raises a StopIteration if there's no next item.
Lists, tuples and generators are all iterable.
Note that the plain for operator uses the same mechanism.
Answering question's comments I can say that reading source is not the best idea in this case. The code that is responsible for execution of compiled code (ceval.c) does not seem to be very verbose for a person that sees Python sources for the first time. Here is the snippet that represents iteration in for loops:
TARGET(FOR_ITER)
/* before: [iter]; after: [iter, iter()] *or* [] */
v = TOP();
/*
Here tp_iternext corresponds to next() in Python
*/
x = (*v->ob_type->tp_iternext)(v);
if (x != NULL) {
PUSH(x);
PREDICT(STORE_FAST);
PREDICT(UNPACK_SEQUENCE);
DISPATCH();
}
if (PyErr_Occurred()) {
if (!PyErr_ExceptionMatches(
PyExc_StopIteration))
break;
PyErr_Clear();
}
/* iterator ended normally */
x = v = POP();
Py_DECREF(v);
JUMPBY(oparg);
DISPATCH();
To find what actually happens here you need to dive into bunch of other files which verbosity is not much better. Thus I think that in such cases documentation and sites like SO are the first place to go while the source should be checked only for uncovered implementation details.
X must be an iterable object, meaning it needs to have an __iter__() method.
So, to start a for..in loop, or a list comprehension, first X's __iter__() method is called to obtain an iterator object; then that object's next() method is called for each iteration until StopIteration is raised, at which point the iteration stops.
I'm not sure what your third question means, and how to provide a meaningful answer to your fourth question except that your iterator should not construct the entire list in memory at once.
Maybe this helps (tutorial http://docs.python.org/tutorial/classes.html Section 9.9):
Behind the scenes, the for statement
calls iter() on the container object.
The function returns an iterator
object that defines the method next()
which accesses elements in the
container one at a time. When there
are no more elements, next() raises a
StopIteration exception which tells
the for loop to terminate.
To answer your questions:
How does the mechanism behind in works?
It is the exact same mechanism as used for ordinary for loops, as others have already noted.
Which functions\methods within X does it call?
As noted in a comment below, it calls iter(X) to get an iterator. If X has a method function __iter__() defined, this will be called to return an iterator; otherwise, if X defines __getitem__(), this will be called repeatedly to iterate over X. See the Python documentation for iter() here: http://docs.python.org/library/functions.html#iter
If X can comply to more than one method, what's the precedence?
I'm not sure what your question is here, exactly, but Python has standard rules for how it resolves method names, and they are followed here. Here is a discussion of this:
Method Resolution Order (MRO) in new style Python classes
How to write an efficient X, so that list comprehension will be quick?
I suggest you read up more on iterators and generators in Python. One easy way to make any class support iteration is to make a generator function for iter(). Here is a discussion of generators:
http://linuxgazette.net/100/pramode.html