We Keep Coding

Accessing iterator in 'for in' loop

From my understanding, when code like the following is run:
for i in MyObject:
print(i)
MyObject's __iter__ function is run, and the for loop uses the iterator it returns to run the loop.
is it possible to access this iterator object mid-loop? Is it a hidden local variable, or something like that?
I would like to do the following:
for i in MyObject:
blah = forloopiterator()
modify_blah(blah)
print(i)
I want to do this because I am building a debugger, and I need to modify the iterator after it has been instantiated (adding an object to be iterated during this loop, mid-execution). I am aware that this a hack and should not be done conventionally. Modifying MyObject.items (which is what the iterator is iterating over) directly doesn't work, sicne the iterator only evaluates once. So I need to modify the iterator directly.

It is possible to do what you want to do, as long as you're willing to rely on multiple undocumented internals of your Python interpreter (in my case, CPython 3.7)—but it isn't going to do you any good.
The iterator is not exposed to locals, or anywhere else (not even to a debugger). But as pointed out by Patrick Haugh, you can get at it indirectly, via get_referrers. For example:
for ref in gc.get_referrers(seq):
if isinstance(ref, collections.abc.Iterator):
break
else:
raise RuntimeError('Oops')
Of course if you have two different iterators to the same list, I don't know if there's any way you can decide between them, but let's ignore that problem.
Now, what do you do with this? You've got an iterator over seq, and… now what? You can't replace it with something useful, like an itertools.chain(seq, [1, 2, 3]). There's no public API for mutating list, set, etc. iterators, much less arbitrary iterators.
if you happen to know it's a list iterator… well, the CPython 3.x listiterator does happen to be mutable. The way they're pickled is by creating an empty iterator and calling __setstate__ with a reference to a list and an index:
>>> print(ref.__reduce__())
(<function iter>, ([0, 1, 2, 3, 4, 5, 6, 7, 8, 9],), 7)
>>> ref.__setstate__(3) # resets the iterator to index 3 instead of 7
>>> ref.__reduce__()[1][0].append(10) # adds another value
But this is all kind of silly, because you could get the same effect by just mutating the original list. In fact:
>>> ref.__reduce__()[1][0] is seq
True
So:
lst = list(range(10))
for elem in lst:
print(elem, end=' ')
if elem % 2:
lst.append(elem * 2)
print()
… will print out:
0 1 2 3 4 5 6 7 8 9 2 6 10 14 18
… without having to monkey with the iterator at all.
You can't do the same thing with a set.
Mutating a set while you're in the middle of iterating it will affect the iterator, just as mutating a list will—but what it does is indeterminate. After all, sets have arbitrary order, which is only guaranteed to be consistent as long as you don't add or delete. What happens if you add or delete in the middle? You may get a whole different order, meaning you may end up repeating elements you already iterated, and missing ones you never saw. Python implies that this should be illegal in any implementation, and CPython does actually check it:
s = set(range(10))
for elem in s:
print(elem, end=' ')
if elem % 2:
s.add(elem * 2)
print()
This will just immediately raise:
RuntimeError: Set changed size during iteration
So, what happens if we use the same trick to go behind Python's back, find the set_iterator, and try to change it?
s = {1, 2, 3}
for elem in s:
print(elem)
for ref in gc.get_referrers(seq):
if isinstance(ref, collections.abc.Iterator):
break
else:
raise RuntimeError('Oops')
print(ref.__reduce__)
What you'll see in this case will be something like:
2
(<function iter>, ([1, 3],))
1
(<function iter>, ([3],))
3
(<function iter>, ([],))
In other words, when you pickle a set_iterator, it creates a list of the remaining elements, and gives you back instructions to build a new listiterator out of that list. Mutating that temporary list obviously has no useful effect.
What about a tuple? Obviously you can't just mutate the tuple itself, because tuples are immutable. But what about the iterator?
Under the covers, in CPython, tuple_iterator shares the same structure and code as listiterator (as does the iterator type that you get from calling iter on an "old-style sequence" type that defines __len__ and __getitem__ but not __iter__). So, you can do the exact same trick to get at the iterator, and toreduce` it.
But once you do, ref.__reduce__()[1][0] is seq is going to be true again—in other words, it's a tuple, the same tuple you already had, and still immutable.

No, it is not possible to access this iterator (unless maybe with the Python C API, but that is just a guess). If you need it, assign it to a variable before the loop.
it = iter(MyObject)
for i in it:
print(i)
# do something with it
Keep in mind that manually advancing the iterator can raise a StopIteration exception.
for i in it:
if check_skip_next_element(i):
try: next(it)
except StopIteration: break
The use of break is discussable. In this case it has the same semantics as continue but you may just use pass if you want to keep going until the end of the for-block.

If you want to insert an additional object into a loop mid-iteration in a debugger, you don't need to do it by modifying the iterator. Instead, after the end of the loop, jump to the first line of the loop body, then set the loop variable to the object you want. Here's a PDB example. With the following file:
import pdb
def f():
pdb.set_trace()
for i in range(5):
print(i)
f()
I've recorded a debugging session that inserts a 15 into the loop:
> /tmp/asdf.py(5)f()
-> for i in range(5):
(Pdb) n
> /tmp/asdf.py(6)f()
-> print(i)
(Pdb) n
0
> /tmp/asdf.py(5)f()
-> for i in range(5):
(Pdb) j 6
> /tmp/asdf.py(6)f()
-> print(i)
(Pdb) i = 15
(Pdb) n
15
> /tmp/asdf.py(5)f()
-> for i in range(5):
(Pdb) n
> /tmp/asdf.py(6)f()
-> print(i)
(Pdb) n
1
> /tmp/asdf.py(5)f()
-> for i in range(5):
(Pdb) c
2
3
4
(Due to a PDB bug, you have to jump, then set the loop variable. PDB will lose the change to the loop variable if you jump immediately after setting it.)

If you are not aware of the pdb debugger in python, please give it a try. It's a very interactive debugger I have ever come across.
python debugger
I am sure we can control the loop iterations manually with pdb. But altering list mid way, not sure. Give it a try.

To access the iterator of a given object, you can use the iter() built-in function.
>>> it = iter(MyObject)
>>> it.next()

python newbie: for j in range

I am new to learn python these days. While reading a book, I found a line of code that I can't understand.
Please see line 46 under print_progression() method, print(' '.join(str(next(self)) for j in range(n))).
class Progression:
'''Iterator producing a generic progression.
Default iterator produces the whole number, 0, 1, 2, ...
'''
def __init__(self, start = 0):
'''
Initialize current to the first value of the progression.
'''
self._current = start
def _advance(self):
'''
Update self.current to a new value.
This should be overriden by a subclass to customize progression.
By convension, if current is set to None, this designates the
end of a finite progression.
'''
self._current += 1
def __next__(self):
'''
Return the next element, or else raise StopIteration error.
'''
# Our convention to end a progression
if self._current is None:
raise StopIteration()
else:
# record current value to return
answer = self._current
# advance to prepare for next time
self._advance()
# return the answer
return answer
def __iter__(self):
'''
By convention, an iterator must return itself as an iterator.
'''
return self
def print_progression(self, n):
'''
Print next n values of the progression.
'''
print(' '.join(str(next(self)) for j in range(n)))
class ArithmeticProgression(Progression): # inherit from Progression
pass
if __name__ == '__main__':
print('Default progression:')
Progression().print_progression(10)
'''Output is
Default progression:
0 1 2 3 4 5 6 7 8 9 10'''
I have no idea how next(self) and j works.
I think it should be str(Progression.next()). (solved)
I cannot find j anywhere. What is j for? Why not using while loop such as while Progression.next() <= range(n)?
For my final thought, it should be
print(' '.join(str(next(self)) while next(self) <= range(n)))
Save this newbie.
Thanks in advance!

I think #csevier added a reasonable discussion about your first question, but I'm not sure the second question is answered as clearly for you based on your comments so I'm going to try a different angle.
Let's say you did:
for x in range(10):
print(x)
That's reasonably understandable - you created a list [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] and you printed each of the values in that list in-turn. Now let's say that we wanted to just print "hello" 10 times; well we could modify our existing code very simply:
for x in range(10):
print(x)
print('hello')
Umm, but now the x is messing up our output. There isn't a:
do this 10 times:
print('hello')
syntax. We could use a while loop but that means defining an extra counter:
loop_count = 0
while loop_count < 10:
print('hello')
loop_count += 1
That's baggage. So, the better way would be just to use for x in range(10): and just not bother doing print(x); the value is there to make our loop work, not because it's actually useful in any other way. This is the same for j (though I've used x in my examples because I think you're more likely to encounter it in tutorials, but you could use almost any name you want). Also, while loops are generally used for loops that can run indefinitely, not for iterating over an object with fixed size: see here.

Welcome to the python community! This is a great question. In python, as in other languages, there are many ways to do things. But when you follow a convention that the python community does, that is often referred to as a "pythonic" solution. The method print_progression is a common pythonic solution to iteration of a user defined data structure. In the case above, lets explain first how the code works and then why we would do it that way.
Your print_progression method takes advantage of the fact that your Progression class implements the iteration protocol by implementing the next and iter dunder/magic methods. Because those are implemented you can iterate your class instance both internally as next(self) has done, and externally next(Progression()) which is the exactly what you were getting at with you number 1. Because this protocol is implemented already, this class can by used in any builtin iterator and generator context for any client! Thats a polymorphic solution. Its just used internally as well because you don't need to do it in 2 different ways.
Now for the unused J variable. They are just using that so they can use the for loop. Just using range(n) would just return an itterable but not iterate over it. I dont quite agree with the authors use of the variable named J, its often more common to denote an unused variable that is just used because it needs to be as a single underscore. I like this a little better:
print(' '.join(str(next(self)) for _ in range(n)))

Function that returns an accumulator in Python

I am reading Hackers and Painters and am confused by a problem mentioned by the author to illustrate the power of different programming languages.
The problem is:
We want to write a function that generates accumulators—a function that takes a number n, and returns a function that takes another number i and returns n incremented by i. (That’s incremented by, not plus. An accumulator has to accumulate.)
The author mentions several solutions with different programming languages. For example, Common Lisp:
(defun foo (n)
(lambda (i) (incf n i)))
and JavaScript:
function foo(n) { return function (i) { return n += i } }
However, when it comes to Python, the following codes do not work:
def foo(n):
s = n
def bar(i):
s += i
return s
return bar
f = foo(0)
f(1) # UnboundLocalError: local variable 's' referenced before assignment
A simple modification will make it work:
def foo(n):
s = [n]
def bar(i):
s[0] += i
return s[0]
return bar
I am new to Python. Why doesn the first solution not work while the second one does? The author mentions lexical variables but I still don't get it.

s += i is just sugar for s = s + i.*
This means you assign a new value to the variable s (instead of mutating it in place). When you assign to a variable, Python assumes it is local to the function. However, before assigning it needs to evaluate s + i, but s is local and still unassigned -> Error.
In the second case s[0] += i you never assign to s directly, but only ever access an item from s. So Python can clearly see that it is not a local variable and goes looking for it in the outer scope.
Finally, a nicer alternative (in Python 3) is to explicitly tell it that s is not a local variable:
def foo(n):
s = n
def bar(i):
nonlocal s
s += i
return s
return bar
(There is actually no need for s - you could simply use n instead inside bar.)
*The situation is slightly more complex, but the important issue is that computation and assignment are performed in two separate steps.

An infinite generator is one implementation. You can call __next__ on a generator instance to extract successive results iteratively.
def incrementer(n, i):
while True:
n += i
yield n
g = incrementer(2, 5)
print(g.__next__()) # 7
print(g.__next__()) # 12
print(g.__next__()) # 17
If you need a flexible incrementer, one possibility is an object-oriented approach:
class Inc(object):
def __init__(self, n=0):
self.n = n
def incrementer(self, i):
self.n += i
return self.n
g = Inc(2)
g.incrementer(5) # 7
g.incrementer(3) # 10
g.incrementer(7) # 17

In Python if we use a variable and pass it to a function then it will be Call by Value whatever changes you make to the variable it will not be reflected to the original variable.
But when you use a list instead of a variable then the changes that you make to the list in the functions are reflected in the original List outside the function so this is called call by reference.
And this is the reason for the second option does work and the first option doesn't.

Pythonic enumeration of while loop

Python has an elegant way of automatically generating a counter variable in for loops: the enumerate function. This saves the need of initializing and incrementing a counter variable. Counter variables are also ugly because they are often useless once the loop is finished, yet their scope is not the scope of the loop, so they occupy the namespace without need (although I am not sure whether enumerate actually solves this).
My question is, whether there is a similar pythonic solution for while loops. enumerate won't work for while loops since enumerate returns an iterator. Ideally, the solution should be "pythonic" and not require function definitions.
For example:
x=0
c=0
while x<10:
x=int(raw_input())
print x,c
c+=1
In this case we would want to avoid initializing and incrementing c.
Clarification:
This can be done with an endless for loop with manual termination as some have suggested, but I am looking for a solution that makes the code clearer, and I don't think that solution makes the code clearer in this case.

Improvement (in readability, I'd say) to Ignacio's answer:
x = 0
for c in itertools.takewhile(lambda c: x < 10, itertools.count()):
x = int(raw_input())
print x, c
Advantages:
Only the while loop condition is in the loop header, not the side-effect raw_input.
The loop condition can depend on any condition that a normal while loop could. It's not necessary to "import" the variables referenced into the takewhile, as they are already visible in the lambda scope. Additionally it can depend on the count if you want, though not in this case.
Simplified: enumerate no longer appears at all.

Again with the itertools...
import itertools
for c, x in enumerate(
itertools.takewhile(lambda v: v < 10,
(int(raw_input()) for z in itertools.count())
)
):
print c, x

If you want zero initialization before the while loop, you can use a Singleton with a counter:
class Singleton(object):
_instance = None
def __new__(cls, *args, **kwargs):
if not cls._instance:
cls._instance = super(Singleton, cls).__new__(
cls, *args, **kwargs)
cls.count=0
else:
cls.count+=1
return cls._instance
Then there will only be one instance of Singleton and each additional instance just adds one:
>>> Singleton().count # initial instance
0
>>> Singleton().count
1
>>> Singleton().count
2
>>> Singleton().count
3
Then your while loop becomes:
while Singleton():
x=int(raw_input('x: '))
if x>10: break
print 'While loop executed',Singleton().count,'times'
Entering 1,2,3,11 it prints:
x: 1
x: 2
x: 3
x: 11
While loop executed 4 times
If you do not mind a single line initialization before the while loop, you can just subclass an interator:
import collections
class WhileEnum(collections.Iterator):
def __init__(self,stop=None):
self.stop=stop
self.count=0
def next(self): # '__next__' on Py 3, 'next' on Py 2
if self.stop is not None:
self.remaining=self.stop-self.count
if self.count>=self.stop: return False
self.count+=1
return True
def __call__(self):
return self.next()
Then your while loop becomes:
enu=WhileEnum()
while enu():
i=int(raw_input('x: '))
if i>10: break
print enu.count
I think the second is the far better approach. You can have multiple enumerators and you can also set a limit on how many loops to go:
limited_enum=WhileEnum(5)

I don't think it's possible to do what you want in the exact way you want it. If I understand right, you want a while loop that increments a counter each time through, without actually exposing a visible counter outside the scope of the loop. I think the way to do this would be to rewrite your while loop as a nonterminating for loop, and check the end condition manually. For your example code:
import itertools
x = 0
for c in itertools.count():
x = int(raw_input())
print x, c
if x >= 10:
break
The problem is that fundamentally you're doing iteration, with the counter. If you don't want to expose that counter, it needs to come from the loop construct. Without defining a new function, you're stuck with a standard loop and an explicit check.
On the other hand, you could probably also define a generator for this. You'd still be iterating, but you could at least wrap the check up in the loop construct.

Python functions within lists

So today in computer science I asked about using a function as a variable. For example, I can create a function, such as returnMe(i) and make an array that will be used to call it. Like h = [help,returnMe] and then I can say h1 and it would call returnMe("Bob"). Sorry I was a little excited about this. My question is is there a way of calling like h.append(def function) and define a function that only exists in the array?
EDIT:
Here Is some code that I wrote with this!
So I just finished an awesome FizzBuzz with this solution thank you so much again! Here's that code as an example:
funct = []
s = ""
def newFunct(str, num):
return (lambda x: str if(x%num==0) else "")
funct.append(newFunct("Fizz",3))
funct.append(newFunct("Buzz",5))
for x in range(1,101):
for oper in funct:
s += oper(x)
s += ":"+str(x)+"\n"
print s

You can create anonymous functions using the lambda keyword.
def func(x,keyword='bar'):
return (x,keyword)
is roughly equivalent to:
func = lambda x,keyword='bar':(x,keyword)
So, if you want to create a list with functions in it:
my_list = [lambda x:x**2,lambda x:x**3]
print my_list[0](2) #4
print my_list[1](2) #8

Not really in Python. As mgilson shows, you can do this with trivial functions, but they can only contain expressions, not statements, so are very limited (you can't assign to a variable, for example).
This is of course supported in other languages: in Javascript, for example, creating substantial anonymous functions and passing them around is a very idiomatic thing to do.

You can create the functions in the original scope, assign them to the array and then delete them from their original scope. Thus, you can indeed call them from the array but not as a local variable. I am not sure if this meets your requirements.
#! /usr/bin/python3.2
def a (x): print (x * 2)
def b (x): print (x ** 2)
l = [a, b]
del a
del b
l [0] (3) #works
l [1] (3) #works
a (3) #fails epicly

You can create a list of lambda functions to increment by every number from 0 to 9 like so:
increment = [(lambda arg: (lambda x: arg + x))(i) for i in range(10)]
increment[0](1) #returns 1
increment[9](10) #returns 19
Side Note:
I think it's also important to note that this (function pointers not lambdas) is somewhat like how python holds methods in most classes, except instead of a list, it's a dictionary with function names pointing to the functions. In many but not all cases instance.func(args) is equivalent to instance.__dict__['func'](args) or type(class).__dict__['func'](args)