I have a generator that will keep giving numbers that follow a specific formula. For sake of argument let's say this is the function:
# this is not the actual generator, just an example
def Generate():
i = 0
while 1:
yield i
i+=1
I then want to get a list of numbers from that generator that are below a certain threshold. I'm trying to figure out a pythonic way of doing this. I don't want to edit the function definition. I realize you could just use a while loop with your cutoff as the condition, but I'm wondering if there is a better way. I gave this a try, but soon realized why it wouldn't work.
l = [x for x in Generate() x<10000] # will go on infinitely
So is there a correct way of doing this.
Thanks
An itertools solution to create another iterator:
from itertools import takewhile
l = takewhile(lambda x: x < 10000, generate())
Wrap it in list() if you are sure you want a list:
l = list(takewhile(lambda x: x < 10000, generate()))
Or if you want a list and like inventing wheels:
l = []
for x in generate():
if x < 10000:
l.append(x)
else:
break
Wrap your generator within another generator:
def no_more_than(limit):
def limiter(gen):
for item in gen:
if item > limit:
break
yield item
return limiter
def fib():
a,b = 1,1
while 1:
yield a
a,b = b,a+b
cutoff_at_100 = no_more_than(100)
print list(cutoff_at_100(fib()))
Prints:
[1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
itertools.takewhile will only work until it comes across an item that does not fulfill the predicate. If you need to return all values from a possibly unordered iterable, I'd recommend using itertools.ifilter for Python 2.x as in
from itertools import ifilter
f = ifilter(lambda x: x < 400, gen())
f.next()
This filtered a generator yielding random integers between 0 and 400 as hoped.
FWIW itertools.ifilter was deprecated in Python 3.x in favour of the built-in filter() which has slightly different syntax for iterating
f = filter(lambda x: x < 400, gen())
next(f)
Wrap it on a zip generator of a range of the limit:
gen = range(100_000_000_000)
limit = 10
(z[1] for z in zip(range(limit), gen))
zip creates a tuple, that is the reason for z[1]
This may be used on for loops:
for g in (z[1] for z in zip(range(limit), gen)):
print(g)
Or you could use lambda:
wrap = lambda gen, limit: (z[1] for z in zip(range(limit), gen))
for g in wrap(gen, 10):
print(g)
Just use a counter for an infinite generator:
gen=Generate() # your generator function example
l=[gen.next() for i in range(100)]
But since it is a generator, use a generator expression:
seq=(gen.next() for i in xrange(100)) #need x in xrange in Python 2.x; 3.x use range
Edit
OK, then just use a controller:
def controler(gen,limit):
n=gen.next()
while n<limit:
yield n
n=gen.next()
seq=[i for i in controler(Generate(),100)]
replace Generate() with xrange(10000)
Related
Is it possible to use a generator or iterator in a while loop in Python? For example, something like:
i = iter(range(10))
while next(i):
# your code
The point of this would be to build iteration into the while loop statement, making it similar to a for loop, with the difference being that you can now additional logic into the while statement:
i = iter(range(10))
while next(i) and {some other logic}:
# your code
It then becomes a nice for loop/while loop hybrid.
Does anyone know how to do this?
In Python >= 3.8, you can do the following, using assignment expressions:
i = iter(range(10))
while (x := next(i, None)) is not None and x < 5:
print(x)
In Python < 3.8 you can use itertools.takewhile:
from itertools import takewhile
i = iter(range(10))
for x in takewhile({some logic}, i):
# do stuff
"Some logic" here would be a 1-arg callable receciving whatever next(i) yields:
for x in takewhile(lambda e: 5 > e, i):
print(x)
0
1
2
3
4
There are two problems with while next(i):
Unlike a for loop, the while loop will not catch the StopIteration exception that is raised if there is no next value; you could use next(i, None) to return a "falsey" value in that case, but then the while loop will also stop whenever the iterator returns an actual falsey value
The value returned by next will be consumed and no longer available in the loop's body. (In Python 3.8+, that could be solved with an assignment expression, see other answer.)
Instead, you could use a for loop with itertools.takewhile, testing the current element from the iterable, or just any other condition. This will loop until either the iterable is exhausted, or the condition evaluates to false.
from itertools import takewhile
i = iter(range(10))
r = 0
for x in takewhile(lambda x: r < 10, i):
print("using", x)
r += x
print("result", r)
Output:
using 0
...
using 4
result 10
You just need to arrange for your iterator to return a false-like value when it expires. E.g., if we reverse the range so that it counts down to 0:
>>> i = iter(range(5, -1, -1))
>>> while val := next(i):
... print('doing something here with value', val)
...
This will result in:
doing something here with value 5
doing something here with value 4
doing something here with value 3
doing something here with value 2
doing something here with value 1
a = iter(range(10))
try:
next(a)
while True:
print(next(a))
except StopIteration:
print("Stop iteration")
You can do
a = iter(range(10))
try:
a.next()
while True and {True or False logic}:
print("Bonjour")
a.next()
except StopIteration:
print("a.next() Stop iteration")
def nest(x, n):
a = []
for i in range(n):
a.append([x])
return a
print nest("hello", 5)
This gives an output
[['hello'], ['hello'], ['hello'], ['hello'], ['hello']]
The desired output is
[[[[["hello"]]]]]
Every turn through the loop you are adding to the list. You want to be further nesting the list, not adding more stuff onto it. You could do it something like this:
def nest(x, n):
for _ in range(n):
x = [x]
return x
Each turn through the loop, x has another list wrapped around it.
instead of appending you sould wrap x and call recursively the method till call number is lesser than n
def nest(x, n):
if n <= 0:
return x
else:
return [nest(x, n-1)]
Here is a pythonic recursion approach:
In [8]: def nest(x, n):
...: return nest([x], n-1) if n else x
DEMO:
In [9]: nest(3, 4)
Out[9]: [[[[3]]]]
In [11]: nest("Stackoverflow", 7)
Out[11]: [[[[[[['Stackoverflow']]]]]]]
I am writing a prime generator, which is different from anyone in this link
generator in Python generating prime numbers
Here is my code
def sequence():
i = 1
while True:
i += 2
yield i
def prime_generator(n):
i = 2
it = sequence()
while i < n:
it= filter(lambda x: x % i, it)
i = next(it)
yield i
when i run something like
for i in prime_generator(50):
print(i)
It never dump 15, 33, sth like that for me. In a word, It gives me 2 and all odd numbers. what goes wrong here?
The problem is that i inside the lambda isn't "fixed"; when i changes in the outside scope, the previously created lambda functions all use the new value, and so they all do the same check: see if the current value from sequence() is divisible by the last found prime. Which they never are.
Wrapping it into another lambda and then calling it so that the value of i can be fixed works:
def prime_generator(n):
i = 2
it = sequence()
while i < n:
it = (lambda i: filter(lambda x: x % i, it))(i)
i = next(it)
yield i
Edit: also I don't believe your code (nor this) does yield 2, but that can be trivially fixed.
Basically, I'm in following situation - I generate a list, e.g.
l = [2*x for x in range(10)]
which I iterate through later on multipletimes, e.g.
for i in l: print i # 0,2,4,6,8,10,12,14,16,18
for i in l: print i # 0,2,4,6,8,10,12,14,16,18
for i in l: print i # 0,2,4,6,8,10,12,14,16,18
The problem is that the list is way too large to fit into memory, hence I use its generator form, i.e.:
l = (2*x for x in range(10))
However, after this construction only first iteration works:
for i in l: print i # 0,2,4,6,8,10,12,14,16,18
for i in l: print i #
for i in l: print i #
Where is the problem? How may I iterate through it multipletimes?
Your generator is exhausted the first time. You should recreate your generator each time to renew it:
l = (2*x for x in range(10))
for i in l: print i
l = (2*x for x in range(10))
for i in l: print i
(Note: you should use xrange in python 2 because range creates a list in memory)
You can create also a shortcut function to help you or even a generator function:
def gen():
for i in range(10):
yield 2 * i
and then:
for i in gen(): print i
for i in gen(): print i
You can also iterate on the generator directly:
for i in (2*x for x in range(10)): print i
for i in (2*x for x in range(10)): print i
...
So I have a function like this:
def demo_range(limit):
value = 0
while value < limit:
yield value
value = value + 1
and this
def demo_generator_to_list(generator):
return [x for x in range(generator)]
Now in the demo_generator_to_list(generator) I need to fill some code to collapse a generator to:
[0,1,2,3]
from
demo_generator_to_list(demo_range(4))
Just pass the generator to a list() call:
def demo_generator_to_list(generator):
return list(generator)
The list() function will iterate over the generator and add all results to a new list object.
You could still use a list comprehension, but then you don't use range():
return [x for x in generator]
This has no advantage over using list(); it is just slower.