I have generator function and function that works witn results of the first one. For example:
def gen():
a = 2
b = 3
yield (a, b)
def func():
c = gen()[0]
d = gen()[1]
I have error "'gen()' is unscriptable"
How can I fix it and work with result of func?
You have two problems here.
First, generator objects are not sequences, they're iterators. And you can't index an iterator the way you can a sequence, by subscripting it like [1]. You can loop over them with a for statement or a comprehension, or manually call next on them until they're done, but you can't [1] them.
That's why you get an error message that says the generator object is not subscriptable.
Second, you didn't want to subscript the generator anyway. Your generator yields an iterable of multiple pairs. It happens to only yield once, but that's no different from a sequence with just one pair in it—it's still not the same thing as a pair.
Consider the nearest sequence equivalent:
def seq():
a = 2
b = 3
return [(a, b)]
Obviously seq()[0] is going to be the tuple (2, 3), and seq()[1] is going to be an IndexError. So, even if you could subscript generators, your code wouldn't make sense.
What you actually want to do is either take the first pair, or loop over all the pairs (I'm not sure which). And then you can do [0] and [1] to the/each pair.
So, either this:
def func():
for pair in gen():
c = pair[0]
d = pair[1]
… or this:
def func():
pair = next(gen())
c = pair[0]
d = pair[1]
Or, if you really wanted to call it twice for some reason, this:
def func():
for pair in gen():
c = pair[0]
for pair in gen():
d = pair[1]
… or this:
def func():
c = next(gen())[0]
d = next(gen())pair[1]
What you are trying to do is get the first and second element without iterating over an iterator. You need to iterate over it to get values from it like -
for i in gen():
c, d = i # you need this because you are returning a tuple
You can go through this post to learn more about iterators and generators
Related
Iterables like generators can only be iterated once:
def f():
for i in range(10):
yield i
a = f()
for x in a:
print(x) # prints x
for x in a:
print(x) # prints none
Iterables like list can be iterated many times:
a = list(range(10))
for x in a:
print(x) # prints x
for x in a:
print(x) # prints x
How can I tell if an iterable can only be iterated once or not?
The motivation for this question comes from the implementation of itertools.cycle:
def cycle(iterable):
# cycle('ABCD') --> A B C D A B C D A B C D ...
saved = []
for element in iterable:
yield element
saved.append(element)
while saved:
for element in saved:
yield element
If we can tell if an iterable can be iterated only once, we can make the implementation more memory-efficient:
def cycle(iterable):
it = iterable
if only_iterated_once(iterable):
it = list(iterable)
while True:
for element in it:
yield element
If the argument can be iterated multiple times, we don't need to save an additional copy.
The main difference between your examples is that in the generator example, a single iterator is created before the loops happen, then that same iterator is used twice. In the list example however, a new iterator is used for each loop.
In the first example, a generator is an iterator itself. When you do
a = f()
The call to f creates a generator (which is an iterator). When you give a to the for loops, they call iter on a, which returns itself. A short MCVE shows this easily:
l = [1]
i = iter(l)
j = iter(i)
print(i is j) # Prints True
One iterator is used for both loops. This means that by the time the second loop starts, the shared iterator will already be exhausted.
In the second example however, when for calls iter on a, a new iterator is created, each time; so two iterators are created. This means that each loop uses its own iterator, so the second loop isn't using an exhausted iterator.
In other words, the way to tell is to think about whether or not you're creating a new iterator with each use, or using an old iterator multiple times.
I have a function that returns a variable list of values and I know you can do this by using a tuple. To assign these variables you can then do something like a, b = func(..). However, if there is only one value returned you have to do a, = func(..) [notice the ,] rather than a = func(..). To achieve the latter you can include a test to see if there is one value to be returned or more (see example below) but I wonder if there is no easier or less verbose way to do this.
def foo(*args):
returnvalues = []
for arg in args:
arg += 100
returnvalues.append(arg)
if len(returnvalues) == 1:
return returnvalues[0]
else:
return tuple(returnvalues)
def baz(*args):
returnvalues = []
for arg in args:
arg += 100
returnvalues.append(arg)
return tuple(returnvalues)
a = foo(10)
b, c = foo(20, 30)
print(f'a={a}, b={b}, c={c}')
a, = baz(10)
b, c = baz(20, 30)
print(f'a={a}, b={b}, c={c}')
#
a=110, b=120, c=130
a=110, b=120, c=130
I believe you are referring to "tuple unpacking". Also known as destructive assignment. The word "tuple" is a bit of a misnomer as you can use any iterable / iterator. So returning a list is fine.
def f():
return [1]
(a,) = f()
b, = f()
You can also use list syntax on the left hand side. There's no difference to the byte code that is generated. It does make unpacking a single item look less like a syntax error in the case of b and slightly less verbose than a.
[c] = f()
I would avoid returning the value itself and not a list in the special case where only one argument is passed. The reason for this is it makes the code harder to be used in a generic manner. Any caller of the function needs to know how many arguments it's passing or check the return value (which is clumsy). For example:
result = f()
if isinstance(result, (list, tuple)):
smallest = min(result)
else:
smallest = result
# as opposed to this when you always return a list / tuple
smallest = min(f())
You can assign the returning value of such a function to a single variable, so that you can use it as a list or tuple:
a = baz(10, 20, 30)
print(', '.join(map(str, a))) # this outputs 110, 120, 130
I'm trying to write a function that returns the next element of a generator and if it is at the end of the generator it resets it and returns the next result. The expected output of the code below would be:
1
2
3
1
2
However that is not what I get obviously. What am I doing that is incorrect?
a = '123'
def convert_to_generator(iterable):
return (x for x in iterable)
ag = convert_to_generator(a)
def get_next_item(gen, original):
try:
return next(gen)
except StopIteration:
gen = convert_to_generator(original)
get_next_item(gen, original)
for n in range(5):
print(get_next_item(ag,a))
1
2
3
None
None
Is itertools.cycle(iterable) a possible alternative?
You need to return the result of your recursive call:
return get_next_item(gen, original)
which still does not make this a working approach.
The generator ag used in your for-loop is not changed by the rebinding of the local variable gen in your function. It will stay exhausted...
As has been mentioned in the comments, check out itertools.cycle.
the easy way is just use itertools.cycle, otherwise you would need to remember the elements in the iterable if said iterable is an iterator (aka a generator) becase those can't be reset, if its not a iterator, you can reuse it many times.
the documentation include a example implementation
def cycle(iterable):
# cycle('ABCD') --> A B C D A B C D A B C D ...
saved = []
for element in iterable:
yield element
saved.append(element)
while saved:
for element in saved:
yield element
or for example, to do the reuse thing
def cycle(iterable):
# cycle('ABCD') --> A B C D A B C D A B C D ...
if iter(iterable) is iter(iterable): # is a iterator
saved = []
for element in iterable:
yield element
saved.append(element)
else:
saved = iterable
while saved:
for element in saved:
yield element
example use
test = cycle("123")
for i in range(5):
print(next(test))
now about your code, the problem is simple, it don't remember it state
def get_next_item(gen, original):
try:
return next(gen)
except StopIteration:
gen = convert_to_generator(original) # <-- the problem is here
get_next_item(gen, original) #and you should return something here
in the marked line a new generator is build, but you would need to update your ag variable outside this function to get the desire behavior, there are ways to do it, like changing your function to return the element and the generator, there are other ways, but they are not recommended or more complicated like building a class so it remember its state
get_next_item is a generator, that returns an iterator, that gives you the values it yields via the __next__ method. For that reason, your statement doesn't do anything.
What you want to do is this:
def get_next_item(gen, original):
try:
return next(gen)
except StopIteration:
gen = convert_to_generator(original)
for i in get_next_item(gen, original):
return i
or shorter, and completely equivalent (as long as gen has a __iter__ method, which it probably has):
def get_next_item(gen, original):
for i in gen:
yield i
for i in get_next_item(convert_to_generator(original)):
yield i
Or without recursion (which is a big problem in python, as it is 1. limited in depth and 2. slow):
def get_next_item(gen, original):
for i in gen:
yield i
while True:
for i in convert_to_generator(original):
yield i
If convert_to_generator is just a call to iter, it is even shorter:
def get_next_item(gen, original):
for i in gen:
yield i
while True:
for i in original:
yield i
or, with itertools:
import itertools
def get_next_item(gen, original):
return itertools.chain(gen, itertools.cycle(original))
and get_next_item is equivalent to itertools.cycle if gen is guaranteed to be an iterator for original.
Side note: You can exchange for i in x: yield i for yield from x (where x is some expression) with Python 3.3 or higher.
i would like to perform a calculation using python, where the current value (i) of the equation is based on the previous value of the equation (i-1), which is really easy to do in a spreadsheet but i would rather learn to code it
i have noticed that there is loads of information on finding the previous value from a list, but i don't have a list i need to create it! my equation is shown below.
h=(2*b)-h[i-1]
can anyone give me tell me a method to do this ?
i tried this sort of thing, but that will not work as when i try to do the equation i'm calling a value i haven't created yet, if i set h=0 then i get an error that i am out of index range
i = 1
for i in range(1, len(b)):
h=[]
h=(2*b)-h[i-1]
x+=1
h = [b[0]]
for val in b[1:]:
h.append(2 * val - h[-1]) # As you add to h, you keep up with its tail
for large b list (brr, one-letter identifier), to avoid creating large slice
from itertools import islice # For big list it will keep code less wasteful
for val in islice(b, 1, None):
....
As pointed out by #pad, you simply need to handle the base case of receiving the first sample.
However, your equation makes no use of i other than to retrieve the previous result. It's looking more like a running filter than something which needs to maintain a list of past values (with an array which might never stop growing).
If that is the case, and you only ever want the most recent value,then you might want to go with a generator instead.
def gen():
def eqn(b):
eqn.h = 2*b - eqn.h
return eqn.h
eqn.h = 0
return eqn
And then use thus
>>> f = gen()
>>> f(2)
4
>>> f(3)
2
>>> f(2)
0
>>>
The same effect could be acheived with a true generator using yield and send.
First of, do you need all the intermediate values? That is, do you want a list h from 0 to i? Or do you just want h[i]?
If you just need the i-th value you could us recursion:
def get_h(i):
if i>0:
return (2*b) - get_h(i-1)
else:
return h_0
But be aware that this will not work for large i, as it will exceed the maximum recursion depth. (Thanks for pointing this out kdopen) In that case a simple for-loop or a generator is better.
Even better is to use a (mathematically) closed form of the equation (for your example that is possible, it might not be in other cases):
def get_h(i):
if i%2 == 0:
return h_0
else:
return (2*b)-h_0
In both cases h_0 is the initial value that you start out with.
h = []
for i in range(len(b)):
if i>0:
h.append(2*b - h[i-1])
else:
# handle i=0 case here
You are successively applying a function (equation) to the result of a previous application of that function - the process needs a seed to start it. Your result looks like this [seed, f(seed), f(f(seed)), f(f(f(seed)), ...]. This concept is function composition. You can create a generalized function that will do this for any sequence of functions, in Python functions are first class objects and can be passed around just like any other object. If you need to preserve the intermediate results use a generator.
def composition(functions, x):
""" yields f(x), f(f(x)), f(f(f(x)) ....
for each f in functions
functions is an iterable of callables taking one argument
"""
for f in functions:
x = f(x)
yield x
Your specs require a seed and a constant,
seed = 0
b = 10
The equation/function,
def f(x, b = b):
return 2*b - x
f is applied b times.
functions = [f]*b
Usage
print list(composition(functions, seed))
If the intermediate results are not needed composition can be redefined as
def composition(functions, x):
""" Returns f(x), g(f(x)), h(g(f(x)) ....
for each function in functions
functions is an iterable of callables taking one argument
"""
for f in functions:
x = f(x)
return x
print composition(functions, seed)
Or more generally, with no limitations on call signature:
def compose(funcs):
'''Return a callable composed of successive application of functions
funcs is an iterable producing callables
for [f, g, h] returns f(g(h(*args, **kwargs)))
'''
def outer(f, g):
def inner(*args, **kwargs):
return f(g(*args, **kwargs))
return inner
return reduce(outer, funcs)
def plus2(x):
return x + 2
def times2(x):
return x * 2
def mod16(x):
return x % 16
funcs = (mod16, plus2, times2)
eq = compose(funcs) # mod16(plus2(times2(x)))
print eq(15)
While the process definition appears to be recursive, I resisted the temptation so I could stay out of maximum recursion depth hades.
I got curious, searched SO for function composition and, of course, there are numerous relavent Q&A's.
I have a complex variable like the one below...
test_list[[test1, test1_var1, test1,var2], [test2, test2_var1]]
I have written a function to extract the variables from the desired test, see below...
def find_test(test_list, search_term):
for index in range(len(test_list)):
if test_list[index][0] == search_term:
return test_list[index][1:]
This returns something like the following...
[test1_var1, test1_var2]
I would like to be able to return the variables as individual variables and not elements of a list. How would I go about doing this? How do I return variable number of variables? (sort of like *args but for return instead of arguments)
In python, returning multiple variables corresponds to returning any iterable, so there's no practical difference between returning a list or "multiple variables":
def f():
return 1,2
def g():
return [1,2]
a,b=f()
c,d=g()
The only difference between these two functions is that f returns a tuple, and g returns a list - which is indifferent, if you use multiple assignment on the return value.
actually you can do what you want, by using a list:
def find_test(test_list, search_term):
for index in range(len(test_list)):
if test_list[index][0] == search_term:
return test_list[index][1:]
the destructuring array syntax is there for you:
foo, bar = find_text(x, y)
if you want to get the result as a list you can:
l = find_text(x,y)
if you want to get only one element:
foo, _ = find_text(x,y)
_, bar = find_text(x,y)
if you like to read, here are a few resources:
https://docs.python.org/release/1.5.1p1/tut/tuples.html
http://robert-lujo.com/post/40871820711/python-destructuring
Just use the built-in filter and expand the results in your function call:
def search(haystack, needle):
return filter(lambda x: x[0] == needle, haystack)[0][1:]
a,b = search(test_list, 'test1')
Keep in mind if your result is more than two items, the above will fail.