I have a function that must return several values using a for loop. I do not wish to store the values inside a list or a dict. Because of the use of the return, I only get the first value. How can I return all values successively? I tried using generators and yield but I'm not sure how to use it.
here is the function:
import random
def my_function():
for i in range(3):
return(dict(x=[[random.randint(0,10)]], y=[[random.randint(0,10)]]), 0)
Are generators and the use of yield suited for my need?
Replace return by yield to create a generator:
import random
def my_function():
for i in range(3):
yield dict(x=[[random.randint(0,10)]], y=[[random.randint(0,10)]]), 0
g = my_function()
for d in g:
print(d)
Output:
({'x': [[0]], 'y': [[10]]}, 0)
({'x': [[0]], 'y': [[1]]}, 0)
({'x': [[3]], 'y': [[0]]}, 0)
You can also use next to consume manually the next value:
g = my_function()
print(next(g))
print(next(g))
print(next(g))
print(next(g)) # Will raise a StopIteration exception
Output:
({'x': [[4]], 'y': [[4]]}, 0)
({'x': [[4]], 'y': [[9]]}, 0)
({'x': [[7]], 'y': [[2]]}, 0)
...
StopIteration:
I hope, this gives you better understanding. next gives you value one by one and if you want all values wrap your function inside a list
import random
def my_function():
for i in range(3):
yield(dict(x=[[random.randint(0,10)]], y=[[random.randint(0,10)]]), 0)
a = my_function()
print(next(a)) # one by one
print(next(a))
print(list(my_function())) # get all values
Related
I'm trying to work out how to write an algorithm to calculate the weights across different lists the most efficient way. I have a dict which contains various ids:
x["Y"]=[id1,id2,id3...]
x["X"]=[id2,id3....]
x["Z"]=[id3]
.
.
I have an associated weight for each of the elements:
w["Y"]=10
w["X"]=10
w["Z"]=5
Given an input, e.g. "Y","Z", I want to get an output of to give me:
(id1,10),(id2,10),(id3,15)
id3 gets 15 because it's in both x["Y"] and x["Z"].
Is there a way way I can do this with vector matrixes?
You can use the itertools library to group together common terms in a list:
import itertools
import operator
a = {'x': [2,3], 'y': [1,2,3], 'z': [3]}
b = {'x': 10, 'y': 10, 'z': 5}
def matrix_weight(letter1,letter2):
final_list = []
for i in a[letter1]:
final_list.append((i, b[letter1]))
for i in a[letter2]:
final_list.append((i, b[letter2]))
# final_list = [(1,10), (2,10), (3,10), (3,5)]
it = itertools.groupby(final_list, operator.itemgetter(0))
for key, subiter in it:
yield key, sum(item[1] for item in subiter)
print list(matrix_weight('y', 'z'))
I'll use the id in strings as in your example, but integer id works similarly.
def id_weights(x, w, keys):
result = {}
for key in keys:
for id in x[key]:
if id not in result:
result[id] = 0
result[id] += w[key]
return [(id, result[id]) for id in sorted(result.keys())]
x = {"Y": ["id1","id2","id3"],
"X": ["id2", "id3"],
"Z": ["id3"]}
w = {"Y": 10, "X": 10, "Z": 5}
if __name__ == "__main__":
keys = ["Y", "Z"]
print id_weights(x, w, keys)
gives
[('id1', 10), ('id2', 10), ('id3', 15)]
Basically I'm looking for an implementation of itertools.product that allows me to change the order in which the combinations are generated.
Example: If I use itertools.product('AB', 'xy') it generates the combinations in this exact order:
[('A', 'x'), ('A', 'y'), ('B', 'x'), ('B', 'y')]
I need an implementation that responds to requests like "Please change A to B next", for example like this:
>>> generator = DynamicOrderProduct({'var1': 'AB', 'var2': 'xy'})
>>> generator.first()
{'var1': 'A', 'var2': 'x'}
>>> generator.change('var1')
{'var1': 'B', 'var2': 'x'}
>>> generator.change('var2')
{'var1': 'B', 'var2':, 'y'}
>>> generator.change('var2') # here it can't generate a new combination by
# changing var2, so it changes var1 instead
{'var1': 'A', 'var2': 'y'}
>>> generator.change('var2')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
Ideally, the generator would accept a list of variables like this:
generator.change(['var1', 'var2'])
It should then attempt to change the value of var1, and if that isn't possible, change the value of var2 instead, and so on.
How would I go about implementing this? Is there something in the standard lib that can help me?
Alright, I've managed to write an iterator that does what I want. It's the ugliest piece of code I've ever written, but it gets the job done.
I'm still hoping for a better solution though - this implementation keeps a set of all returned combinations, which can grow to use quite a bit of memory.
class DynamicOrderProduct:
"""
Given a dict of {variable: [value1,value2,value3,...]}, allows iterating
over the cartesian product of all variable values.
Each step in the iteration returns a mapping of {variable: value}.
To start the iteration, retrieve the first mapping by calling .first().
To retrieve subsequent mappings, call
.next(order_in_which_to_change_variable_values). This function's
parameter should be a list of variables sorted by which variable's value
should change next. If possible, the first variable in the list will
change value. If not, the 2nd variable in the list will change value
instead, and so on. Raises StopIteration if all combinations are
exhausted.
Example:
possible_values = {'B': [0,1], # B can take the value 0 or the value 1
'L': [1,2,3]}
iterator = DynamicOrderProduct(possible_values)
print(iterator.first())
import random
variables = list(possible_values.keys())
while True:
order = random.sample(variables, len(variables))
print('advancing variables in this order:', order)
try:
print(iterator.next(order))
except StopIteration:
break
You may also pass an incomplete list of variables to the .next function.
If no new combination of the given variables is possible, StopIteration is
raised. For example:
iterator = DynamicOrderProduct({var1: [1],
var2: [1,2]})
iterator.first() # this returns {var1: 1, var2: 1}
iterator.next([var1]) # raises StopIteration
Also, you may pass multiple lists to .next in order to change the value of
multiple variables. StopIteration will be raised only if no variable can
change value.
iterator = DynamicOrderProduct({var1: [1,2],
var2: [1,2]})
iterator.first() # this returns {var1: 1, var2: 1}
iterator.next([var1], [var2]) # returns {var1: 2, var2: 2}
"""
def __init__(self, possible_variable_values):
self.possible_variable_values = {k:tuple(v) for k,v in \
possible_variable_values.items()}
self.variable_order = list(possible_variable_values)
self.exhausted_combinations = set()
def first(self):
self.mapping = {var:vals[0] for var,vals in \
self.possible_variable_values.items()}
t = tuple(self.mapping[var] for var in self.variable_order)
self.exhausted_combinations.add(t)
return self.mapping
def next(self, *orders):
def advance(order, index, maxindex=2147483648):
while True: # loop to reduce recursion
try:
variable = order[index]
except IndexError:
raise StopIteration
value = self.mapping[variable]
valindex = self.possible_variable_values[variable].index(value)
start_index = valindex
while True: # change the value until we find a new combination
valindex += 1
try:
possible_values = self.possible_variable_values
value = possible_values[variable][valindex]
except IndexError:
valindex = 0
value = self.possible_variable_values[variable][0]
self.mapping[variable] = value
# if we've tried all values but none of them
# worked, try to change the next variable's
# value instead
if valindex == start_index:
if index+1 >= maxindex:
raise StopIteration
# instead of recursing, update our own parameters and
# start a new iteration
index += 1
break
t = tuple(self.mapping[var] for var in self.variable_order)
# if this combination isn't new, try
# changing the previous variables' values
if t in self.exhausted_combinations:
if index == 0:
continue
try:
return advance(order, 0, index)
except StopIteration:
continue
return t
total_order = []
fail = True
for order in orders:
# each iteration may also change the previous
# iterations' variables
total_order = order + total_order
try:
t = advance(total_order, 0)
except StopIteration:
fail = True
else:
fail = False
if fail:
raise StopIteration
self.exhausted_combinations.add(t)
return self.mapping
I have this kind of file (part):
H DX=615 DY=425 DZ=22.15 -AB C=0 T=0 R=999 *MM /"def" BX=2.5 BY=452.5 BZ=25 ;M20150710.
XBO X=100 Y=50 Z=5 V=1000 R=0 x=0 y=0 D=10 N="P" F=1 ;Test F1/10P.
...
which I want to convert to a new programming system. What I want to do is first read the header (H) and put the DX, DY and DZ values in respectively named variables. I managed to do this, but when I came to process my XBO line (a drilling, from which I need X, Y, Z, V, R, x, y, D, N, F and ;, also in separate variables) my code started looking very ugly very fast.
So I started over, and came up with this:
f = open("input.xxl") # open input file
for line in f:
if Debug==1: print line
for char in line:
charbuffr=charbuffr+char
if "H" in charbuffr:
if Debug==1: print'HEADER found!'
charbuffr=""
if "XBO" in charbuffr:
if Debug==1: print'XBO found!'
charbuffr=""
This correctly identifies the separate commands H and XBO, but I'm kind of stuck now. I can use the same method to extract all the variables, from loops inside the H and XBO loops, but this does not seem like good coding...
Can anyone set me on the right foot please? I don't want a full solution, as I love coding (well my main job is coding for CNC machines, which seems easy now compared to Python), but would love to know which approach is best...
Instead of converting data types by hand, you could use ast. literal_eval. This helper function takes a list of the form ['a=2', 'b="abc"'] and converts into a dictionary {'a': 2, 'b': 'abc'}:
import ast
def dict_from_row(row):
"""Convert a list of strings in the form 'name=value' into a dict."""
res = []
for entry in row:
name, value = entry.split('=')
res.append('"{name}": {value}'.format(name=name, value=value))
dict_string = '{{{}}}'.format(', '.join(res))
return ast.literal_eval(dict_string)
Now parsing the file becomes a bit simpler:
for line in f:
row = line.split()
if not row:
continue
if row[0] == 'H':
header = dict_from_row(row[1:4])
elif line[0] == 'XBO':
xbo = dict_from_row(row[1:11])
Results:
>>> header
{'DX': 615, 'DY': 425, 'DZ': 22.15}
>>> xbo
{'D': 10, 'F': 1, 'R': 0, 'V': 1000, 'X': 100, 'Y': 50, 'Z': 5, 'x': 0, 'y': 0}
As an inspiration, you can do something like this:
for raw_line in f:
line = raw_line.split()
if not line:
continue
if line[0] == 'H':
header = {}
for entry in line[1:4]:
name, value = entry.split('=')
header[name] = float(value)
elif line[0] == 'XBO':
xbo = {}
for entry in line[1:11]:
name, value = entry.split('=')
try:
xbo[name] = int(value)
except ValueError:
xbo[name] = value[1:-1] # stripping of the ""
Now headercontains the extensions of your domain:
{'DX': 615.0, 'DY': 425.0, 'DZ': 22.15}
and xbo the other values:
{'D': 10,
'F': 1,
'N': 'P',
'R': 0,
'V': 1000,
'X': 100,
'Y': 50,
'Z': 5,
'x': 0,
'y': 0}
Access the individual values in the dictionaries:
>>> header['DX']
615.0
Is there a way, lib, or something in python that I can set value in list at an index that does not exist?
Something like runtime index creation at list:
l = []
l[3] = 'foo'
# [None, None, None, 'foo']
And more further, with multi dimensional lists:
l = []
l[0][2] = 'bar'
# [[None, None, 'bar']]
Or with an existing one:
l = [['xx']]
l[0][1] = 'yy'
# [['xx', 'yy']]
There isn't a built-in, but it's easy enough to implement:
class FillList(list):
def __setitem__(self, index, value):
try:
super().__setitem__(index, value)
except IndexError:
for _ in range(index-len(self)+1):
self.append(None)
super().__setitem__(index, value)
Or, if you need to change existing vanilla lists:
def set_list(l, i, v):
try:
l[i] = v
except IndexError:
for _ in range(i-len(l)+1):
l.append(None)
l[i] = v
Not foolproof, but it seems like the easiest way to do this is to initialize a list much larger than you will need, i.e.
l = [None for i in some_large_number]
l[3] = 'foo'
# [None, None, None, 'foo', None, None None ... ]
If you really want the syntax in your question, defaultdict is probably the best way to get it:
from collections import defaultdict
def rec_dd():
return defaultdict(rec_dd)
l = rec_dd()
l[3] = 'foo'
print l
{3: 'foo'}
l = rec_dd()
l[0][2] = 'xx'
l[1][0] = 'yy'
print l
<long output because of defaultdict, but essentially)
{0: {2: 'xx'}, 1: {0: 'yy'}}
It isn't exactly a 'list of lists' but it works more or less like one.
You really need to specify the use case though... the above has some advantages (you can access indices without checking whether they exist first), and some disadvantages - for example, l[2] in a normal dict will return a KeyError, but in defaultdict it just creates a blank defaultdict, adds it, and then returns it.
Other possible implementations to support different syntactic sugars could involve custom classes etc, and will have other tradeoffs.
You cannot create a list with gaps. You could use a dict or this quick little guy:
def set_list(i,v):
l = []
x = 0
while x < i:
l.append(None)
x += 1
l.append(v)
return l
print set_list(3, 'foo')
>>> [None, None, None, 'foo']
I have an array composed of varied sub array like:
[{'x':'xvalue', 'y':'yvalue', 'group':'groupname'}...{'x':'xnvalue', 'y':'ynvalue', 'group':groupnname'}]
I want to create a new array or serialize the same array in the form of:
[{'groupa':['x':'xvalue', 'y':'yvalue'}}...{'groupn':{'x':'xnvalue', 'y':'ynvalue'}]
Apologies for putting the question in a very weird way, but did'nt had any better explanation of the problem.
My preferred scripting language here is python.
Sample data:
{"id":"jMGTsJXWiI","key":"s1","value":{'group' : "x", 't':'45', 'xs':'x5e8'}}
{"id":"545sJXWiI","key":"s3","value":{'group' : "x", 't':'415', 'xs':'xr58'}}
{"id":"xjMdT45","key":"s2","value":{'group' : "y", 't':'405', 'xs':'xs58'}}
Assuming your data is really a list of dictionaries, this would work:
>>> groups
[{'y': 'yvalue', 'x': 'xvalue', 'group': 'groupname'}, {'y': 'ynvalue', 'x': 'xnvalue', 'group': 'groupnname'}]
>>> final_groups = {grp.pop('group'):grp for grp in groups}
>>> final_groups
{'groupname': {'y': 'yvalue', 'x': 'xvalue'}, 'groupnname': {'y': 'ynvalue', 'x': 'xnvalue'}}
This assumes 2.7+ because of dictionary comprehension. If 2.6-, then
>>> final_groups = dict((grp.pop('group'),grp) for grp in groups)
EDIT
To answer the question in your comment.
No, there is no import group. Here is the complete script:
>>> groups = [{'x':'xvalue', 'y':'yvalue', 'group':'groupname'},{'x':'xnvalue', 'y':'ynvalue', 'group':'groupnname'}]
>>> final_groups = dict((grp.pop('group'),grp) for grp in groups)
>>> final_groups
{'groupname': {'y': 'yvalue', 'x': 'xvalue'}, 'groupnname': {'y': 'ynvalue', 'x': 'xnvalue'}}
The {...} is 2.7+ specific. It is called a dictioary comprehension and if your python version is less that 2.7 then you can't do it like this, and instead can do it like I have listed above.
EDIT 2
How about something like:
final_groups = dict(
[
('%s.%s' % (item['value'].pop('group'), item['key']), item['value']) for item in groups
]
)
OUTPUT
{'y.s2': {'xs': 'xs58', 't': '405'}, 'x.s3': {'xs': 'xr58', 't': '415'}, 'x.s1': {'xs': 'x5e8', 't': '45'}}
One-liner:
>>> lis=[{'x':'xvalue', 'y':'yvalue', 'group':'groupname'},{'x':'xnvalue', 'y':'ynvalue', 'group':'groupnname'}]
>>> [{x['group']:{y:x[y] for y in x if y !='group'} for x in lis}]
[{'groupname': {'y': 'yvalue', 'x': 'xvalue'}, 'groupnname': {'y': 'ynvalue', 'x': 'xnvalue'}}]
using for loop:
lis=[{'x':'xvalue', 'y':'yvalue', 'group':'groupname'},{'x':'xnvalue', 'y':'ynvalue', 'group':'groupnname'}]
lis1=[{} for _ in range(len(lis))] # lis1= [{},{}]
for i,x in enumerate(lis):
lis1[i][x['group']]={} #creates lis1[{'groupname':{}}]
for y in x:
if y!='group':
lis1[i][x['group']][y]=x[y] #add values to lis1[{'groupname':{}}]
print(lis1)