I have a list like this
attach=['a','b','c','d','e','f','g','k']
I wanna pair each two elements that followed by each other:
lis2 = [('a', 'b'), ('c', 'd'), ('e', 'f'), ('g', 'k')]
I did the following:
Category=[]
for i in range(len(attach)):
if i+1< len(attach):
Category.append(f'{attach[i]},{attach[i+1]}')
but then I have to remove half of rows because it also give 'b' ,'c' and so on. I thought maybe there is a better way
You can use zip() to achieve this as:
my_list = ['a','b','c','d','e','f','g','k']
new_list = list(zip(my_list[::2], my_list[1::2]))
where new_list will hold:
[('a', 'b'), ('c', 'd'), ('e', 'f'), ('g', 'k')]
This will work to get only the pairs, i.e. if number of the elements in the list are odd, you'll loose the last element which is not as part of any pair.
If you want to preserve the last odd element from list as single element tuple in the final list, then you can use itertools.zip_longest() (in Python 3.x, or itertools.izip_longest() in Python 2.x) with list comprehension as:
from itertools import zip_longest # In Python 3.x
# from itertools import izip_longest ## In Python 2.x
my_list = ['a','b','c','d','e','f','g','h', 'k']
new_list = [(i, j) if j is not None else (i,) for i, j in zip_longest(my_list[::2], my_list[1::2])]
where new_list will hold:
[('a', 'b'), ('c', 'd'), ('e', 'f'), ('g', 'h'), ('k',)]
# last odd number as single element in the tuple ^
You have to increment iterator i.e by i by 2 when moving forward
Category=[]
for i in range(0, len(attach), 2):
Category.append(f'{attach[i]},{attach[i+1]}')
Also, you don't need the if condition, if the len(list) is always even
lis2 = [(lis[i],lis[i+1]) for i in range(0,len(lis),2)]
lis2
You can use list comprehension
I am currently sitting on a problem considering Markov chains were an input is given in the form of a list of strings. This input has to be transformed into a Markov chain. I have been sitting on this problem already a couple of hours.
My idea: As you can see below I have tried to use the counter from collections to count all transitions, which has worked. Now I am trying to count all the tuples where A and B are the first elements. This gives me all possible transitions for A.
Then I'll count the transitions like (A, B).
Then I want to use these to create a matrix with all probabilities.
def markov(seq):
states = Counter(seq).keys()
liste = []
print(states)
a = zip(seq[:-1], seq[1:])
print(list(a))
print(markov(["A","A","B","B","A","B","A","A","A"]))
So far I can't get the counting of the tuples to work.
Any help or new ideas on how to solve this is appreciated
To count the tuple, you can create another counter.
b = Counter()
for word_pair in a:
b[word_pair] += 1
b will keep the count of the pair.
To create the matrix, you can use numpy.
c = np.array([[b[(i,j)] for j in states] for i in states], dtype = float)
I will leave the task of normalizing each row sum to 1 as an exercise.
I didn't get exactly what you wanted but here is what I think it is:
from collections import Counter
def count_occurence(seq):
counted_states = []
transition_dict = {}
for tup in seq:
if tup not in counted_states:
transition_dict[tup] = seq.count(tup)
counted_states.append(tup)
print(transition_dict)
#{('A', 'A'): 3, ('A', 'B'): 2, ('B', 'B'): 1, ('B', 'A'): 2}
def markov(seq):
states = Counter(seq).keys()
print(states)
#dict_keys(['A', 'B'])
a = list(zip(seq[:-1], seq[1:]))
print(a)
#[('A', 'A'), ('A', 'B'), ('B', 'B'), ('B', 'A'), ('A', 'B'), ('B',
#'A'), ('A', 'A'), ('A', 'A')]
return a
seq = markov(["A","A","B","B","A","B","A","A","A"])
count_occurence(seq)
I though this would be straightforward, unfortunately, it is not.
I am trying to build a function to take an iterable of dictionaries (i.e., a list of unique dictionaries) and return a list of lists of unique groupings of the dictionaries.
If I have x players I would like to form k teams of n size.
This question and set of answers from CMSDK is the closest thing to a solution I can find. In adapting it from processing strings of letters to dictionaries I am finding my Python skills inadequate.
The original function that I am adapting comes from the second answer:
import itertools as it
def unique_group(iterable, k, n):
"""Return an iterator, comprising groups of size `k` with combinations of size `n`."""
# Build separate combinations of `n` characters
groups = ("".join(i) for i in it.combinations(iterable, n)) # 'AB', 'AC', 'AD', ...
# Build unique groups of `k` by keeping the longest sets of characters
return (i for i in it.product(groups, repeat=k)
if len(set("".join(i))) == sum((map(len, i)))) # ('AB', 'CD'), ('AB', 'CE'), ...
My current adaptation (that utterly fails with an error of TypeError: object of type 'generator' has no len() because of the call to map(len, i)):
def unique_group(iterable, k, n):
groups = []
groups.append((i for i in it.combinations(iterable, n)))
return ( i for i in it.product(groups, repeat=k) if len(set(i)) == sum((map(len, i))) )
For a bit of context: I am trying to programmatically divide a group of players into teams for Christmas Trivia based on their skills. The list of dictionaries is formed from a yaml file that looks like
- name: Patricia
skill: 4
- name: Christopher
skill: 6
- name: Nicholas
skill: 7
- name: Bianca
skill: 4
Which, after yaml.load produces a list of dictionaries:
players = [{'name':'Patricia', 'skill':4},{'name':'Christopher','skill':6},
{'name':'Nicholas','skill':7},{'name':'Bianca','skill':4}]
So I expect output that would look like a list of these (where k = 2 and n = 2) :
(
# Team assignment grouping 1
(
# Team 1
( {'name': 'Patricia', 'skill': 4}, {'name': 'Christopher', 'skill': 6} ),
# Team 2
( {'name': 'Nicholas', 'skill': 7}, {'name': 'Bianca', 'skill': 4} )
),
# Team assignment grouping 2
(
# Team 1
( {'name': 'Patricia', 'skill': 4}, {'name': 'Bianca', 'skill': 4} ),
# Team 2
( {'name': 'Nicholas', 'skill': 7}, {'name': 'Christopher', 'skill': 6} )
),
...,
# More unique lists
)
Each team assignment grouping needs to have unique players across teams (i.e., there cannot be the same player on multiple teams in a team assignment grouping), and each team assignment grouping needs to be unique.
Once I have the list of team assignment combinations I will sum up the skills in every group, take the difference between the highest skill and lowest skill, and choose the grouping (with variance) with the lowest difference between highest and lowest skills.
I will admit I do not understand this code fully. I understand the first assignment to create a list of all the combinations of the letters in a string, and the return statement to find the product under the condition that the product does not contain the same letter in different groups.
My initial attempt was to simply take the it.product(it.combinations(iterable, n), repeat=k) but this does not achieve uniqueness across groups (i.e., I get the same player on different teams in one grouping).
Thanks in advance, and Merry Christmas!
Update:
After a considerable amount of fiddling I have gotten the adaptation to this:
This does not work
def unique_group(iterable, k, n):
groups = []
groups.append((i for i in it.combinations(iterable, n)))
return (i for i in it.product(groups, repeat=k)\
if len(list({v['name']:v for v in it.chain.from_iterable(i)}.values())) ==\
len(list([x for x in it.chain.from_iterable(i)])))
I get a bug
Traceback (most recent call last):
File "./optimize.py", line 65, in <module>
for grouping in unique_group(players, team_size, number_of_teams):
File "./optimize.py", line 32, in <genexpr>
v in it.chain.from_iterable(i)})) == len(list([x for x in
File "./optimize.py", line 32, in <dictcomp>
v in it.chain.from_iterable(i)})) == len(list([x for x in
TypeError: tuple indices must be integers or slices, not str
Which is confusing the crap out of me and makes clear I don't know what my code is doing. In ipython I took this sample output:
assignment = (
({'name': 'Patricia', 'skill': 4}, {'name': 'Bianca', 'skill': 4}),
({'name': 'Patricia', 'skill': 4}, {'name': 'Bianca', 'skill': 4})
)
Which is clearly undesirable and formulated the following test:
len(list({v['name']:v for v in it.chain.from_iterable(assignment)})) == len([v for v in it.chain.from_iterable(assignment)])
Which correctly responds False. But it doesn't work in my method. That is probably because I am cargo cult coding at this point.
I understand what it.chain.from_iterable(i) does (it flattens the tuple of tuples of dictionaries to just a tuple of dictionaries). But it seems that the syntax {v['name']:v for v in ...} does not do what I think it does; either that or I'm unpacking the wrong values! I am trying to test the unique dictionaries against the total dictionaries based on Flatten list of lists and Python - List of unique dictionaries but the answer giving me
>>> L=[
... {'id':1,'name':'john', 'age':34},
... {'id':1,'name':'john', 'age':34},
... {'id':2,'name':'hanna', 'age':30},
... ]
>>> list({v['id']:v for v in L}.values())
Isn't as easy to adapt in this circumstance as I thought, and I'm realizing I don't really know what is getting returned in the it.product(groups, repeat=k). I'll have to investigate more.
This is where I'd leverage the new dataclasses with sets. You can make a dataclass hashable by setting frozen=True in the decorator. First you'd add your players to a set to get unique players. Then you'd get all the combinations of players for n size teams. Then you could create a set of unique teams. Then create valid groupings whereas no player is represented more than once across teams. Finally you could calculate the max disparity in the total team skill level across the grouping (leveraging combinations yet again) and use that to sort your valid groupings. So something like this.
from dataclasses import dataclass
from itertools import combinations
from typing import FrozenSet
import yaml
#dataclass(order=True, frozen=True)
class Player:
name: str
skill: int
#dataclass(order=True, frozen=True)
class Team:
members: FrozenSet[Player]
def total_skill(self):
return sum(p.skill for p in self.members)
def is_valid(grouping):
players = set()
for team in grouping:
for player in team.members:
if player in players:
return False
players.add(player)
return True
def max_team_disparity(grouping):
return max(
abs(t1.total_skill() - t2.total_skill())
for t1, t2 in combinations(grouping, 2)
)
def best_team_matchups(player_file, k, n):
with open(player_file) as f:
players = set(Player(p['name'], p['skill']) for p in yaml.load(f))
player_combs = combinations(players, n)
unique_teams = set(Team(frozenset(team)) for team in player_combs)
valid_groupings = set(g for g in combinations(unique_teams, k) if is_valid(g))
for g in sorted(valid_groupings, key=max_team_disparity):
print(g)
best_team_matchups('test.yaml', k=2, n=4)
Example output:
(
Team(members=frozenset({
Player(name='Chr', skill=6),
Player(name='Christopher', skill=6),
Player(name='Nicholas', skill=7),
Player(name='Patricia', skill=4)
})),
Team(members=frozenset({
Player(name='Bia', skill=4),
Player(name='Bianca', skill=4),
Player(name='Danny', skill=8),
Player(name='Nicho', skill=7)
}))
)
A list of dicts is not a good data structure for mapping what you actually want to rearrange, the player names, to their respective attributes, the skill ratings. You should transform the list of dicts to a name-to-skill mapping dict first:
player_skills = {player['name']: player['skill'] for player in players}
# player_skills becomes {'Patricia': 4, 'Christopher': 6, 'Nicholas': 7, 'Blanca': 4}
so that you can recursively deduct a combination of n players from the pool of players iterable, until the number of groups reaches k:
from itertools import combinations
def unique_group(iterable, k, n, groups=0):
if groups == k:
yield []
pool = set(iterable)
for combination in combinations(pool, n):
for rest in unique_group(pool.difference(combination), k, n, groups + 1):
yield [combination, *rest]
With your sample input, list(unique_group(player_skills, 2, 2)) returns:
[[('Blanca', 'Christopher'), ('Nicholas', 'Patricia')],
[('Blanca', 'Nicholas'), ('Christopher', 'Patricia')],
[('Blanca', 'Patricia'), ('Christopher', 'Nicholas')],
[('Christopher', 'Nicholas'), ('Blanca', 'Patricia')],
[('Christopher', 'Patricia'), ('Blanca', 'Nicholas')],
[('Nicholas', 'Patricia'), ('Blanca', 'Christopher')]]
You can get the combination with the lowest variance in total skill ratings by using the min function with a key function that returns the skill difference between the team with the highest total skill ratings and the one with the lowest, which takes only O(n) in time complexity:
def variance(groups):
total_skills = [sum(player_skills[player] for player in group) for group in groups]
return max(total_skills) - min(total_skills)
so that min(unique_group(player_skills, 2, 2), key=variance) returns:
[('Blanca', 'Nicholas'), ('Christopher', 'Patricia')]
Instead of trying to create every possible grouping of k sets of n elements (possibly including repeats!), and then filtering down to the ones that don't have any overlap, let's directly build groupings that meet the criterion. This also avoids generating redundant groupings in different orders (the original code could also do this by using combinations rather than product in the last step).
The approach is:
Iterate over possibilities (combinations of n elements in the input) for the first set - by which I mean, the one that contains the first of the elements that will be chosen.
For each, recursively find possibilities for the remaining sets. They cannot use elements from the first set, and they also cannot use elements from before the first set (or else the first set wouldn't be first).
In order to combine the results elegantly, we use a recursive generator: rather than trying to build lists that contain results from the recursive calls, we just yield everything we need to. We represent each collection of group_count many elements with a tuple of tuples (the inner tuples are the groups). At the base case, there is exactly one way to make no groups of elements - by just... doing that... yeah... - so we need to yield one value which is a tuple of no tuples of an irrelevant number of elements each - i.e., an empty tuple. In the other cases, we prepend the tuple for the current group to each result from the recursive call, yielding all those results.
from itertools import combinations
def non_overlapping_groups(group_count, group_size, population):
if group_count == 0:
yield ()
return
for indices in combinations(range(len(population)), group_size):
current = (tuple(population[i] for i in indices),)
remaining = [
x for i, x in enumerate(population)
if i not in indices and i > indices[0]
] if indices else population
for recursive in non_overlapping_groups(group_count - 1, group_size, remaining):
yield current + recursive
Let's try it:
>>> list(non_overlapping_groups(2, 3, 'abcdef'))
[(('a', 'b', 'c'), ('d', 'e', 'f')), (('a', 'b', 'd'), ('c', 'e', 'f')), (('a', 'b', 'e'), ('c', 'd', 'f')), (('a', 'b', 'f'), ('c', 'd', 'e')), (('a', 'c', 'd'), ('b', 'e', 'f')), (('a', 'c', 'e'), ('b', 'd', 'f')), (('a', 'c', 'f'), ('b', 'd', 'e')), (('a', 'd', 'e'), ('b', 'c', 'f')), (('a', 'd', 'f'), ('b', 'c', 'e')), (('a', 'e', 'f'), ('b', 'c', 'd'))]
>>> list(non_overlapping_groups(3, 2, 'abcdef'))
[(('a', 'b'), ('c', 'd'), ('e', 'f')), (('a', 'b'), ('c', 'e'), ('d', 'f')), (('a', 'b'), ('c', 'f'), ('d', 'e')), (('a', 'c'), ('b', 'd'), ('e', 'f')), (('a', 'c'), ('b', 'e'), ('d', 'f')), (('a', 'c'), ('b', 'f'), ('d', 'e')), (('a', 'd'), ('b', 'c'), ('e', 'f')), (('a', 'd'), ('b', 'e'), ('c', 'f')), (('a', 'd'), ('b', 'f'), ('c', 'e')), (('a', 'e'), ('b', 'c'), ('d', 'f')), (('a', 'e'), ('b', 'd'), ('c', 'f')), (('a', 'e'), ('b', 'f'), ('c', 'd')), (('a', 'f'), ('b', 'c'), ('d', 'e')), (('a', 'f'), ('b', 'd'), ('c', 'e')), (('a', 'f'), ('b', 'e'), ('c', 'd'))]
>>> # Some quick sanity checks
>>> len(list(non_overlapping_groups(2, 3, 'abcdef')))
10
>>> # With fewer input elements, obviously we can't do it.
>>> len(list(non_overlapping_groups(2, 3, 'abcde')))
0
>>> # Adding a 7th element, any element could be the odd one out,
>>> # and in each case we get another 10 possibilities, making 10 * 7 = 70.
>>> len(list(non_overlapping_groups(2, 3, 'abcdefg')))
70
I performance tested this against a modified version of the original (which also shows how to make it work properly with non-strings, and optimizes the sum calculation):
def unique_group(group_count, group_size, population):
groups = list(it.combinations(population, group_size))
return (
i for i in combinations(groups, group_count)
if len({e for g in i for e in g}) == group_count * group_size
)
Quickly verifying the equivalence:
>>> len(list(unique_group(3, 2, 'abcdef')))
15
>>> len(list(non_overlapping_groups(3, 2, 'abcdef')))
15
>>> set(unique_group(3, 2, 'abcdef')) == set(non_overlapping_groups(3, 2, 'abcdef'))
True
We see that even for fairly small examples (here, the output has 280 groupings), the brute-force approach has to filter through a lot:
>>> import timeit
>>> timeit.timeit("list(g(3, 3, 'abcdefghi'))", globals={'g': unique_group}, number=100)
5.895461600041017
>>> timeit.timeit("list(g(3, 3, 'abcdefghi'))", globals={'g': non_overlapping_groups}, number=100)
0.2303082060534507
There are quite a lot of questions about the unique (Cartesian) product of lists, but I am looking for something peculiar that I haven't found in any of the other questions.
My input will always consist of two lists. When the lists are identical, I want to get all combinations but when they are different I need the unique product (i.e. order does not matter). However, in addition I also need the order to be preserved, in the sense that the order of the input lists matters. In fact, what I need is that the items in the first list should always be the first item of the product tuple.
I have the following working code, which does what I want with the exception I haven't managed to find a good, efficient way to keep the items ordered as described above.
import itertools
xs = ['w']
ys = ['a', 'b', 'c']
def get_up(x_in, y_in):
if x_in == y_in:
return itertools.combinations(x_in, 2)
else:
ups = []
for x in x_in:
for y in y_in:
if x == y:
continue
# sort so that cases such as (a,b) (b,a) get filtered by set later on
ups.append(sorted((x, y)))
ups = set(tuple(up) for up in ups)
return ups
print(list(get_up(xs, ys)))
# [('c', 'w'), ('b', 'w'), ('a', 'w')]
As you can see, the result is a list of unique tuples that are ordered alphabetically. I used the sorting so I could filter duplicate entries by using a set. But because the first list (xs) contains the w, I want the tuples to have that w as a first item.
[('w', 'c'), ('w', 'b'), ('w', 'a')]
If there's an overlap between two lists, the order of the items that occur in both lists don't matter., so for xs = ['w', 'a', 'b'] and ys = ['a', 'b', 'c'] the order for a doesn't matter
[('w', 'c'), ('w', 'b'), ('w', 'a'), ('a', 'b'), ('a', 'c'), ('b', 'c')]
^
or
[('w', 'c'), ('w', 'b'), ('w', 'a'), ('a', 'c'), ('b', 'a'), ('b', 'c')]
^
Preferably I'd end up with a generator (as combinations returns). I'm also only interested in Python >= 3.6.
Collect the tuples in an order-preserving way (as when the lists are identical), then filter by removing tuples whose inverse is also in the list.
if x_in == y_in:
return itertools.combinations(x_in, 2)
else:
seen = set()
for a,b in itertools.product(x_in, y_in):
if a == b or (b, a) in seen:
continue
else:
yield (a,b)
seen.add((a,b))
This will give you the tuples in (x, y) order; when both (a,b) and (b,a) occur, you get only the order seen first.
I'll give an answer to my own question, though I bet there is a better solution using itertools or others.
xs = ['c', 'b']
ys = ['a', 'b', 'c']
def get_unique_combinations(x_in, y_in):
""" get unique combinations that maintain order, i.e. x is before y """
yielded = set()
for x in x_in:
for y in y_in:
if x == y or (x, y) in yielded or (y, x) in yielded:
continue
yield x, y
yielded.add((x, y))
return None
print(list(get_unique_combinations(xs, ys)))
What is the most efficient way of finding a certain tuple based on e.g. the second element of that tuple in a list and move that tuple to the top of the list
Something of the form:
LL=[('a','a'),('a','b'),('a','c'),('a','d')]
LL.insert(0,LL.pop(LL.index( ... )))
where I would like something in index() that would give me the position of the tuple that has 'c' as second element.
Is there a classic python 1-line approach to do that?
>>> LL.insert(0,LL.pop([x for x, y in enumerate(LL) if y[1] == 'c'][0]))
>>> LL
[('a', 'c'), ('a', 'a'), ('a', 'b'), ('a', 'd')]
>>>
To find position you can:
positions = [i for i, tup in enumerate(LL) if tup[1] == 'c']
You can now take the index of the desired element, pop it and push to the beginning of the list
pos = positions[0]
LL.insert(0, LL.pop(pos))
But you can also sort your list using the item in the tuple as key:
sorted(LL, key=lambda tup: tup[1] == 'c', reverse=True)
if you don't care about order of the other elements
2 lines, however 1 line solutions are all inefficient
>>> LL=[('a','a'),('a','b'),('a','c'),('a','d')]
>>> i = next((i for i, (x, y) in enumerate(LL) if y == 'c'), 0) # 0 default index
>>> LL[0], LL[i] = LL[i], LL[0]
>>> LL
[('a', 'c'), ('a', 'b'), ('a', 'a'), ('a', 'd')]
This does nothing if the index is not found
>>> LL=[('a','a'),('a','b'),('a','c'),('a','d')]
>>> i = next((i for i, (x, y) in enumerate(LL) if y == 'e'), 0) # 0 default index
>>> LL[0], LL[i] = LL[i], LL[0]
>>> LL
[('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')]
The problem with terse, pythonic, 'fancy schmancy' solutions is the code might not be easily maintained and/or reused in other closely aligned contexts.
It seems best to just use 'boiler plate' code to do the search, and then continue with the application specific requirements.
So here is an example of easy to understand search code that can be easily 'plugged into' when these questions come up, including those situations when we need to know if the key is found.
def searchTupleList(list_of_tuples, coord_value, coord_index):
for i in range(0, len(list_of_tuples)):
if list_of_tuples[i][coord_index] == coord_value:
return i # matching index in list
return -1 # not found