Related
i'm know in python i'm take some basic thing about list and tuple but my not full understand the my cod i want create list have three index in each index have tuple with tow index like this [('~','a'),('a','b'),('b','c')] the first index in tuple have tow char or the length context when have like this [('~a','a'),('ab','b'),('bc',' c')] can any one help my ? Her my code
def getNGrams(wordlist, n):
ngrams = []
padded_tokens = "~"*(n) + wordlist
t = tuple(wordlist)
for i in range(3):
t = tuple(padded_tokens[i:i+n])
ngrams.append(t)
return ngrams
IIUC, You can change the function like below and get what you want:
def getNGrams(wordlist, n):
ngrams = []
padded_tokens = "~"*n + wordlist
for idx, i in enumerate(range(len(wordlist))):
t = tuple((padded_tokens[i:i+n], wordlist[idx]))
ngrams.append(t)
return ngrams
print(getNGrams('abc',1))
print(getNGrams('abc',2))
print(getNGrams('abc',3))
Output:
[('~', 'a'), ('a', 'b'), ('b', 'c')]
[('~~', 'a'), ('~a', 'b'), ('ab', 'c')]
[('~~~', 'a'), ('~~a', 'b'), ('~ab', 'c')]
I have a list of pairs (tuples), for simplification something like this:
L = [("A","B"), ("B","C"), ("C","D"), ("E","F"), ("G","H"), ("H","I"), ("G","I"), ("G","J")]
Using python I want efficiently split this list to:
L1 = [("A","B"), ("B","C"), ("C","D")]
L2 = [("E","F")]
L3 = [("G","H"), ("G","I"), ("G","J"), ("H","I")]
How to efficiently split list into groups of pairs, where for pairs in the group there must be always at least one pair which shares one item with others? As stated in one of the answers this is actually network problem. The goal is to efficiently split network into disconnected (isolated) network parts.
Type lists, tuples (sets) may be changed for achieving higher efficiency.
This is more like a network problem, so we can use networkx:
import networkx as nx
G=nx.from_edgelist(L)
l=list(nx.connected_components(G))
# after that we create the map dict , for get the unique id for each nodes
mapdict={z:x for x, y in enumerate(l) for z in y }
# then append the id back to original data for groupby
newlist=[ x+(mapdict[x[0]],)for x in L]
import itertools
#using groupby make the same id into one sublist
newlist=sorted(newlist,key=lambda x : x[2])
yourlist=[list(y) for x , y in itertools.groupby(newlist,key=lambda x : x[2])]
yourlist
[[('A', 'B', 0), ('B', 'C', 0), ('C', 'D', 0)], [('E', 'F', 1)], [('G', 'H', 2), ('H', 'I', 2), ('G', 'I', 2), ('G', 'J', 2)]]
Then to match your output format:
L1,L2,L3=[[y[:2]for y in x] for x in yourlist]
L1
[('A', 'B'), ('B', 'C'), ('C', 'D')]
L2
[('E', 'F')]
L3
[('G', 'H'), ('H', 'I'), ('G', 'I'), ('G', 'J')]
Initialise a list of groups as empty
Let (a, b) be the next pair
Collect all groups that contain any elements with a or b
Remove them all, join them, add (a, b), and insert as a new group
Repeat till done
That'd be something like this:
import itertools, functools
def partition(pred, iterable):
t1, t2 = itertools.tee(iterable)
return itertools.filterfalse(pred, t1), filter(pred, t2)
groups = []
for a, b in L:
unrelated, related = partition(lambda group: any(aa == a or bb == b or aa == b or bb == a for aa, bb in group), groups)
groups = [*unrelated, sum(related, [(a, b)])]
An efficient and Pythonic approach is to convert the list of tuples to a set of frozensets as a pool of candidates, and in a while loop, create a set as group and use a nested while loop to keep expanding the group by adding the first candidate set and then performing set union with other candidate sets that intersects with the group until there is no more intersecting candidate, at which point go back to the outer loop to form a new group:
pool = set(map(frozenset, L))
groups = []
while pool:
group = set()
groups.append([])
while True:
for candidate in pool:
if not group or group & candidate:
group |= candidate
groups[-1].append(tuple(candidate))
pool.remove(candidate)
break
else:
break
Given your sample input, groups will become:
[[('A', 'B'), ('C', 'B'), ('C', 'D')],
[('G', 'H'), ('H', 'I'), ('G', 'J'), ('G', 'I')],
[('E', 'F')]]
Keep in mind that sets are unordered in Python, which is why the order of the above output doesn't match your expected output, but for your purpose the order should not matter.
You can use the following code:
l = [("A","B"), ("B","C"), ("C","D"), ("E","F"), ("G","H"), ("H","I"), ("G","I"), ("G","J")]
result = []
if len(l) > 1:
tmp = [l[0]]
for i in range(1,len(l)):
if l[i][0] == l[i-1][1] or l[i][1] == l[i-1][0] or l[i][1] == l[i-1][1] or l[i][0] == l[i-1][0]:
tmp.append(l[i])
else:
result.append(tmp)
tmp = [l[i]]
result.append(tmp)
else:
result = l
for elem in result:
print(elem)
output:
[('A', 'B'), ('B', 'C'), ('C', 'D')]
[('E', 'F')]
[('G', 'H'), ('H', 'I'), ('G', 'I'), ('G', 'J')]
Note: this code is based on the hypothesis that your initial array is sorted. If this is not the case it will not work as it does only one pass on the whole list to create the groups (complexity O(n)).
Explanations:
result will store your groups
if len(l) > 1: if you have only one element in your list or an empty list no need to do any processing you have the answer
You will to a one pass on each element of the list and compare the 4 possible equality between the tuple at position i and the one at position i-1.
tmp is used to construct your groups, as long as the condition is met you add tuples to tmp
when the condition is not respected you add tmp (the current group that has been created to the result, reinitiate tmp with the current tuple) and you continue.
You can use a while loop and start iteration from first member of L(using a for loop inside). Check for the whole list if any member(either of the two) is shared or not. Then append it to a list L1 and pop that member from original list L. Then while loop would run again (till list L is nonempty). And for loop inside would run for each element in list to append to a new list L2. You can try this. (I will provide code it doesn't help)
I though this would be straightforward, unfortunately, it is not.
I am trying to build a function to take an iterable of dictionaries (i.e., a list of unique dictionaries) and return a list of lists of unique groupings of the dictionaries.
If I have x players I would like to form k teams of n size.
This question and set of answers from CMSDK is the closest thing to a solution I can find. In adapting it from processing strings of letters to dictionaries I am finding my Python skills inadequate.
The original function that I am adapting comes from the second answer:
import itertools as it
def unique_group(iterable, k, n):
"""Return an iterator, comprising groups of size `k` with combinations of size `n`."""
# Build separate combinations of `n` characters
groups = ("".join(i) for i in it.combinations(iterable, n)) # 'AB', 'AC', 'AD', ...
# Build unique groups of `k` by keeping the longest sets of characters
return (i for i in it.product(groups, repeat=k)
if len(set("".join(i))) == sum((map(len, i)))) # ('AB', 'CD'), ('AB', 'CE'), ...
My current adaptation (that utterly fails with an error of TypeError: object of type 'generator' has no len() because of the call to map(len, i)):
def unique_group(iterable, k, n):
groups = []
groups.append((i for i in it.combinations(iterable, n)))
return ( i for i in it.product(groups, repeat=k) if len(set(i)) == sum((map(len, i))) )
For a bit of context: I am trying to programmatically divide a group of players into teams for Christmas Trivia based on their skills. The list of dictionaries is formed from a yaml file that looks like
- name: Patricia
skill: 4
- name: Christopher
skill: 6
- name: Nicholas
skill: 7
- name: Bianca
skill: 4
Which, after yaml.load produces a list of dictionaries:
players = [{'name':'Patricia', 'skill':4},{'name':'Christopher','skill':6},
{'name':'Nicholas','skill':7},{'name':'Bianca','skill':4}]
So I expect output that would look like a list of these (where k = 2 and n = 2) :
(
# Team assignment grouping 1
(
# Team 1
( {'name': 'Patricia', 'skill': 4}, {'name': 'Christopher', 'skill': 6} ),
# Team 2
( {'name': 'Nicholas', 'skill': 7}, {'name': 'Bianca', 'skill': 4} )
),
# Team assignment grouping 2
(
# Team 1
( {'name': 'Patricia', 'skill': 4}, {'name': 'Bianca', 'skill': 4} ),
# Team 2
( {'name': 'Nicholas', 'skill': 7}, {'name': 'Christopher', 'skill': 6} )
),
...,
# More unique lists
)
Each team assignment grouping needs to have unique players across teams (i.e., there cannot be the same player on multiple teams in a team assignment grouping), and each team assignment grouping needs to be unique.
Once I have the list of team assignment combinations I will sum up the skills in every group, take the difference between the highest skill and lowest skill, and choose the grouping (with variance) with the lowest difference between highest and lowest skills.
I will admit I do not understand this code fully. I understand the first assignment to create a list of all the combinations of the letters in a string, and the return statement to find the product under the condition that the product does not contain the same letter in different groups.
My initial attempt was to simply take the it.product(it.combinations(iterable, n), repeat=k) but this does not achieve uniqueness across groups (i.e., I get the same player on different teams in one grouping).
Thanks in advance, and Merry Christmas!
Update:
After a considerable amount of fiddling I have gotten the adaptation to this:
This does not work
def unique_group(iterable, k, n):
groups = []
groups.append((i for i in it.combinations(iterable, n)))
return (i for i in it.product(groups, repeat=k)\
if len(list({v['name']:v for v in it.chain.from_iterable(i)}.values())) ==\
len(list([x for x in it.chain.from_iterable(i)])))
I get a bug
Traceback (most recent call last):
File "./optimize.py", line 65, in <module>
for grouping in unique_group(players, team_size, number_of_teams):
File "./optimize.py", line 32, in <genexpr>
v in it.chain.from_iterable(i)})) == len(list([x for x in
File "./optimize.py", line 32, in <dictcomp>
v in it.chain.from_iterable(i)})) == len(list([x for x in
TypeError: tuple indices must be integers or slices, not str
Which is confusing the crap out of me and makes clear I don't know what my code is doing. In ipython I took this sample output:
assignment = (
({'name': 'Patricia', 'skill': 4}, {'name': 'Bianca', 'skill': 4}),
({'name': 'Patricia', 'skill': 4}, {'name': 'Bianca', 'skill': 4})
)
Which is clearly undesirable and formulated the following test:
len(list({v['name']:v for v in it.chain.from_iterable(assignment)})) == len([v for v in it.chain.from_iterable(assignment)])
Which correctly responds False. But it doesn't work in my method. That is probably because I am cargo cult coding at this point.
I understand what it.chain.from_iterable(i) does (it flattens the tuple of tuples of dictionaries to just a tuple of dictionaries). But it seems that the syntax {v['name']:v for v in ...} does not do what I think it does; either that or I'm unpacking the wrong values! I am trying to test the unique dictionaries against the total dictionaries based on Flatten list of lists and Python - List of unique dictionaries but the answer giving me
>>> L=[
... {'id':1,'name':'john', 'age':34},
... {'id':1,'name':'john', 'age':34},
... {'id':2,'name':'hanna', 'age':30},
... ]
>>> list({v['id']:v for v in L}.values())
Isn't as easy to adapt in this circumstance as I thought, and I'm realizing I don't really know what is getting returned in the it.product(groups, repeat=k). I'll have to investigate more.
This is where I'd leverage the new dataclasses with sets. You can make a dataclass hashable by setting frozen=True in the decorator. First you'd add your players to a set to get unique players. Then you'd get all the combinations of players for n size teams. Then you could create a set of unique teams. Then create valid groupings whereas no player is represented more than once across teams. Finally you could calculate the max disparity in the total team skill level across the grouping (leveraging combinations yet again) and use that to sort your valid groupings. So something like this.
from dataclasses import dataclass
from itertools import combinations
from typing import FrozenSet
import yaml
#dataclass(order=True, frozen=True)
class Player:
name: str
skill: int
#dataclass(order=True, frozen=True)
class Team:
members: FrozenSet[Player]
def total_skill(self):
return sum(p.skill for p in self.members)
def is_valid(grouping):
players = set()
for team in grouping:
for player in team.members:
if player in players:
return False
players.add(player)
return True
def max_team_disparity(grouping):
return max(
abs(t1.total_skill() - t2.total_skill())
for t1, t2 in combinations(grouping, 2)
)
def best_team_matchups(player_file, k, n):
with open(player_file) as f:
players = set(Player(p['name'], p['skill']) for p in yaml.load(f))
player_combs = combinations(players, n)
unique_teams = set(Team(frozenset(team)) for team in player_combs)
valid_groupings = set(g for g in combinations(unique_teams, k) if is_valid(g))
for g in sorted(valid_groupings, key=max_team_disparity):
print(g)
best_team_matchups('test.yaml', k=2, n=4)
Example output:
(
Team(members=frozenset({
Player(name='Chr', skill=6),
Player(name='Christopher', skill=6),
Player(name='Nicholas', skill=7),
Player(name='Patricia', skill=4)
})),
Team(members=frozenset({
Player(name='Bia', skill=4),
Player(name='Bianca', skill=4),
Player(name='Danny', skill=8),
Player(name='Nicho', skill=7)
}))
)
A list of dicts is not a good data structure for mapping what you actually want to rearrange, the player names, to their respective attributes, the skill ratings. You should transform the list of dicts to a name-to-skill mapping dict first:
player_skills = {player['name']: player['skill'] for player in players}
# player_skills becomes {'Patricia': 4, 'Christopher': 6, 'Nicholas': 7, 'Blanca': 4}
so that you can recursively deduct a combination of n players from the pool of players iterable, until the number of groups reaches k:
from itertools import combinations
def unique_group(iterable, k, n, groups=0):
if groups == k:
yield []
pool = set(iterable)
for combination in combinations(pool, n):
for rest in unique_group(pool.difference(combination), k, n, groups + 1):
yield [combination, *rest]
With your sample input, list(unique_group(player_skills, 2, 2)) returns:
[[('Blanca', 'Christopher'), ('Nicholas', 'Patricia')],
[('Blanca', 'Nicholas'), ('Christopher', 'Patricia')],
[('Blanca', 'Patricia'), ('Christopher', 'Nicholas')],
[('Christopher', 'Nicholas'), ('Blanca', 'Patricia')],
[('Christopher', 'Patricia'), ('Blanca', 'Nicholas')],
[('Nicholas', 'Patricia'), ('Blanca', 'Christopher')]]
You can get the combination with the lowest variance in total skill ratings by using the min function with a key function that returns the skill difference between the team with the highest total skill ratings and the one with the lowest, which takes only O(n) in time complexity:
def variance(groups):
total_skills = [sum(player_skills[player] for player in group) for group in groups]
return max(total_skills) - min(total_skills)
so that min(unique_group(player_skills, 2, 2), key=variance) returns:
[('Blanca', 'Nicholas'), ('Christopher', 'Patricia')]
Instead of trying to create every possible grouping of k sets of n elements (possibly including repeats!), and then filtering down to the ones that don't have any overlap, let's directly build groupings that meet the criterion. This also avoids generating redundant groupings in different orders (the original code could also do this by using combinations rather than product in the last step).
The approach is:
Iterate over possibilities (combinations of n elements in the input) for the first set - by which I mean, the one that contains the first of the elements that will be chosen.
For each, recursively find possibilities for the remaining sets. They cannot use elements from the first set, and they also cannot use elements from before the first set (or else the first set wouldn't be first).
In order to combine the results elegantly, we use a recursive generator: rather than trying to build lists that contain results from the recursive calls, we just yield everything we need to. We represent each collection of group_count many elements with a tuple of tuples (the inner tuples are the groups). At the base case, there is exactly one way to make no groups of elements - by just... doing that... yeah... - so we need to yield one value which is a tuple of no tuples of an irrelevant number of elements each - i.e., an empty tuple. In the other cases, we prepend the tuple for the current group to each result from the recursive call, yielding all those results.
from itertools import combinations
def non_overlapping_groups(group_count, group_size, population):
if group_count == 0:
yield ()
return
for indices in combinations(range(len(population)), group_size):
current = (tuple(population[i] for i in indices),)
remaining = [
x for i, x in enumerate(population)
if i not in indices and i > indices[0]
] if indices else population
for recursive in non_overlapping_groups(group_count - 1, group_size, remaining):
yield current + recursive
Let's try it:
>>> list(non_overlapping_groups(2, 3, 'abcdef'))
[(('a', 'b', 'c'), ('d', 'e', 'f')), (('a', 'b', 'd'), ('c', 'e', 'f')), (('a', 'b', 'e'), ('c', 'd', 'f')), (('a', 'b', 'f'), ('c', 'd', 'e')), (('a', 'c', 'd'), ('b', 'e', 'f')), (('a', 'c', 'e'), ('b', 'd', 'f')), (('a', 'c', 'f'), ('b', 'd', 'e')), (('a', 'd', 'e'), ('b', 'c', 'f')), (('a', 'd', 'f'), ('b', 'c', 'e')), (('a', 'e', 'f'), ('b', 'c', 'd'))]
>>> list(non_overlapping_groups(3, 2, 'abcdef'))
[(('a', 'b'), ('c', 'd'), ('e', 'f')), (('a', 'b'), ('c', 'e'), ('d', 'f')), (('a', 'b'), ('c', 'f'), ('d', 'e')), (('a', 'c'), ('b', 'd'), ('e', 'f')), (('a', 'c'), ('b', 'e'), ('d', 'f')), (('a', 'c'), ('b', 'f'), ('d', 'e')), (('a', 'd'), ('b', 'c'), ('e', 'f')), (('a', 'd'), ('b', 'e'), ('c', 'f')), (('a', 'd'), ('b', 'f'), ('c', 'e')), (('a', 'e'), ('b', 'c'), ('d', 'f')), (('a', 'e'), ('b', 'd'), ('c', 'f')), (('a', 'e'), ('b', 'f'), ('c', 'd')), (('a', 'f'), ('b', 'c'), ('d', 'e')), (('a', 'f'), ('b', 'd'), ('c', 'e')), (('a', 'f'), ('b', 'e'), ('c', 'd'))]
>>> # Some quick sanity checks
>>> len(list(non_overlapping_groups(2, 3, 'abcdef')))
10
>>> # With fewer input elements, obviously we can't do it.
>>> len(list(non_overlapping_groups(2, 3, 'abcde')))
0
>>> # Adding a 7th element, any element could be the odd one out,
>>> # and in each case we get another 10 possibilities, making 10 * 7 = 70.
>>> len(list(non_overlapping_groups(2, 3, 'abcdefg')))
70
I performance tested this against a modified version of the original (which also shows how to make it work properly with non-strings, and optimizes the sum calculation):
def unique_group(group_count, group_size, population):
groups = list(it.combinations(population, group_size))
return (
i for i in combinations(groups, group_count)
if len({e for g in i for e in g}) == group_count * group_size
)
Quickly verifying the equivalence:
>>> len(list(unique_group(3, 2, 'abcdef')))
15
>>> len(list(non_overlapping_groups(3, 2, 'abcdef')))
15
>>> set(unique_group(3, 2, 'abcdef')) == set(non_overlapping_groups(3, 2, 'abcdef'))
True
We see that even for fairly small examples (here, the output has 280 groupings), the brute-force approach has to filter through a lot:
>>> import timeit
>>> timeit.timeit("list(g(3, 3, 'abcdefghi'))", globals={'g': unique_group}, number=100)
5.895461600041017
>>> timeit.timeit("list(g(3, 3, 'abcdefghi'))", globals={'g': non_overlapping_groups}, number=100)
0.2303082060534507
I am using the following code to dedup and count a given list:
def my_dedup_count(l):
l.append(None)
new_l = []
current_x = l[0]
current_count = 1
for x in l[1:]:
if x == current_x:
current_count += 1
else:
new_l.append((current_x, current_count))
current_x = x
current_count = 1
return new_l
With my testing code:
my_test_list = ['a','a','b','b','b','c','c','d']
my_dedup_count(my_test_list)
result is:
[('a', 2), ('b', 3), ('c', 2), ('d', 1)]
The code is doing fine and the output is correct. However, I feel my code is quite lengthy and am wondering would anyone suggest a more elegant way to improve the above code? Thanks!
Yes, don't re-invent the wheel. Use the standard library instead; you want to use the collections.Counter() class here:
from collections import Counter
def my_dedup_count(l):
return Counter(l).items()
You may want to just return the counter itself and use all functionality it provides (such as giving you a key-count list sorted by counts).
If you expected only consecutive runs to be counted (so ['a', 'b', 'a'] results in [('a', 1), ('b', 1), ('a', 1)], then use itertools.groupby():
from itertools import groupby
def my_dedup_count(l):
return [(k, sum(1 for _ in g)) for k, g in groupby(l)]
I wrote two versions of some shorter ways to write what you accomplished.
This first option ignores ordering, and all like values in the list will be deduplicated.
from collections import defaultdict
def my_dedup_count(test_list):
foo = defaultdict(int)
for el in test_list:
foo[el] += 1
return foo.items()
my_test_list = ['a','a','b','b','b','c','c','d', 'a', 'a', 'd']
>>> [('a', 4), ('c', 2), ('b', 3), ('d', 2)]
This second option respects order and only deduplicates consecutive duplicate values.
def my_dedup_count(my_test_list):
output = []
succession = 1
for idx, el in enumerate(my_test_list):
if idx+1 < len(my_test_list) and el == my_test_list[idx+1]:
succession += 1
else:
output.append((el, succession))
succession = 1
return output
my_test_list = ['a','a','b','b','b','c','c','d', 'a', 'a', 'd']
>>> [('a', 2), ('b', 3), ('c', 2), ('d', 1), ('a', 2), ('d', 1)]
What is the most efficient way of finding a certain tuple based on e.g. the second element of that tuple in a list and move that tuple to the top of the list
Something of the form:
LL=[('a','a'),('a','b'),('a','c'),('a','d')]
LL.insert(0,LL.pop(LL.index( ... )))
where I would like something in index() that would give me the position of the tuple that has 'c' as second element.
Is there a classic python 1-line approach to do that?
>>> LL.insert(0,LL.pop([x for x, y in enumerate(LL) if y[1] == 'c'][0]))
>>> LL
[('a', 'c'), ('a', 'a'), ('a', 'b'), ('a', 'd')]
>>>
To find position you can:
positions = [i for i, tup in enumerate(LL) if tup[1] == 'c']
You can now take the index of the desired element, pop it and push to the beginning of the list
pos = positions[0]
LL.insert(0, LL.pop(pos))
But you can also sort your list using the item in the tuple as key:
sorted(LL, key=lambda tup: tup[1] == 'c', reverse=True)
if you don't care about order of the other elements
2 lines, however 1 line solutions are all inefficient
>>> LL=[('a','a'),('a','b'),('a','c'),('a','d')]
>>> i = next((i for i, (x, y) in enumerate(LL) if y == 'c'), 0) # 0 default index
>>> LL[0], LL[i] = LL[i], LL[0]
>>> LL
[('a', 'c'), ('a', 'b'), ('a', 'a'), ('a', 'd')]
This does nothing if the index is not found
>>> LL=[('a','a'),('a','b'),('a','c'),('a','d')]
>>> i = next((i for i, (x, y) in enumerate(LL) if y == 'e'), 0) # 0 default index
>>> LL[0], LL[i] = LL[i], LL[0]
>>> LL
[('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')]
The problem with terse, pythonic, 'fancy schmancy' solutions is the code might not be easily maintained and/or reused in other closely aligned contexts.
It seems best to just use 'boiler plate' code to do the search, and then continue with the application specific requirements.
So here is an example of easy to understand search code that can be easily 'plugged into' when these questions come up, including those situations when we need to know if the key is found.
def searchTupleList(list_of_tuples, coord_value, coord_index):
for i in range(0, len(list_of_tuples)):
if list_of_tuples[i][coord_index] == coord_value:
return i # matching index in list
return -1 # not found