We Keep Coding

Slice array of tuples by its elements

For example, I have an array of tuples.
l=[(456,33,1),
(556,22,1),
(123,33,2),
(557,32,2),
(435,21,2)]
I want make the arrays of lists by the last element in each tuple, so that
l=[[(456,33),
(556,22)],
[(123,33),
(557,32),
(435,21)]]
How could I do that? Is there any elegant way?

l = [(456, 33, 1), (556, 22, 1), (123, 33, 2), (557, 32, 2), (435, 21, 2)]
out = {}
for v in l:
out.setdefault(v[-1], []).append(v[:-1])
out = list(out.values())
print(out)
Prints:
[[(456, 33), (556, 22)],
[(123, 33), (557, 32), (435, 21)]]

Another way of doing this would be with itertools.groupby.
from itertools import groupby
from operator import itemgetter
res = [[x[:-1] for x in g] for _, g in groupby(sorted(l, key=itemgetter(2)), itemgetter(2))]
which produces the desired:
[[(456, 33), (556, 22)], [(123, 33), (557, 32), (435, 21)]]
Note that if you can guarantee that the original list l is sorted by the 3rd item in the tuples, you can replace sorted(l, key=itemgetter(2)) with just l.

An alternative attempt is using this:
l=[(456,33,1),
(556,22,1),
(123,33,2),
(557,32,2),
(435,21,2)]
dict_ = {}
for v in l:
val1, val2, id_ = v
dict_[id_] = [[val1, val2]] if id_ not in dict_ else dict_[id_] + [[val1, val2]]
l = [dict_[x] for x in dict_]

Convert list of lists to unique dictionary in python

I have the following list of lists: [[100, 23], [101, 34], [102, 35] ... ]
How can i convert it to a dictionary like that: {100: 23, 101: 34, 102: 35 ... }
Here is what i tried:
myDict = dict(map(reversed, myList))
This code works, but it will give the opposite of what i want in the dictionary: {23: 100, 101:34..}

You can pass a list with this format to dict() and it will convert it for you.
myList = [[100, 23], [101, 34], [102, 35]]
myDict = dict(myList)
print(myDict)

According to docs page for dict()
. . . the positional argument must be an iterable object. Each item in the iterable must itself be an iterable with exactly two objects. The first object of each item becomes a key in the new dictionary, and the second object the corresponding value.
and here is an example for the same doc page:
...
>>> d = dict([('two', 2), ('one', 1), ('three', 3)])
returns a dictionary equal to {"one": 1, "two": 2, "three": 3}

L = []
A = {}
for x,y in L:
A[x] = y

Find numbers within a range in an arbitrary number of nested lists

I have an arbitrary number of nested lists (let's say two for simplicity) with the same length that look something like that:
EDIT
In this edit I change the example lists to two specific, that seem to cause trouble:
l1 = [[96, 110], [49, 95, 122], [173, 218], [30], [80, 159], [95, 119, 150, 168]]
l2 = [[25, 110], [63, 126], [130, 222], [42], [3], [94, 119, 150, 176]]
Now I want a function that checks, for every index, if there exist entries (and which and how many) in every list that lay within a given range and returns them.
Let's say the range is 20. In this example I would like to return
[[[110, 96], [110, 110]], [[63, 49], [126, 122]], [222, 218], [42, 30], [], [[95,94],[119, 119], [150, 150], [176, 168]]]
I know that for two lists I can use itertools like this:
result = []
for i in range(len(l1): # the lists have the same length
result.append(
[[a,b] for (a, b) in itertools.product(l1[i], l2[i])
if a-20 <= b <=a+20])
In that example I would need to check whether or not my entry in the nested list is an int or not and use another way to compare my entries, but that's rather secondary.
The big question is how to do it with more than two lists. I have thougth about a recursive solution, but could not work out something properly.
EDIT
With more than two lists I mean I have more lists like l1 or l2 with the same length as the others.
The solutions given by #MishaMelnyk and #AlainT are already really helpfull, but the results depent on the the order of the list
Result for the given solutions with order l1, l2:
[[[110, 96], [110, 110]], [[63, 49], [126, 122]], [[222, 218]], [[42, 30]], [], [[119, 119], [150, 150], [176, 168]]]
or order l2, l1
[[[110, 110]], [], [], [[30, 42]], [], [[95, 94], [119, 119], [150, 150], [168, 150]]]
Happy for any suggestions

This is what I made based on what you mentioned:
l1 = [[80,112,270],[20,78], 6, [99,134,240,300]]
l2 = [30, [22,84],[7,122,189,279],[67,100]]
l3 = [60, [25, 70], [2], [110]]
def makeZip(maxRange, *args):
for l in args: #For each index in the lists, converts any integers to lists
for i in range(len(l)):
if type(l[i]) == int:
l[i] = [l[i]]
z = zip(*args)
#Zip makes lists for each video with all of the entries
#Basically Equivilant to transposing a matrix in Lin Alg
matches = []
for l in z: #For each video, generates matching pairs
videoMatches = []
for m in makeMatch(maxRange, l): #For all of the pairs, add to list
videoMatches.append(m)
matches.append(videoMatches) #Add the list to the main list
return matches
def makeMatch(maxRange, l):
if len(l) == 1: #If only one list (person) then return all of the values sequentially (generator)
for i in l[0]:
yield [i]
return
matches = []
for n in makeMatch(maxRange, l[1:]): #for each of the generated match from a recursive call
for i in l[0]: #For all of the values of the current person
if all([abs(i - x) < maxRange for x in n]): #Check if value works for all of the values already in the match
matches.append([i] + n) #Sucessful match
for m in matches: #when done with all of the matches, return them sequentially (generator)
yield m
for m in makeZip(20, l1, l2, l3):
print(m)
You might want to rename the variables though. Hopefully, the output is what should be for three lists.
One problem that you might have with this solution is that I'm pretty sure O(numVideos^numPeople) in the worst case where everything matches. Might be wrong about the complexity though.

Once you have solved the problem for two lists, you can use it iteratively by starting with the first two, then merge list 1 and list 2 and perform the check between the merged lists and list 3, then merge list 3 to that and process the merged list with list 4 , and so on.
The comparison logic between two list could be greatly accelerated by sorting sublists in list1 and using bisect_left to find the first element 'b' that is >= to a-20, then progress sequentially in the sorted elements until you get beyond a+20. You can do that for every item 'a' in the corresponding sublist of list 2. This will give you a time complexity of O(NlogM) instead of O(N*M) which will become even more important as you merge lists in a multi-list process.
Here is a concrete example of the multi-list process.
Note that I did not include the bisect search optimization in the matchSubLists function (that would only be needed if your sublists are large enough)
def matchSubLists(sA,sB,match):
return [ (a,b) for b in sB for a in sA if match(a,b) ]
def match2Lists(A,B,match):
return [ matchSubLists(sA,sB,match) for sA,sB in zip(A,B)]
def merge2Lists(A,B):
return [ sA+sB for sA,sB in zip(A,B) ]
def matchMultiLists(*L,match):
result = [[] for _ in L[0]]
merged = L[0]
for Ln in L[1:]:
matches = match2Lists(merged,Ln,match)
result = merge2Lists(result,matches)
merged = merge2Lists(merged,Ln)
return result
ouput:
l1 = [[80,112,270], [20,78], [6], [99,134,240,300]]
l2 = [[30], [22,84], [7,122,189,279], [67,100]]
l3 = [[60], [25, 70], [2], [110]]
result = matchMultiLists(l1,l2,l3, match=lambda a,b:abs(a-b)<=20)
print(result)
[
[(80, 60)],
[(20, 22), (78, 84), (20, 25), (22, 25), (78, 70), (84, 70)],
[(6, 7), (6, 2), (7, 2)],
[(99, 100), (99, 110), (100, 110)]
]
I used one entry sublists instead of int values to work with a more consistent data structure and avoid unnecessary exceptions in the logic
[EDIT]
If you want the output to be the same independently of the order of the lists in the call to matchMultiList, you can add a sort before returning the result:
def matchMultiLists(*L,match):
result = [[] for _ in L[0]]
merged = L[0]
for Ln in L[1:]:
matches = match2Lists(merged,Ln,match)
result = merge2Lists(result,matches)
merged = merge2Lists(merged,Ln)
# consistently ordered result (2-level sort)
result = [ sorted( map(tuple,map(sorted,sR)) ) for sR in result ]
return result
Since you can use matchMultiLists with two lists, you don't need to add the sort to the match2Lists() function. In fact the 3 one-line functions could be defined inside of the matchMultiLists() function to avoid exposing them.
output:
l1=[[96, 110], [49, 95, 122], [173, 218], [30], [80, 159], [95, 119, 150, 168]]
l2=[[25, 110], [63, 126], [130, 222], [42], [3], [94, 119, 150, 176]]
range20 = lambda a,b:abs(a-b)<=20
print(matchMultiLists(l1,l2, match=range20))
[[(96, 110), (110, 110)], [(49, 63), (122, 126)], [(218, 222)], [(30, 42)], [], [(94, 95), (119, 119), (150, 150), (150, 168), (168, 176)]]
print(matchMultiLists(l2,l1, match=range20))
[[(96, 110), (110, 110)], [(49, 63), (122, 126)], [(218, 222)], [(30, 42)], [], [(94, 95), (119, 119), (150, 150), (150, 168), (168, 176)]]

You can expand your solution to first choose all possible couples of sublists (at the same index) using combinations, and then go through all couples of items using product. Something like:
import itertools
result = []
for sub_lists in zip(l1, l2 ,l3):
for couple_subs in itertools.combinations(sub_lists, 2):
result.append(
[[a,b] for a, b in itertools.product(*couple_subs)
if abs(a-b) <= 20])
To handle unknown level of nesting you can first flatten the sub-lists before passing them to product:
def flatten(l):
for el in l:
if isinstance(el, list):
yield from flatten(el)
else:
yield el
And now you can use that in the code above as:
[[a,b] for a, b in itertools.product(flatten(couple_subs[0]), flatten(couple_subs[1]))
if abs(a-b) <= 20]
Ex:
import itertools
l1 = [[1, [4, 11], 2], [3,4]]
l2 = [[5,6], [7,8]]
l3 = [[9, 10], [11, 12]]
result = []
def flatten(l):
for el in l:
if isinstance(el, list):
yield from flatten(el)
else:
yield el
for sub_lists in zip(l1, l2 ,l3):
for couple_subs in itertools.combinations(sub_lists, 2):
result.append(
[[a,b] for a, b in itertools.product(flatten(couple_subs[0]), flatten(couple_subs[1]))
if abs(a-b) <= 3])
print(result)
Gives:
[[[4, 5], [4, 6], [2, 5]], [[11, 9], [11, 10]], [[6, 9]], [[4, 7]], [], [[8, 11]]]

Sorting tuples with a custom sorting function using a range?

I would like to sort a list of tuples based on the two last columns:
mylist = [(33, 36, 84),
(34, 37, 656),
(23, 38, 42)]
I know I can do this like:
final = sorted(mylist, key:lambda x: [ x[1], x[2]])
Now my problem is that I want to compare the second column of my list with a special condition: if the difference between two numbers is less than an offset they should be taken as equal ( 36 == 37 == 38) and the third column should be used to sort the list. The end result I wish to see is:
mylist = [(23, 38, 42)
(33, 36, 84),
(34, 37, 656)]
I was thinking of creating my own integer type and overriding the equal operator. Is this possible? is it overkill? Is there a better way to solve this problem?

I think the easiest way is to create a new class that compares like you want it to:
mylist = [(33, 36, 84),
(34, 37, 656),
(23, 38, 42)]
offset = 2
class Comp(object):
def __init__(self, tup):
self.tup = tup
def __lt__(self, other): # sorted works even if only __lt__ is implemented.
# If the difference is less or equal the offset of the second item compare the third
if abs(self.tup[1] - other.tup[1]) <= offset:
return self.tup[2] < other.tup[2]
# otherwise compare them as usual
else:
return (self.tup[1], self.tup[2]) < (other.tup[1], other.tup[2])
A sample run shows your expected result:
>>> sorted(mylist, key=Comp)
[(23, 38, 42), (33, 36, 84), (34, 37, 656)]
I think it's a bit cleaner than using functools.cmp_to_key but that's a matter of personal preference.

Sometimes an old-style sort based on a cmp function is easier than doing one based on a key. So -- write a cmp function and then use functools.cmp_to_key to convert it to a key:
import functools
def compare(s,t,offset):
_,y,z = s
_,u,v = t
if abs(y-u) > offset: #use 2nd component
if y < u:
return -1
else:
return 1
else: #use 3rd component
if z < v:
return -1
elif z == v:
return 0
else:
return 1
mylist = [(33, 36, 84),
(34, 37, 656),
(23, 38, 42)]
mylist.sort(key = functools.cmp_to_key(lambda s,t: compare(s,t,2)))
for t in mylist: print(t)
output:
(23, 38, 42)
(33, 36, 84)
(34, 37, 656)

In https://wiki.python.org/moin/HowTo/Sorting look for "The Old Way Using the cmp Parameter". This allows you to write your own comparison function, instead of just setting the key and using comparison operators.
There is a danger to making a sort ordering like this. Look up "strict weak ordering." You could have multiple different valid orderings. This can break other code which assumes there is one correct way to sort things.
Now to actually answer your question:
mylist = [(33, 36, 84),
(34, 37, 656),
(23, 38, 42)]
def custom_sort_term(x, y, offset = 2):
if abs(x-y) <= offset:
return 0
return x-y
def custom_sort_function(x, y):
x1 = x[1]
y1 = y[1]
first_comparison_result = custom_sort_term(x1, y1)
if (first_comparison_result):
return first_comparison_result
x2 = x[2]
y2 = y[2]
return custom_sort_term(x2, y2)
final = sorted(mylist, cmp=custom_sort_function)
print final
[(23, 38, 42), (33, 36, 84), (34, 37, 656)]

Not pretty, but I tried be general in my interpretation of OP's problem statement
I expanded test case, then applied unsophisticated blunt force
# test case expanded
mylist = [(33, 6, 104),
(31, 36, 84),
(35, 86, 84),
(30, 9, 4),
(23, 38, 42),
(34, 37, 656),
(33, 88, 8)]
threshld = 2 # different final output can be seen if changed to 1, 3, 30
def collapse(nums, threshld):
"""
takes sorted (increasing) list of numbers, nums
replaces runs of consequetive nums
that successively differ by threshld or less
with 1st number in each run
"""
cnums = nums[:]
cur = nums[0]
for i in range(len(nums)-1):
if (nums[i+1] - nums[i]) <= threshld:
cnums[i+1] = cur
else:
cur = cnums[i+1]
return cnums
mylists = [list(i) for i in mylist] # change the tuples to lists to modify
indxd=[e + [i] for i, e in enumerate(mylists)] # append the original indexing
#print(*indxd, sep='\n')
im0 = sorted(indxd, key=lambda x: [ x[1]]) # sort by middle number
cns = collapse([i[1] for i in im0], threshld) # then collapse()
#print(cns)
for i in range(len(im0)): # overwrite collapsed into im0
im0[i][1] = cns[i]
#print(*im0, sep='\n')
im1 = sorted(im0, key=lambda x: [ x[1], x[2]]) # now do 2 level sort
#print(*sorted(im0, key=lambda x: [ x[1], x[2]]), sep='\n')
final = [mylist[im1[i][3]] for i in range(len(im1))] # rebuid using new order
# of original indices
print(*final, sep='\n')
(33, 6, 104)
(30, 9, 4)
(23, 38, 42)
(31, 36, 84)
(34, 37, 656)
(33, 88, 8)
(35, 86, 84)

removing something from a list of tuples

Say I have a list:
[(12,34,1),(123,34,1),(21,23,1)]
I want to remove the 1 from each tuple in the list so it becomes
[(12,34),(123,34),(21,23)]

You want to truncate your tuples, use a list comprehension:
[t[:-1] for t in listoftuples]
or, as a simple demonstration:
>>> listoftuples = [(12,34,1),(123,34,1),(21,23,1)]
>>> [t[:-1] for t in listoftuples]
[(12, 34), (123, 34), (21, 23)]

Tuples are immutable, so you can not remove an item. However, you can create a new tuple from the old tuple not including the elements you do not want to. So, to delete an arbitrary item from each tuple from a list of tuples, you can do:
def deleteItem(lst, toDel):
return [tuple(x for x in y if x != toDel) for y in lst]
Result:
>>> lst = [(12,34,1),(123,34,1),(21,23,1)]
>>> deleteItem(lst, 1)
[(12, 34), (123, 34), (21, 23)]

>>> a=[(12, 34, 1), (123, 34, 1), (21, 23, 1)]
>>> [filter (lambda a: a != 1, x) for x in a]
[(12, 34), (123, 34), (21, 23)]
THis will remove all 1 from the tuple irrespective of index

Since you can't change tuples (as they are immutable), I suggest using a list of lists:
my_list = [[12,34,1],[123,34,1],[21,23,1]]
for i in my_list:
i.remove(1)
return my_list
This returns: [[12, 34], [123, 34], [21, 21]].

python 3.2
1. [(i,v)for i,v,c in list1]
2. list(map(lambda x:x[:2],list1))