Related
I have a list s:
s=[[1,2,1],[2,2,1],[1,2,1]]
Case 1: Remove the duplicate groups in the list
Case 2: Remove the duplicate values within the groups
Desired Result:
Case 1 : [[1,2,1],[2,2,1]]
Case 2 : [[1,2],[2,1],[1,2]]
I tried using the list(set(s)) but it throws up an error:
unhashable type: 'list'
IIUC,
Case 1:
Convert the lists to tuple for hashing, then apply a set on the list of tuples to remove the duplicates. Finally, convert back to lists.
out1 = list(map(list, set(map(tuple, s))))
# [[1, 2, 1], [2, 2, 1]]
Case 2:
For each sublist, remove the duplicates while keeping order with conversion to dictionary keys (that are unique), then back to list:
out2 = [list(dict.fromkeys(l)) for l in s]
# [[1, 2], [2, 1], [1, 2]]
You need to know that it's not possible in Python to have a set of lists. The reason is that lists are not hashable. The simplest way to do your task in the first case is to use a new list of lists without duplicates such as below:
temp_list = []
for each_element in [[1,2,1],[2,2,1],[1,2,1]]:
if each_element not in temp_list:
temp_set.append(each_element)
print(temp_list)
The output:
[[1, 2, 1], [2, 2, 1]]
The case2 is more simple:
temp_list = []
for each_element in [[1,2,1],[2,2,1],[1,2,1]]:
temp_list.append(list(set(each_element)))
print(temp_list)
And this is the output:
[[1, 2], [1, 2], [1, 2]]
However these codes are not the pythonic way of doing things, they are very simple to be understood by beginners.
So, say I have the following list and variable i:
data = [ [1, 2, 3, 1, 1], [1, 2, 3, 0, 0], [1, 2, 3, 2, 1] ]
i = 3
I would like to create a new list which will sum together and merge numbers from each sublist the ith element upwards to produce the following:
new_data = [ [1, 2, 5], [1, 2, 3], [1, 2, 6] ]
Where I have summed from each sublist the 3rd element upwards and merged the elements together. I would like to ask how, by using the standard library in Python, I can go about achieving this for an arbitrary (integer) value of i. I have the following idea in mind but I'm not sure how to put it into real Python code:
(pseudo-code)
new_data = []
for sublist in data:
new_data.append(elements before the ith element)
new_data.append(ith element onwards summed together)
You can slice your inner list by index and sum the rest with the one-liner:
>>> new_data = [row[:i-1] + [sum(row[i-1:])] for row in data]
>>> new_data
[[1, 2, 5], [1, 2, 5], [1, 2, 5]]
You can find a nice, "pythonic" one-liner in taras' answer:
new_data = [row[:i-1] + [sum(row[i-1:])] for row in data].
From pseudo-code to correct python code
I'll focus my answer on helping you transform your pseudo-code into python code.
Pseudo-code:
new_data = []
for sublist in data:
new_data.append(elements before the ith element)
new_data.append(ith element onwards summed together)
The first issue with your pseudo-code is that you are making a distinction between the list data and its sublists sublist, but you are not making a distinction between the list new_data and its sublists. Let me add a variable new_sublist in the loop:
new_data = []
for sublist in data:
new_sublist = []
new_sublist.append(elements before the ith element)
new_sublist.append(ith element onwards summed together)
new_data.append(new_sublist)
The second issue with your pseudo code: you make two calls to .append in each iteration of the loop. However, these two calls are not similar: the first call is supposed to append several elements, whereas the second call is supposed to append one element. Python makes a distinction between the two operations; if you want to add more than one element at once, use .extend instead of .append. The code becomes:
new_data = []
for sublist in data:
new_sublist = []
new_sublist.extend([elements before the ith element])
new_sublist.append(ith element onwards summed together)
new_data.append(new_sublist)
Finally we can turn your pseudo-code into proper python code, using a list slice to get the elements before the ith element, and builtin function sum along with a list slice to sum the ith element onwards:
new_data = []
for sublist in data:
new_sublist = []
new_sublist.extend(sublist[:i])
new_sublist.append(sum(sublist[i:]))
new_data.append(new_sublist)
Note that in python, looping over a list to construct a new list is such a common operation that we use the compact and elegant list comprehension syntax to do it instead of a multiline for-loop:
new_data = [sublist[:i] + [sum(sublist[i:])] for sublist in data]
Relevant documentation
list.extend and list.append;
list slices;
builtin function sum;
list comprehensions.
You can do it like so:
def merger(data, pos):
pos -= 1 # 0 based, your i above is 1 based
# collection list for result
result = []
# go over each given inner part
for inner in data:
# add it up to the correct position
result.append(inner[:pos])
# extend it by the sum of the remainder values
result[-1].append(sum(inner[pos:]))
return result
data = [ [1, 2, 3, 1, 1], [1, 2, 3, 1, 1], [1, 2, 3, 1, 1] ]
i = 3
print(merger(data,i))
Output:
[[1, 2, 5], [1, 2, 5], [1, 2, 5]]
or in short:
def merger(data, pos):
return [[inner[: pos - 1] + [sum(inner[pos - 1 :])] for inner in data]]
data = [ [1, 2, 3, 1, 1], [1, 2, 3, 1, 1], [1, 2, 3, 1, 1] ]
i = 3
new_data = [sublist[:i-1] + [sum(sublist[i-1:])] for sublist in data]
print(new_data)
>>> [[1, 2, 5], [1, 2, 5], [1, 2, 5]]
There are a few similar questions to this one but not exactly the same:
I want to dynamically decrease a given input array or list of lists. For example:
matrix = [[0,1,2], [3,4,5],[6,7,8]]
Starting at 0 I need to iterate through and remove the final index - the iterative. So the output I would like to store in a new list is:
#output
[0,1,2], ,[3,4], [6]]
[0,1,2], ,[3,4], [6]] ==> which then flattens to [0,1,2,3,4,6]
Here's what I'm currently going after:
def get_list(matrix, stop_index):
temp = []
for i in range(0, stop_index):
for m in matrix:
temp.append(matrix[0:stop_index])
outside_list.append(temp)
return outside_list
I believe I am seeing well my over reliance on packages and libraries, so I am really trying to do this without outside packages or imports
Thank you for any help! I don't forget to green check mark.
Using list comprehension
l = [[0,1,2], [3,4,5],[6,7,8]]
ll = [ x[:len(l)-l.index(x)] for x in l]
# [[0, 1, 2], [3, 4], [6]]
print([x for y in ll for x in y ])
# [0, 1, 2, 3, 4, 6]
Simpler syntax:
matrix = [[0,1,2], [3,4,5],[6,7,8]]
outside_list = list()
for i in range(len(matrix)):
# matrix[i] is used to access very sublist in the matrix,
#[:3-i] is to slice every sublist from the beginning to (3 - current position)
outside_list.append(matrix[i][:3-i])
print(outside_list)
Some useful refernces
List slicing https://stackoverflow.com/a/509295/8692977
List comprehension: https://stackoverflow.com/a/34835952/8692977
Q1:
I have an arraylist
x= [[1,2,-1],[1,-1,0],[-1,0,1]]
finally I want to get x = [[1,2,-1],[1,-1,0]] because [1,-1,0] and [-1,0,1] are the same but just different order.
Q2:
For
temp = [[0,0,0],[0,0,0],[0,0,0],[0,0,0]]
The same idea, I want to get temp = [[0,0,0]], which means droping all the other duplicates in the arraylist just like Q1.
My code does not work. It says list index out of range, but I use temp2 to len(temp1) changes.....why?
temp1 = [[0,0,0],[0,0,0],[0,0,0],[0,0,0]]
temp2 = temp1
for i in range(0, len(temp1)):
for j in range(i+1, len(temp1)):
if(set(temp1[i]) == set(temp1[j])):
temp2.remove(temp2[i])
You shouldn't change the list you're iterating over! Also temp2 = temp1 doesn't make a copy. You only have two names that refer to the same list afterwards. If you want to make a (shallow) copy, you could use temp2 = temp1.copy() or temp2 = temp1[:] or temp2 = list(temp1).
A general note: Using two iterations will have quadratic runtime behaviour it would be faster to keep the already processed items in a set which has O(1) lookup (most of the time):
temp1 = [[0,0,0],[0,0,0],[0,0,0],[0,0,0]]
temp2 = [] # simply creating a new list is probably easier.
seen = set()
for item in temp1:
# lists are not hashable so convert it to a frozenset (takes care of the order as well)
item_as_tuple = frozenset(item)
if item_as_tuple not in seen:
temp2.append(item)
seen.add(item_as_tuple)
If you can and want to use a third-party package, I have one that contains an iterator that does exactly that iteration_utilities.unique_everseen:
>>> from iteration_utilities import unique_everseen
>>> temp1 = [[0,0,0], [0,0,0], [0,0,0], [0,0,0]]
>>> list(unique_everseen(temp1, key=frozenset))
[[0, 0, 0]]
>>> x = [[1,2,-1], [1,-1,0], [-1,0,1]]
>>> list(unique_everseen(x, key=frozenset))
[[1, 2, -1], [1, -1, 0]]
Q1. If you want to consider lists equal when they contain the same elements, one way to do this is to sort them before comparison, like here:
def return_unique(list_of_lists):
unique = []
already_added = set()
for item in list_of_lists:
# Convert to tuple, because lists are not hashable.
# We consider two things to be the same regardless of the order
# so before converting to tuple, we also sort the list.
# This way [1, -1, 0] and [-1, 0, 1] both become (-1, 0, 1)
sorted_tuple = tuple(sorted(item))
# Check if we've already seen this tuple.
# If we haven't seen it yet, add the original list (in its
# original order) to the list of unique items
if sorted_tuple not in already_added:
already_added.add(sorted_tuple)
unique.append(item)
return unique
temp1 = [[1, 2, -1], [1, -1, 0], [-1, 0, 1]]
temp2 = [[0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0]]
print(return_unique(temp1))
print(return_unique(temp2))
Q2. Just assigning temp2 = temp1 does not create a new independent copy -- they both still refer to the same list. In this case, it would be possible to create an independent copy using copy.deepcopy:
import copy
temp2 = copy.deepcopy(temp1)
in order to remove dups we can first sort the list:
lsts = [[1,2,-1],[1,-1,0],[-1,0,1]]
lsts = [sorted(x) for x in lsts]
then convert the lists to tuples and add them to a set which will eliminate duplications (we cannot add lists to a set since they're not hashable, so we have to convert them to tuples first):
res = set()
for x in lsts:
res.add(tuple(x))
then we can convert the tuples and the set back to lists:
lsts = list(list(x) for x in res)
print(lsts) # [[-1, 1, 2], [-1, 0, 1]]
The reason you're failing is because you're modifying the list you're iterating, so by removing items you make the list shorter and then you're trying to access an index which no longer exists, but you can fix it by iterating the list without using indexes:
temp1 = [[0,0,0],[0,0,0],[0,0,0],[0,0,0]]
for x in temp1:
temp2 = temp1[:] # create a real copy of temp1
temp2.remove(x) # remove x so we won't consider it as dup of itself
for y in temp2:
if set(x) == set(y):
temp1.remove(x)
print(temp1) # [[0, 0, 0]]
A set will do:
lsts = [[1,2,-1],[1,-1,0],[-1,0,1]]
result = {tuple(sorted(x)) for x in lsts}
You may use groupby
from itertools import groupby
[i for i,k in groupby(x, lambda j:sorted(j))]
output:
[[-1, 1, 2], [-1, 0, 1]]
This works for Q2.
temp1 = [[0,0,0],[0,0,0],[0,0,0],[0,0,0]]
temp2 = []
for element in temp1:
if element not in temp2:
temp2.append(element)
temp2
>>>[[0, 0, 0]]
I am trying to append small (num) lists to a final list (lst).
The small lists are modified in a loop and at each modification they are appended to the final list.
But the result is strange and I did not like the solution.
Here is the code.
n = 3
num = []
lst = []
for i in range(n) :
num.append(0)
lst.append(num)
for j in range(n-1) :
for i in range(n) :
num[i] += 1
lst.append(num)
for each in lst :
print each
The result of this code is:
[2, 2, 2]
[2, 2, 2]
[2, 2, 2]
[2, 2, 2]
[2, 2, 2]
[2, 2, 2]
[2, 2, 2]
However, if instead of using lst.append(num), I use lst.append(list(num)), I get the expected result:
[0, 0, 0]
[1, 0, 0]
[1, 1, 0]
[1, 1, 1]
[2, 1, 1]
[2, 2, 1]
[2, 2, 2]
Is this really the correct way for appending values of a list to another list?
UPDATE:
I see that when I do lst.append(num) this does not mean that the values of num are appended to lst. It is a reference and because I change the content of num, in the end, all entries of my list lst will reference to the final num. However, if I do this:
lst = []
for i in range(3) :
num = i
lst.append(num)
for each in lst :
print each
I get what is expected, i.e., a list lst with the values [0, 1 , 2]. And in this case, num is changed at every iteration and its final value is num=2.
Lets have a closer look at this loop.
for i in range(n) :
num[i] += 1
lst.append(num)
You are appending same num multiple times. So, all elements of lst has the same reference to list num and you are updating it at each iteration. That is why you are getting same value at each element of lst after the loop has completed.
But, when you are doing lst.append(list(num)), you are generating a new list each time. So, each element of lst has different reference. And you are getting expected output.
So, to answer your question, the way you are adding a list to a list is fine. That is how you should do it. It has nothing to do with your expected outcome.
I think the explanation for your result is that when you do num[i] += 1 you are actually modifying the preceding copies of num that were appended in lst at previous stages of the loop. When you do lst.append(list(num)) you are creating a unique version of num.