Related
For example, I have three lists (of the same length)
A = [1,2,3]
B = [a,b,c]
C = [x,y,z]
and i want to merge it into something like:
[[1,a,x],[2,b,y],[3,c,z]].
Here is what I have so far:
define merger(A,B,C):
answer =
for y in range (len(A)):
a = A[y]
b = B[y]
c = C[y]
temp = [a,b,c]
answer = answer.extend(temp)
return answer
Received error:
'NoneType' object has no attribute 'extend'
It looks like your code is meant to say answer = [], and leaving that out will cause problems. But the major problem you have is this:
answer = answer.extend(temp)
extend modifies answer and returns None. Leave this as just answer.extend(temp) and it will work. You likely also want to use the append method rather than extend - append puts one object (the list temp) at the end of answer, while extend appends each item of temp individually, ultimately giving the flattened version of what you're after: [1, 'a', 'x', 2, 'b', 'y', 3, 'c', 'z'].
But, rather than reinventing the wheel, this is exactly what the builtin zip is for:
>>> A = [1,2,3]
>>> B = ['a', 'b', 'c']
>>> C = ['x', 'y', 'z']
>>> list(zip(A, B, C))
[(1, 'a', 'x'), (2, 'b', 'y'), (3, 'c', 'z')]
Note that in Python 2, zip returns a list of tuples; in Python 3, it returns a lazy iterator (ie, it builds the tuples as they're requested, rather than precomputing them). If you want the Python 2 behaviour in Python 3, you pass it through list as I've done above. If you want the Python 3 behaviour in Python 2, use the function izip from itertools.
To get a list of lists, you can use the built-in function zip() and list comprehension to convert each element of the result of zip() from a tupleto a list:
A = [1, 2, 3]
B = [4, 5, 6]
C = [7, 8, 9]
X = [list(e) for e in zip(A, B, C,)]
print X
>>> [[1, 4, 7], [2, 5, 8], [3, 6, 9]]
Assuming you are doing this for class and not learning all of the tricks that make Python a great tool here is what you need. You had two problems, first if you want to extend then you do it in place but your desired result shows that you want to append, not extend
def merger(A,B,C):
answer = []
for y in range (len(A)):
a=A[y]
b=B[y]
c=C[y]
temp = [a,b,c]
answer.append(temp)
return answer
>>> answer
[[1, 'a', 'x'], [2, 'b', 'y'], [3, 'c', 'z']]
I was just wondering the same thing. I'm a total noob using code academy. This is what i came up to combine two lists index at index
toppings = ['pepperoni', 'pineapple', 'cheese', 'sausage', 'olives', 'anchovies', 'mushrooms']
prices = [2,6,1,3,2,7,2]
num_pizzas = len(toppings)
print("We sell "+str(num_pizzas)+" different kinds of pizza!")
***pizzas = list(zip(toppings, prices))***
print (pizzas)
the list pizzas printed out ...[('pepperoni', 2), ('pineapple', 6), ('cheese', 1), ('sausage', 3), ('olives', 2), ('anchovies', 7), ('mushrooms', 2)]
I have something like this:-
List = [["a","1"],["b","2"]]
and what I wanna do is keep single letter integers as integers.
so output should be like
List = [["a",1],["b",2]]
Assuming that you stored that in a list called "data", you can do the following.
new_data = [[k,int(v)] for k,v in data]
Refer below for details:
data = [["a","1"],["b","2"]]
new_data = [[k,int(v)] for k,v in data]
print new_data
Output:
[['a', 1], ['b', 2]]
Changed question before rollback: Sort heterogeneous list
It seems that you completely changed your question after people answered the first version. Please don't do that. You could simply ask a new question.
If you want to sort a heterogeneous list, you can provide a custom key which returns a tuple. The first element is 0 for strings and 1 for integers. This way, the strings would appear before the integers.
If the object is an integer, the second element is set to -x in order to sort the integers in decreasing order:
def custom_order(x):
if isinstance(x, int):
return (1, -x)
else:
return (0, x)
print(sorted([1,2,3,4,5,"a","b","c","d"], key=custom_order))
# ['a', 'b', 'c', 'd', 5, 4, 3, 2, 1]
This code should work on Python2 and Python3. It will fail on Python3 if an element is neither a string nor an int.
Original question: convert nested strings to ints
You could use a nested list comprehension with a ternary operator to check if the string looks like an integer:
>>> data = [["a","1"],["b","2"]]
>>> [[int(s) if s.isdecimal() else s for s in l] for l in data]
[['a', 1], ['b', 2]]
As a bonus, it would work in any order and with sub-lists of any size:
>>> data = [["a","1"],["b","2"],["3", "c"], ["4", "5", "d"]]
>>> [[int(s) if s.isdecimal() else s for s in l] for l in data]
[['a', 1], ['b', 2], [3, 'c'], [4, 5, 'd']]
I have three lists:
del_ids = [2, 4]
ids = [3, 2, 4, 1]
other = ['a', 'b', 'c', 'd']
and my goal is to remove del_ids with the result being
ids = [3, 1]
other = ['a', 'd']
I have tried to do a mask for elements to keep (mask = [id not in del_ids for id in ids]) and I plan to apply this mask on both lists.
But I feel that this is not a pythonic solution. Can you please tell me how I can do this better?
zip, filter and unzip again:
ids, other = zip(*((id, other) for id, other in zip(ids, other) if id not in del_ids))
The zip() call pairs each id with the corresponding other element, the generator expression filters out any pair where the id is listed in del_ids, and the zip(*..) then teases out the remaining pairs into separate lists again.
Demo:
>>> del_ids = [2, 4]
>>> ids = [3, 2, 4, 1]
>>> other = ['a', 'b', 'c', 'd']
>>> zip(*((id, other) for id, other in zip(ids, other) if id not in del_ids))
[(3, 1), ('a', 'd')]
zip, filter, unzip :
ids, other = zip(*filter(lambda (id,_): not id in del_ids, zip(ids, other)))
In order to avoid learning tricky syntax, do it in two steps.
other = [o for my_id, o in zip(ids, other) if my_id not in del_ids]
ids = [my_id for my_id in ids if my_id not in del_ids]
Drawback
You must execute the statements in correct order, so there's risk of bugs if for some reason the order changes.
Advantage
It's straight forward, so you don't have to search Stackoverflow next time you want to do it.
Converting to pandas data frame and applying the mask:
del_ids = [2, 4]
ids = [3, 2, 4, 1]
other = ['a', 'b', 'c', 'd']
df = pd.DataFrame({'ids':ids,'other':other})
df = df[~df.ids.isin(del_ids)]
ids = df['ids'].tolist()
other = df['other'].tolist()
Recently I noticed that when I am converting a list to set the order of elements is changed and is sorted by character.
Consider this example:
x=[1,2,20,6,210]
print(x)
# [1, 2, 20, 6, 210] # the order is same as initial order
set(x)
# set([1, 2, 20, 210, 6]) # in the set(x) output order is sorted
My questions are -
Why is this happening?
How can I do set operations (especially set difference) without losing the initial order?
A set is an unordered data structure, so it does not preserve the insertion order.
This depends on your requirements. If you have an normal list, and want to remove some set of elements while preserving the order of the list, you can do this with a list comprehension:
>>> a = [1, 2, 20, 6, 210]
>>> b = set([6, 20, 1])
>>> [x for x in a if x not in b]
[2, 210]
If you need a data structure that supports both fast membership tests and preservation of insertion order, you can use the keys of a Python dictionary, which starting from Python 3.7 is guaranteed to preserve the insertion order:
>>> a = dict.fromkeys([1, 2, 20, 6, 210])
>>> b = dict.fromkeys([6, 20, 1])
>>> dict.fromkeys(x for x in a if x not in b)
{2: None, 210: None}
b doesn't really need to be ordered here – you could use a set as well. Note that a.keys() - b.keys() returns the set difference as a set, so it won't preserve the insertion order.
In older versions of Python, you can use collections.OrderedDict instead:
>>> a = collections.OrderedDict.fromkeys([1, 2, 20, 6, 210])
>>> b = collections.OrderedDict.fromkeys([6, 20, 1])
>>> collections.OrderedDict.fromkeys(x for x in a if x not in b)
OrderedDict([(2, None), (210, None)])
In Python 3.6, set() now should keep the order, but there is another solution for Python 2 and 3:
>>> x = [1, 2, 20, 6, 210]
>>> sorted(set(x), key=x.index)
[1, 2, 20, 6, 210]
Remove duplicates and preserve order by below function
def unique(sequence):
seen = set()
return [x for x in sequence if not (x in seen or seen.add(x))]
How to remove duplicates from a list while preserving order in Python
Answering your first question, a set is a data structure optimized for set operations. Like a mathematical set, it does not enforce or maintain any particular order of the elements. The abstract concept of a set does not enforce order, so the implementation is not required to. When you create a set from a list, Python has the liberty to change the order of the elements for the needs of the internal implementation it uses for a set, which is able to perform set operations efficiently.
In mathematics, there are sets and ordered sets (osets).
set: an unordered container of unique elements (Implemented)
oset: an ordered container of unique elements (NotImplemented)
In Python, only sets are directly implemented. We can emulate osets with regular dict keys (3.7+).
Given
a = [1, 2, 20, 6, 210, 2, 1]
b = {2, 6}
Code
oset = dict.fromkeys(a).keys()
# dict_keys([1, 2, 20, 6, 210])
Demo
Replicates are removed, insertion-order is preserved.
list(oset)
# [1, 2, 20, 6, 210]
Set-like operations on dict keys.
oset - b
# {1, 20, 210}
oset | b
# {1, 2, 5, 6, 20, 210}
oset & b
# {2, 6}
oset ^ b
# {1, 5, 20, 210}
Details
Note: an unordered structure does not preclude ordered elements. Rather, maintained order is not guaranteed. Example:
assert {1, 2, 3} == {2, 3, 1} # sets (order is ignored)
assert [1, 2, 3] != [2, 3, 1] # lists (order is guaranteed)
One may be pleased to discover that a list and multiset (mset) are two more fascinating, mathematical data structures:
list: an ordered container of elements that permits replicates (Implemented)
mset: an unordered container of elements that permits replicates (NotImplemented)*
Summary
Container | Ordered | Unique | Implemented
----------|---------|--------|------------
set | n | y | y
oset | y | y | n
list | y | n | y
mset | n | n | n*
*A multiset can be indirectly emulated with collections.Counter(), a dict-like mapping of multiplicities (counts).
You can remove the duplicated values and keep the list order of insertion with one line of code, Python 3.8.2
mylist = ['b', 'b', 'a', 'd', 'd', 'c']
results = list({value:"" for value in mylist})
print(results)
>>> ['b', 'a', 'd', 'c']
results = list(dict.fromkeys(mylist))
print(results)
>>> ['b', 'a', 'd', 'c']
As denoted in other answers, sets are data structures (and mathematical concepts) that do not preserve the element order -
However, by using a combination of sets and dictionaries, it is possible that you can achieve wathever you want - try using these snippets:
# save the element order in a dict:
x_dict = dict(x,y for y, x in enumerate(my_list) )
x_set = set(my_list)
#perform desired set operations
...
#retrieve ordered list from the set:
new_list = [None] * len(new_set)
for element in new_set:
new_list[x_dict[element]] = element
Building on Sven's answer, I found using collections.OrderedDict like so helped me accomplish what you want plus allow me to add more items to the dict:
import collections
x=[1,2,20,6,210]
z=collections.OrderedDict.fromkeys(x)
z
OrderedDict([(1, None), (2, None), (20, None), (6, None), (210, None)])
If you want to add items but still treat it like a set you can just do:
z['nextitem']=None
And you can perform an operation like z.keys() on the dict and get the set:
list(z.keys())
[1, 2, 20, 6, 210]
One more simpler way can be two create a empty list ,let's say "unique_list" for adding the unique elements from the original list, for example:
unique_list=[]
for i in original_list:
if i not in unique_list:
unique_list.append(i)
else:
pass
This will give you all the unique elements as well as maintain the order.
Late to answer but you can use Pandas, pd.Series to convert list while preserving the order:
import pandas as pd
x = pd.Series([1, 2, 20, 6, 210, 2, 1])
print(pd.unique(x))
Output:
array([ 1, 2, 20, 6, 210])
Works for a list of strings
x = pd.Series(['c', 'k', 'q', 'n', 'p','c', 'n'])
print(pd.unique(x))
Output
['c' 'k' 'q' 'n' 'p']
An implementation of the highest score concept above that brings it back to a list:
def SetOfListInOrder(incominglist):
from collections import OrderedDict
outtemp = OrderedDict()
for item in incominglist:
outtemp[item] = None
return(list(outtemp))
Tested (briefly) on Python 3.6 and Python 2.7.
In case you have a small number of elements in your two initial lists on which you want to do set difference operation, instead of using collections.OrderedDict which complicates the implementation and makes it less readable, you can use:
# initial lists on which you want to do set difference
>>> nums = [1,2,2,3,3,4,4,5]
>>> evens = [2,4,4,6]
>>> evens_set = set(evens)
>>> result = []
>>> for n in nums:
... if not n in evens_set and not n in result:
... result.append(n)
...
>>> result
[1, 3, 5]
Its time complexity is not that good but it is neat and easy to read.
It's interesting that people always use 'real world problem' to make joke on the definition in theoretical science.
If set has order, you first need to figure out the following problems.
If your list has duplicate elements, what should the order be when you turn it into a set? What is the order if we union two sets? What is the order if we intersect two sets with different order on the same elements?
Plus, set is much faster in searching for a particular key which is very good in sets operation (and that's why you need a set, but not list).
If you really care about the index, just keep it as a list. If you still want to do set operation on the elements in many lists, the simplest way is creating a dictionary for each list with the same keys in the set along with a value of list containing all the index of the key in the original list.
def indx_dic(l):
dic = {}
for i in range(len(l)):
if l[i] in dic:
dic.get(l[i]).append(i)
else:
dic[l[i]] = [i]
return(dic)
a = [1,2,3,4,5,1,3,2]
set_a = set(a)
dic_a = indx_dic(a)
print(dic_a)
# {1: [0, 5], 2: [1, 7], 3: [2, 6], 4: [3], 5: [4]}
print(set_a)
# {1, 2, 3, 4, 5}
We can use collections.Counter for this:
# tested on python 3.7
>>> from collections import Counter
>>> lst = ["1", "2", "20", "6", "210"]
>>> for i in Counter(lst):
>>> print(i, end=" ")
1 2 20 6 210
>>> for i in set(lst):
>>> print(i, end=" ")
20 6 2 1 210
You can remove the duplicated values and keep the list order of insertion, if you want
lst = [1,2,1,3]
new_lst = []
for num in lst :
if num not in new_lst :
new_lst.append(num)
# new_lst = [1,2,3]
don't use 'sets' for removing duplicate if 'order' is something you want,
use sets for searching i.e.
x in list
takes O(n) time
where
x in set
takes O(1) time *most cases
Here's an easy way to do it:
x=[1,2,20,6,210]
print sorted(set(x))
If I have a list:
my_list = [3,2,2,3,4,1,3,4]
and a tuple
my_tuple = (3,5)
What's the best way of replacing elements in my_list using the tuple:
result = [5,2,2,5,4,1,5,4]
e.g.
for item in my_list:
if(item == my_tuple[0]):
item = my_tuple[1]
More generally, I would have a list of lists, and a list of tuples, and I would like to apply each of the tuples to each of the lists within the list of lists.
The more natural data structure for my_tuple is a dictionary. Consider something like this and use the .get() method:
>>> my_lists = [[3,2,2,3,4,1,3,4], [1,2,3,4,5,6]]
>>> my_tuple_list = [(3,5), (6, 7)]
>>> my_dict = dict(my_tuple_list)
>>> my_dict
{3: 5, 6: 7}
>>> my_lists = [[my_dict.get(x,x) for x in somelist] for somelist in my_lists]
>>> my_lists
[[5, 2, 2, 5, 4, 1, 5, 4], [1, 2, 5, 4, 5, 7]]
Per #Wooble's comment, your code will work if you enumerate.
list_of_lists = [[3,2,2,3,4,1,3,4], [1,3,5,3,4,6,3]]
list_of_tuples = [(3,5), (1,9)]
def tup_replace(mylist, mytuple):
for i, item in enumerate(mylist):
if item == mytuple[0]:
mylist[i] = mytuple[1]
return mylist
then you can just nest that some more to work on a list of list and list of tuples.
for mylist in list_of_lists:
for mytuple in list_of_tuples:
mylist = tup_replace(mylist, mytuple)
print mylist
That said, the dictionary approach is probably better.
Using if item == my_tuple[0], ... in a general case sounds like you are making a switch statement that you want to apply to each item in your list. Use a dictionary instead if you can. (Why isn't there a switch or case statement in python?)
Convert your list of tuples to a lookup dictionary (python's switch statement):
replacements = dict(my_tuples) #thanks to #julio
Then for a single list, reproduce the list with a comprehension, but replace each value with the new value from replacements if it exists:
replaced_list = [replacements.get(original,original) for original in my_list]
I guess there is a more efficient way to do it, but that's for a single list with a list of tuples. You say you also need to do it for a list of lists? Just nest that?
Could you explain more about where you are getting this data and why you need to do it?
If you are trying to replace every 3 in your list with 5, this will do:
[x == my_tuple[0] and my_tuple[1] or x for x in my_list]
If you want to do this, with more than one "translational" tuple, then I really suggest to use a dictionary instead:
trans = {3: 5, 4: 6}
[trans.get(x,x) for x in my_list]
And in the more general case where you have more than one list:
ll = [[3, 2, 3, 4], [5, 4, 3, 4]]
trans = {3: 5, 4: 6}
for i in range(len(ll)):
ll[i] = [trans.get(x,x) for x in ll[i]]
Supposing that you want to replace every old list in ll with the new one.