Related
I have nested tuples in a list like
l = [(1, 'a', 'b'), (2, 'b', 'c'), (3, 'e', 'a')]
I want to know how many 'a' and 'b' in the list in total. So I currently use the following code to get the result.
amount_a_and_b = len([None for _, elem2, elem3 in l if elem2 == 'a' or elem3 == 'b'])
But I got amount_a_and_b = 1, so how to get the right answer?
Also, is there a more elegant way (less code or higher performance or using builtins) to do this?
I'd flatten the list with itertools.chain.from_iterable() and pass it to a collections.Counter() object:
from collections import Counter
from itertools import chain
counts = Counter(chain.from_iterable(l))
amount_a_and_b = counts['a'] + counts['b']
Or use sum() to count how many times a value appears in the flattened sequence:
from itertools import chain
amount_a_and_b = sum(1 for v in chain.from_iterable(l) if v in {'a', 'b'})
The two approaches are pretty much comparable in speed on Python 3.5.1 on my Macbook Pro (OS X 10.11):
>>> from timeit import timeit
>>> from collections import Counter
>>> from itertools import chain
>>> l = [(1, 'a', 'b'), (2, 'b', 'c'), (3, 'e', 'a')] * 1000 # make it interesting
>>> def counter():
... counts = Counter(chain.from_iterable(l))
... counts['a'] + counts['b']
...
>>> def summing():
... sum(1 for v in chain.from_iterable(l) if v in {'a', 'b'})
...
>>> timeit(counter, number=1000)
0.5640139860006457
>>> timeit(summing, number=1000)
0.6066895100011607
You want to avoid putting data in a datastructure. The [...] syntax constructs a new list and fills it with the content you put in ... , after which the length of the array is taken and the array is never used. If the list if very large, this uses a lot of memory, and it is inelegant in general. You can also use iterators to loop over the existing data structure, e.g., like so:
sum(sum(c in ('a', 'b') for c in t) for t in l)
The c in ('a', 'b') predicate is a bool which evaluates to a 0 or 1 when cast to an int, causing the sum() to only count the tuple entry if the predicate evaluates to True.
Just for fun, functional method using reduce:
>>> l = [(1, 'a', 'b'), (2, 'b', 'c'), (3, 'e', 'a')]
>>> from functools import reduce
>>> reduce(lambda x, y: (1 if 'a' in y else 0) + (1 if 'b' in y else 0) + x, l, 0)
4
You can iterate over both the list and the sub-lists in one list comprehension:
len([i for sub_list in l for i in sub_list if i in ("a", "b")])
I think that's fairly concise.
To avoid creating a temporary list, you could use a generator expression to create a sequence of 1s and pass that to sum:
sum(1 for sub_list in l for i in sub_list if i in ("a", "b"))
Although this question already has an accepted answer, just wondering why all of them as so complex. I would think that this would suffice.
>>> l = [(1, 'a', 'b'), (2, 'b', 'c'), (3, 'e', 'a')]
>>> total = sum(tup.count('a') + tup.count('b') for tup in l)
>>> total
4
Or
>>> total = sum(1 for tup in l for v in tup if v in {'a', 'b'})
I want to create a list of possible combinations of a list of lists (example will explain better)
list=[[a,b],[c,d],[f]]
The result should be
acf
adf
bcf
bdf
The length of the list can vary, and the length of the lists within the variable list can also vary. How would I make this/these loop(s) programmatically? (preferably explained in Python or pseudo-language)
That's what itertools.product is for:
>>> lst = ['ab','cd','f']
>>> from itertools import product
>>> list(product(*lst))
[('a', 'c', 'f'), ('a', 'd', 'f'), ('b', 'c', 'f'), ('b', 'd', 'f')]
import itertools
list=[['a','b'],['c','d'],['f']]
for comb in itertools.product(*list):
print ''.join(comb)
You can do it recursively:
def printCombos(arrays, combo):
if len(arrays) == 0:
print combo
else:
for i in arrays[0]:
combo.append(i)
printCombos(arrays[1:], combo)
combo.pop()
l=[['a','b'],['c','d'],['f']]
printCombos(l, [])
curlist = []
for firstobj in listoflists[0]:
for secondobj in listoflists[1]:
for lastobj in listoflists[2]:
curlist.append(firstobj)
curlist.append(secondobj)
curlist.append(lastobj)
print ','.join(curlist)
Currently I have a file with 6 rows of numbers and each row containing 9 numbers. The point is to test each row of numbers in the file if it completes a magic square. So for example, say a row of numbers from the file is 4 3 8 9 5 1 2 7 6. The first three numbers need to be the first row in a matrix. The next three numbers need to be the second row, and same for the third.
Therefore you would need to end up with a matrix of:
[['4','3','8'],['9','5','1'],['2','7','6']]
I need to test the matrix to see if it is a valid magic square (Rows add up to 15, columns add to 15, and diagonals add to 15).
My code is currently:
def readfile(fname):
"""Return a list of lines from the file"""
f = open(fname, 'r')
lines = f.read()
lines = lines.split()
f.close()
return lines
def assignValues(lines):
magicSquare = []
rows = 3
columns = 3
for row in range(rows):
magicSquare.append([0] * columns)
for row in range(len(magicSquare)):
for column in range(len(magicSquare[row])):
magicSquare[row][column] = lines[column]
return magicSquare
def main():
lines = readfile(input_fname)
matrix = assignValues(lines)
print(matrix)
Whenever I run my code to test it, I'm getting:
[['4', '3', '8'], ['4', '3', '8'], ['4', '3', '8']]
So as you can see I am only getting the first 3 numbers into my matrix.
Finally, my question is how would I go by continuing my matrix with the following 6 numbers of the line of numbers? I'm not sure if it is something I can do in my loop, or if I am splitting my lines wrong, or am I completely on the wrong track?
Thanks.
To test if each row in your input file contains magic square data you need to re-organize the code slightly. I've used a different technique to Francis to fill the matrix. It might be a bit harder to understand how zip(*[iter(seq)] * size) works, but it's a very useful pattern. Please let me know if you need an explanation for it.
My code uses a list of tuples for the matrix, rather than a list of lists, but tuples are more suitable here anyway, since the data in the matrix doesn't need to be modified. Also, I convert the input data from str into int, since you need to do arithmetic on the numbers to test if matrix is a magic square.
#! /usr/bin/env python
def make_square(seq, size):
return zip(*[iter(seq)] * size)
def main():
fname = 'mydata'
size = 3
with open(fname, 'r') as f:
for line in f:
nums = [int(s) for s in line.split()]
matrix = make_square(nums, size)
print matrix
#Now call the function to test if the data in matrix
#really is a magic square.
#test_square(matrix)
if __name__ == '__main__':
main()
Here's a modified version of make_square() that returns a list of lists instead of a list of tuples, but please bear in mind that a list of tuples is actually better than a list of lists if you don't need the mutability that lists give you.
def make_square(seq, size):
square = zip(*[iter(seq)] * size)
return [list(t) for t in square]
I suppose I should mention that there's actually only one possible 3 x 3 magic square that uses all the numbers from 1 to 9, not counting rotations and reflections. But I guess there's no harm in doing a brute-force demonstration of that fact. :)
Also, I have Python code that I wrote years ago (when I was first learning Python) which generates magic squares of size n x n for odd n >= 5. Let me know if you'd like to see it.
zip and iterator objects
Here's some code that briefly illustrates what the zip() and iter() functions do.
''' Fun with zip '''
numbers = [1, 2, 3, 4, 5, 6]
letters = ['a', 'b', 'c', 'd', 'e', 'f']
#Using zip to create a list of tuples containing pairs of elements of numbers & letters
print zip(numbers, letters)
#zip works on other iterable objects, including strings
print zip(range(1, 7), 'abcdef')
#zip can handle more than 2 iterables
print zip('abc', 'def', 'ghi', 'jkl')
#zip can be used in a for loop to process two (or more) iterables simultaneously
for n, l in zip(numbers, letters):
print n, l
#Using zip in a list comprehension to make a list of lists
print [[l, n] for n, l in zip(numbers, letters)]
#zip stops if one of the iterables runs out of elements
print [[n, l] for n, l in zip((1, 2), letters)]
print [(n, l) for n, l in zip((3, 4), letters)]
#Turning an iterable into an iterator object using the iter function
iletters = iter(letters)
#When we take some elements from an iterator object it remembers where it's up to
#so when we take more elements from it, it continues from where it left off.
print [[n, l] for n, l in zip((1, 2, 3), iletters)]
print [(n, l) for n, l in zip((4, 5), iletters)]
#This list will just contain a single tuple because there's only 1 element left in iletters
print [(n, l) for n, l in zip((6, 7), iletters)]
#Rebuild the iletters iterator object
iletters = iter('abcdefghijkl')
#See what happens when we zip multiple copies of the same iterator object.
print zip(iletters, iletters, iletters)
#It can be convenient to put multiple copies of an iterator object into a list
iletters = iter('abcdefghijkl')
gang = [iletters] * 3
#The gang consists of 3 references to the same iterator object
print gang
#We can pass each iterator in the gang to zip as a separate argument
#by using the "splat" syntax
print zip(*gang)
#A more compact way of doing the same thing:
print zip(* [iter('abcdefghijkl')]*3)
Here's the same code running in the interactive interpreter so you can easily see the output of each statement.
>>> numbers = [1, 2, 3, 4, 5, 6]
>>> letters = ['a', 'b', 'c', 'd', 'e', 'f']
>>>
>>> #Using zip to create a list of tuples containing pairs of elements of numbers & letters
... print zip(numbers, letters)
[(1, 'a'), (2, 'b'), (3, 'c'), (4, 'd'), (5, 'e'), (6, 'f')]
>>>
>>> #zip works on other iterable objects, including strings
... print zip(range(1, 7), 'abcdef')
[(1, 'a'), (2, 'b'), (3, 'c'), (4, 'd'), (5, 'e'), (6, 'f')]
>>>
>>> #zip can handle more than 2 iterables
... print zip('abc', 'def', 'ghi', 'jkl')
[('a', 'd', 'g', 'j'), ('b', 'e', 'h', 'k'), ('c', 'f', 'i', 'l')]
>>>
>>> #zip can be used in a for loop to process two (or more) iterables simultaneously
... for n, l in zip(numbers, letters):
... print n, l
...
1 a
2 b
3 c
4 d
5 e
6 f
>>> #Using zip in a list comprehension to make a list of lists
... print [[l, n] for n, l in zip(numbers, letters)]
[['a', 1], ['b', 2], ['c', 3], ['d', 4], ['e', 5], ['f', 6]]
>>>
>>> #zip stops if one of the iterables runs out of elements
... print [[n, l] for n, l in zip((1, 2), letters)]
[[1, 'a'], [2, 'b']]
>>> print [(n, l) for n, l in zip((3, 4), letters)]
[(3, 'a'), (4, 'b')]
>>>
>>> #Turning an iterable into an iterator object using using the iter function
... iletters = iter(letters)
>>>
>>> #When we take some elements from an iterator object it remembers where it's up to
... #so when we take more elements from it, it continues from where it left off.
... print [[n, l] for n, l in zip((1, 2, 3), iletters)]
[[1, 'a'], [2, 'b'], [3, 'c']]
>>> print [(n, l) for n, l in zip((4, 5), iletters)]
[(4, 'd'), (5, 'e')]
>>>
>>> #This list will just contain a single tuple because there's only 1 element left in iletters
... print [(n, l) for n, l in zip((6, 7), iletters)]
[(6, 'f')]
>>>
>>> #Rebuild the iletters iterator object
... iletters = iter('abcdefghijkl')
>>>
>>> #See what happens when we zip multiple copies of the same iterator object.
... print zip(iletters, iletters, iletters)
[('a', 'b', 'c'), ('d', 'e', 'f'), ('g', 'h', 'i'), ('j', 'k', 'l')]
>>>
>>> #It can be convenient to put multiple copies of an iterator object into a list
... iletters = iter('abcdefghijkl')
>>> gang = [iletters] * 3
>>>
>>> #The gang consists of 3 references to the same iterator object
... print gang
[<iterator object at 0xb737eb8c>, <iterator object at 0xb737eb8c>, <iterator object at 0xb737eb8c>]
>>>
>>> #We can pass each iterator in the gang to zip as a separate argument
... #by using the "splat" syntax
... print zip(*gang)
[('a', 'b', 'c'), ('d', 'e', 'f'), ('g', 'h', 'i'), ('j', 'k', 'l')]
>>>
>>> #A more compact way of doing the same thing:
... print zip(* [iter('abcdefghijkl')]*3)
[('a', 'b', 'c'), ('d', 'e', 'f'), ('g', 'h', 'i'), ('j', 'k', 'l')]
>>>
it only gets the first 3 column always because
magicSquare[row][column] = lines[column]
thus
def assignValues(lines):
magicSquare = []
rows = 3
columns = 3
for row in range(rows):
magicSquare.append([0] * columns)
for line in range((sizeof(lines)/9)) #since the input is already split this means that the size of 'lines' divided by 9 is equal to the number of rows of numbers
for row in range(len(magicSquare)):
for column in range(len(magicSquare[row])):
magicSquare[row][column] = lines[(9*line)+(3*row)+column]
return magicSquare
note that (3*row)+column will move to it 3 columns to the right every iteration
and that (9*line)+(3*row)+column will move to it 9 columns (a whole row) to the right every iteration
once you get this you are now ready to process in finding out for the magic square
def testMagicSquare(matrix):
rows = 3
columns = 3
for a in len(matrix)
test1 = 0
test2 = 0
test3 = 0
for b in range(3)
if(sum(matrix[a][b])==15) test1=1 #flag true if whole row is 15 but turns false if a row is not 15
else test1=0
if((matrix[a][0][b]+matrix[a][1][b]+matrix[a][2][b])==15) test2=1 #flag true if column is 15 but turns false if a column is not 15
else test2=0
if(((matrix[a][0][0]+matrix[a][1][1]+matrix[a][2][2])==15) and
((matrix[a][0][2]+matrix[a][1][1]+matrix[a][2][0])==15)) test3=1 #flag true if diagonal is 15 but turns false if diagonal is not 15
else test3=0
if(test1>0 and test2>0 and test3>0) println('line ' + a + ' is a magic square')
else println('line ' + a + ' is not a magic square')
My program is not doing a great job. In a loop, data from each processor (list of tuple) are gathered into the master processor that needs to clean it by removing similar element.
I found a lot of interesting clue on internet and especially in this site about union of list. However, i have not managed to apply it to my problem.
My aim is to get rid of tuple whose its two last element are similar to another tuple in the list . for example:
list1=[[a,b,c],[d,e,f],[g,h,i]]
list2=[[b,b,c],[d,e,a],[k,h,i]]
the result should be:
final=[[a,b,c],[d,e,f],[g,h,i],[d,e,a]]
Right now I'm using loops and break but I'm hoping to make this process faster.
here is what my code looks like (result and temp are the lists I want to get union from)
on python2.6.
for k in xrange(len(temp)):
u=0
#index= next((j for j in xrange(lenres) if temp[k][1:3] == result[j][1:3]),None)
for j in xrange(len(result)):
if temp[k][1:3] == result[j][1:3]:
u=1
break
if u==0:
#if index is None:
result.append([temp[k][0],temp[k][1],temp[k][2]])
Thanks for your help
Herve
Below is our uniques function. It takes arguments l (list) and f (function), returns list with duplicates removed (in the same order). Duplicates are defined by: b is duplicate of a iff f(b) == f(a).
def uniques(l, f = lambda x: x):
return [x for i, x in enumerate(l) if f(x) not in [f(y) for y in l[:i]]]
We define lastTwo as follows:
lastTwo = lambda x: x[-2:]
For your problem we use it as follows:
>>> list1
[('a', 'b', 'c'), ('d', 'e', 'f'), ('g', 'h', 'i')]
>>> list2
[('b', 'b', 'c'), ('d', 'e', 'a'), ('k', 'h', 'i')]
>>> uniques(list1+list2, lastTwo)
[('a', 'b', 'c'), ('d', 'e', 'f'), ('g', 'h', 'i'), ('d', 'e', 'a')]
If the usecase you describe comes up a lot you may want to define
def hervesMerge(l1, l2):
return uniques(l1+l2, lambda x: x[-2:])
Identity is our default f but it can be anything (so long as it is defined for all elements of the list, since they can be of any type).
f can be sum of a list, odd elements of a list, prime factors of an integer, anything. (Just remember that if its injective theres no point! Add by constant, linear functions, etc will work no differently than identity bc its f(x) == f(y) w/ x != y that makes the difference)
>>> list1
[(1, 2, 3, 4), (2, 5), (6, 2, 2), (3, 4), (8, 3), (1, 1, 1, 1, 1, 1, 1, 1, 1, 1)]
>>> uniques(list1, sum)
[(1, 2, 3, 4), (2, 5), (8, 3)]
>>> uniques(list1, lambda x: reduce(operator.mul, x)) #product
[(1, 2, 3, 4), (2, 5), (3, 4), (1, 1, 1, 1, 1, 1, 1, 1, 1, 1)]
>>> uniques([1,2,3,4,1,2]) #defaults to identity
[1, 2, 3, 4]
You seemed concerned about speed, but my answer really focused on shortness/flexibility without significant (or any?) speed improvment. For bigger lists where speed is z concern, you want to take advantage of hashable checks and the fact that list1 and list2 are known to have no duplicates
>>> s = frozenset(i[-2:] for i in list1)
>>> ans = list(list1) #copy list1
>>> for i in list2:
if i[-2:] not in s: ans.append(i)
>>> ans
[('a', 'b', 'c'), ('d', 'e', 'f'), ('g', 'h', 'i'), ('d', 'e', 'a')]
OR allowing disordering
>>> d = dict()
>>> for i in list2 + list1:
d[i[-2:]] = i
>>> d.values()
[('d', 'e', 'f'), ('a', 'b', 'c'), ('g', 'h', 'i'), ('d', 'e', 'a')]
--Edit--
You should always be able to avoid un-pythonic looping like you post in your question. Here is your exact code with the loops changed:
for k in temp:
u=0
for j in result:
if k[1:3] == j[1:3]:
u=1
break
if u==0:
#if index is None:
result.append([k[0],k[1],k[2]]) // k
result and temp are iterable, and for anything iterable you can put it directly in the for loop without eanges. If for some reason you explicitly need the index (this is not such a case, but I have one above) you can use enumerate.
Here's a simple solution using a set:
list1=[('a','b','c'),('d','e','f'),('g','h','i')]
list2=[('b','b','c'),('d','e','a'),('k','h','i')]
set1 = set([A[1:3] for A in list1])
final = list1 + [A for A in list2 if A[1:3] not in set1]
However, if your list1 and list2 aren't actually made of tuples, then you will have to put tuple() around A[1:3].
This question already has answers here:
Operation on every pair of element in a list
(5 answers)
Closed 8 months ago.
I have a list L = [a, b, c] and I want to generate a list of tuples :
[(a,a), (a,b), (a,c), (b,a), (b,b), (b,c)...]
I tried doing L * L but it didn't work. Can someone tell me how to get this in python.
You can do it with a list comprehension:
[ (x,y) for x in L for y in L]
edit
You can also use itertools.product as others have suggested, but only if you are using 2.6 onwards. The list comprehension will work will all versions of Python from 2.0. If you do use itertools.product bear in mind that it returns a generator instead of a list, so you may need to convert it (depending on what you want to do with it).
The itertools module contains a number of helpful functions for this sort of thing. It looks like you may be looking for product:
>>> import itertools
>>> L = [1,2,3]
>>> itertools.product(L,L)
<itertools.product object at 0x83788>
>>> list(_)
[(1, 1), (1, 2), (1, 3), (2, 1), (2, 2), (2, 3), (3, 1), (3, 2), (3, 3)]
Take a look at the itertools module, which provides a product member.
L =[1,2,3]
import itertools
res = list(itertools.product(L,L))
print(res)
Gives:
[(1,1),(1,2),(1,3),(2,1), .... and so on]
Two main alternatives:
>>> L = ['a', 'b', 'c']
>>> import itertools
>>> list(itertools.product(L, L))
[('a', 'a'), ('a', 'b'), ('a', 'c'), ('b', 'a'), ('b', 'b'), ('b', 'c'), ('c', 'a'), ('c', 'b'), ('c', 'c')]
>>> [(one, two) for one in L for two in L]
[('a', 'a'), ('a', 'b'), ('a', 'c'), ('b', 'a'), ('b', 'b'), ('b', 'c'), ('c', 'a'), ('c', 'b'), ('c', 'c')]
>>>
the former one needs Python 2.6 or better -- the latter works in just about any Python version you might be tied to.
x = [a,b,c]
y = []
for item in x:
for item2 in x:
y.append((item, item2))
Maybe not the Pythonic way but working
Ok I tried :
L2 = [(x,y) for x in L for x in L] and this got L square.
Is this the best pythonic way to do this? I would expect L * L to work in python.
The most old fashioned way to do it would be:
def perm(L):
result = []
for i in L:
for j in L:
result.append((i,j))
return result
This has a runtime of O(n^2) and is therefore quite slow, but you could consider it to be "vintage" style code.