I'm working with lists that could have "ambiguous" values in certain places. The lists are small enough that implementing backtracking search seems silly. Currently, I'm representing ambiguous values in my lists with sub-lists containing possible values. For instance, the list:
[1, 2, [3,4]]
Could be either the list [1,2,3] or [1,2,4]. Lists may have multiple ambiguous values in them, though ambiguous elements may not themselves contain ambiguous elements. Given a list with ambiguous values in it, I'm trying to generate a list of all the possible lists that list could represent. The previous list should return [[1,2,3],[1,2,4]].
Is there an elegant way to do this? I tried to recursively build each list backwards and append to an empty list, but I can't quite wrap my brain around how to do it.
You can use itertools.product, but you'll have to modify your source list slightly:
>>> import itertools
>>> l = [[1], [2], [3, 4]]
>>> list(itertools.product(*l))
[(1, 2, 3), (1, 2, 4)]
Related
My goal
My question is about a list comprehension that does not puts elements in the resulting list as they are (which would results in a nested list), but extends the results into a flat list. So my question is not about flattening a nested list, but how to get a flat list while avoiding to make a nested list in the first place.
Example
Consider a have class instances with attributes that contains a list of integers:
class Foo:
def __init__(self, l):
self.l = l
foo_0 = Foo([1, 2, 3])
foo_1 = Foo([4, 5])
list_of_foos = [foo_0, foo_1]
Now I want to have a list of all integers in all instances of Foo. My best solution using extend is:
result = []
for f in list_of_foos:
result.extend(f.l)
As desired, result is now [1, 2, 3, 4, 5].
Is there something better? For example list comprehensions?
Since I expect list comprehension to be faster, I'm looking for pythonic way get the desired result with a list comprehension. My best approach is to get a list of lists ('nested list') and flatten this list again - which seems quirky:
result = [item for sublist in [f.l for f in list_of_foos] for item in sublist]
What functionaly I'm looking for
result = some_module.list_extends(f.l for f in list_of_foos)
Questions and Answers I read before
I was quite sure there is an answer to this problem, but during my search, I only found list.extend and list comprehension where the reason why a nested list occurs is different; and python list comprehensions; compressing a list of lists? where the answers are about avoiding the nested list, or how to flatten it.
You can use multiple fors in a single comprehension:
result = [
n
for foo in list_of_foos
for n in foo.l
]
Note that the order of fors is from the outside in -- same as if you wrote a nested for-loop:
for foo in list_of_foos:
for n in foo.l:
print(n)
If you want to combine multiple lists, as if they were all one list, I'd immediately think of itertools.chain. However, you have to access an attribute on each item, so we're also going to need operator.attrgetter. To get those together, I used map and itertools.chain.from_iterable()
https://docs.python.org/3/library/itertools.html#itertools.chain.from_iterable
from itertools import chain
from operator import attrgetter
class Foo:
def __init__(self, l):
self.l = l
foo_0 = Foo([1, 2, 3])
foo_1 = Foo([4, 5])
list_of_foos = [foo_0, foo_1]
for item in chain.from_iterable(map(attrgetter('l'), list_of_foos)):
print(item)
That demonstrates iterating through iterators with chain, as if they were one. If you don't specifically need to keep the list around, don't. But in case you do, here is the comprehension:
final = [item for item in chain.from_iterable(map(attrgetter('l'), list_of_foos))]
print(final)
[1, 2, 3, 4, 5]
In a list, you can make good use to + operator to concatenate two or more list together. It acts like an extend function to your list.
foo_0.l + foo_1.l
Out[7]: [1, 2, 3, 4, 5]
or you can use sum to perform this operation
sum([foo_0.l, foo_1.l], [])
Out[15]: [1, 2, 3, 4, 5]
In fact, it's in one of the post you have read ;)
Description:
Type A:
([1, 2, 3, 4, 5, 6, 7, 8], [1, 3, 4, 5, 6, 7, 8]) ➞ 2
Type B:
([True, True, False, False, True], [False, True, False, True]) ➞ True
Type C:
(["Jane", "is", "pretty", "ugly"], ["Jane", "is", "pretty"]) ➞ "ugly"
Type D:
(["different", "types", "5", 5, [5], (5,)], ["5", "different", [5], "types", 5]) ➞ (5,)
Progress:
For A/C, I would first turn them into two sets, then use set.difference() or "-" to find the difference. However, this method does not apply to B, but I can still use "Counter()" to get the repeated times of each element and then find out the difference.
Problem:
Now I am stuck with D.
I think whether it's "set.difference()" or "-" or "Counter()", they can work pretty well with lists consisted of the same data type, but D is consisted of string, integer, list, tuple, i.e., different data types, so how to deal with it in Python?
Next Step:
To me, the most straightforward way is to find a function that can at least count the types of these elements in D, so I can compare them first before I move on to the values of these elements. Is it a good way?
One brute-force way is to loop over items in the second list, removing the item from the first list:
a = ["different", "types", "5", 5, [5], (5,)]
b = ["5", "different", [5], "types", 5]
a_copy = a.copy()
for x in b:
a_copy.remove(x)
print(a_copy) # [(5,)]
It assumes that b is a sublist of a (up to permutation).
The main issue is that set elements and dictionary keys (a Counter uses a dict internally) must be hashable, but there is no guarantee that the elements in the two given lists are hashable. In your example D, there is the list [5], which is unhashable because it is mutable.
According to Python docs:
Most of Python’s immutable built-in objects are hashable; mutable containers (such as lists or dictionaries) are not; immutable containers (such as tuples and frozensets) are only hashable if their elements are hashable.
If there are no limitations on the items you can get in the input lists, this problem seems difficult without using a brute-force solution: compare each element in the first list to each element in the second.
If there's a match: remove the match from the second list.
If there's no match, add it to a third list that contains the list difference.
Make copies of the lists beforehand using .copy() if they need to be preserved.
At the end, move all the remaining elements in the second list to the third list and return it.
This algorithm is not very efficient: O(nm) time if the first list is size n and second list is size m. But it may be necessary given the lack of assumptions we can make about the input.
Alternatively, if we can assume that one-dimensional sub-lists are the only unhashable type we'll see in the input lists, we can circumvent the issue by converting the list into its hashable counterpart: a tuple. We can still use your Counter idea, but now we have to differentiate between (5,) and [5]. Here's my idea:
Instead of storing an element x in the Counter, we store the tuple (type(x), x).
The exception is if x is a list, in which case we store (type(x), tuple(x)).
(5,) stores as (<class 'tuple'>, (5,))
[5] stores as (<class 'list'>, (5,))
Now you can simply compare the Counters to find the difference. This is faster than the brute-force solution, but you'll have to keep adding exceptions as you allow more and more unhashable elements in the input lists (like set -> frozenset). With this method, at some point you'll have to lay down some restriction on what the inputs can contain.
I am looking for a method to generate all possible combinations of a list of lists, under the condition that there should only be "elementwise" combinations. Thus, if we have the lists [1,2] and [3,4], then the outcome [3,2] is allowed, but [4,2] is not allowed. With itertools.product(*lists) the last outcome is included.
Thus, I want the following output: [1,2], [3,2], [1,4], [3,4], and the following options should be 'skipped': [4,2], [1,3]. Note that the order is important! Thus I do not allow [2,3], just [3,2].
I know that I can check for this afterwards, but since I am generating many many lists in my code, I rather avoid this.
You could store the two lists in one list of lists and then after transposing the container list, use itertools.product() on it.
import itertools
original_list = [[1,2], [3,4]]
transposed_list = list(map(list, zip(*original_list)))
print(list(itertools.product(*transposed_list)))
Outputs:
[(1, 2), (1, 4), (3, 2), (3, 4)]
EDIT | Explaining how the list was transposed:
By definition, transposing is the process of exchanging places.
*original_list means [1,2] and [3,4]. The asterisk refers to the elements of the list, rather than the list as a whole
zip basically pairs values together 'element-wise':
e.g. with our original_list we have [1,2] and [3,4]. Calling zip on our elements will result in (1,3) and (2,4). The values were paired element-wise.
Note that the resulting pairs are not in list form.
map applies a function to every element in an input list. In our example,
we want to apply the (built-in) list function to our element-wise pairs - i.e. convert each tuple of pairs into a list. This is what turns (1,3) and (2,4) into [1,3] and [2,4]
Finally, convert our result from the mapping into a containing list, holding our element-wise pairs, producing [ [1,3], [2,4] ]
I want to make a list of items from the elements of tuples in a list such that those elements don't belong to some other list, and I know that each tuple contains one element from the list I don't want it to belong to and one element that's not in that list. For example, with
tuples = [(2,1), (1,4), (1,7), (3,10), (4,3)]
exclude = [1, 3]
I am looking to create the list
[2, 4, 7, 10]
This is easy enough to accomplish in a clumsy for loop, but it seems like there's a more pythonic way using some function or list comprehension. Any ideas?
Didn't actually understand the question. Assuming this may be you want
>>>list(set([j for i in tuples for j in i if not j in exclude]))
[2, 4, 10, 7]
Assuming your requirement is to convert list of tuples to a list and then getting unique elements in the list, exclusing the list exclude and then sorting them.
from itertools import chain
tuples_final = sorted(list(set(chain(*tuples))-set(exclude)))
You forgot a 4 in your example, the code will return :
>>>[num for tup in tuples for num in tup if num not in exclude]
[2, 4, 7, 10, 4]
If I have the following Python code
>>> x = []
>>> x = x + [1]
>>> x = x + [2]
>>> x = x + [3]
>>> x
[1, 2, 3]
Will x be guaranteed to always be [1,2,3], or are other orderings of the interim elements possible?
Yes, the order of elements in a python list is persistent.
In short, yes, the order is preserved. In long:
In general the following definitions will always apply to objects like lists:
A list is a collection of elements that can contain duplicate elements and has a defined order that generally does not change unless explicitly made to do so. stacks and queues are both types of lists that provide specific (often limited) behavior for adding and removing elements (stacks being LIFO, queues being FIFO). Lists are practical representations of, well, lists of things. A string can be thought of as a list of characters, as the order is important ("abc" != "bca") and duplicates in the content of the string are certainly permitted ("aaa" can exist and != "a").
A set is a collection of elements that cannot contain duplicates and has a non-definite order that may or may not change over time. Sets do not represent lists of things so much as they describe the extent of a certain selection of things. The internal structure of set, how its elements are stored relative to each other, is usually not meant to convey useful information. In some implementations, sets are always internally sorted; in others the ordering is simply undefined (usually depending on a hash function).
Collection is a generic term referring to any object used to store a (usually variable) number of other objects. Both lists and sets are a type of collection. Tuples and Arrays are normally not considered to be collections. Some languages consider maps (containers that describe associations between different objects) to be a type of collection as well.
This naming scheme holds true for all programming languages that I know of, including Python, C++, Java, C#, and Lisp (in which lists not keeping their order would be particularly catastrophic). If anyone knows of any where this is not the case, please just say so and I'll edit my answer. Note that specific implementations may use other names for these objects, such as vector in C++ and flex in ALGOL 68 (both lists; flex is technically just a re-sizable array).
If there is any confusion left in your case due to the specifics of how the + sign works here, just know that order is important for lists and unless there is very good reason to believe otherwise you can pretty much always safely assume that list operations preserve order. In this case, the + sign behaves much like it does for strings (which are really just lists of characters anyway): it takes the content of a list and places it behind the content of another.
If we have
list1 = [0, 1, 2, 3, 4]
list2 = [5, 6, 7, 8, 9]
Then
list1 + list2
Is the same as
[0, 1, 2, 3, 4] + [5, 6, 7, 8, 9]
Which evaluates to
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Much like
"abdcde" + "fghijk"
Produces
"abdcdefghijk"
You are confusing 'sets' and 'lists'. A set does not guarantee order, but lists do.
Sets are declared using curly brackets: {}. In contrast, lists are declared using square brackets: [].
mySet = {a, b, c, c}
Does not guarantee order, but list does:
myList = [a, b, c]
I suppose one thing that may be concerning you is whether or not the entries could change, so that the 2 becomes a different number, for instance. You can put your mind at ease here, because in Python, integers are immutable, meaning they cannot change after they are created.
Not everything in Python is immutable, though. For example, lists are mutable---they can change after being created. So for example, if you had a list of lists
>>> a = [[1], [2], [3]]
>>> a[0].append(7)
>>> a
[[1, 7], [2], [3]]
Here, I changed the first entry of a (I added 7 to it). One could imagine shuffling things around, and getting unexpected things here if you are not careful (and indeed, this does happen to everyone when they start programming in Python in some way or another; just search this site for "modifying a list while looping through it" to see dozens of examples).
It's also worth pointing out that x = x + [a] and x.append(a) are not the same thing. The second one mutates x, and the first one creates a new list and assigns it to x. To see the difference, try setting y = x before adding anything to x and trying each one, and look at the difference the two make to y.
Yes the list will remain as [1,2,3] unless you perform some other operation on it.
aList=[1,2,3]
i=0
for item in aList:
if i<2:
aList.remove(item)
i+=1
aList
[2]
The moral is when modifying a list in a loop driven by the list, takes two steps:
aList=[1,2,3]
i=0
for item in aList:
if i<2:
aList[i]="del"
i+=1
aList
['del', 'del', 3]
for i in range(2):
del aList[0]
aList
[3]
Yes lists and tuples are always ordered while dictionaries are not