Is it possible to define a recursive list comprehension in Python?
Possibly a simplistic example, but something along the lines of:
nums = [1, 1, 2, 2, 3, 3, 4, 4]
willThisWork = [x for x in nums if x not in self] # self being the current comprehension
Is anything like this possible?
No, there's no (documented, solid, stable, ...;-) way to refer to "the current comprehension". You could just use a loop:
res = []
for x in nums:
if x not in res:
res.append(x)
of course this is very costly (O(N squared)), so you can optimize it with an auxiliary set (I'm assuming that keeping the order of items in res congruent to that of the items in nums, otherwise set(nums) would do you;-)...:
res = []
aux = set()
for x in nums:
if x not in aux:
res.append(x)
aux.add(x)
this is enormously faster for very long lists (O(N) instead of N squared).
Edit: in Python 2.5 or 2.6, vars()['_[1]'] might actually work in the role you want for self (for a non-nested listcomp)... which is why I qualified my statement by clarifying there's no documented, solid, stable way to access "the list being built up" -- that peculiar, undocumented "name" '_[1]' (deliberately chosen not to be a valid identifier;-) is the apex of "implementation artifacts" and any code relying on it deserves to be put out of its misery;-).
Starting Python 3.8, and the introduction of assignment expressions (PEP 572) (:= operator), which gives the possibility to name the result of an expression, we could reference items already seen by updating a variable within the list comprehension:
# items = [1, 1, 2, 2, 3, 3, 4, 4]
acc = []; [acc := acc + [x] for x in items if x not in acc]
# acc = [1, 2, 3, 4]
This:
Initializes a list acc which symbolizes the running list of elements already seen
For each item, this checks if it's already part of the acc list; and if not:
appends the item to acc (acc := acc + [x]) via an assignment expression
and at the same time uses the new value of acc as the mapped value for this item
Actually you can! This example with an explanation hopefully will illustrate how.
define recursive example to get a number only when it is 5 or more and if it isn't, increment it and call the 'check' function again. Repeat this process until it reaches 5 at which point return 5.
print [ (lambda f,v: v >= 5 and v or f(f,v+1))(lambda g,i: i >= 5 and i or g(g,i+1),i) for i in [1,2,3,4,5,6] ]
result:
[5, 5, 5, 5, 5, 6]
>>>
essentially the two anonymous functions interact in this way:
let f(g,x) = {
expression, terminal condition
g(g,x), non-terminal condition
}
let g(f,x) = {
expression, terminal condition
f(f,x), non-terminal condition
}
make g,f the 'same' function except that in one or both add a clause where the parameter is modified so as to cause the terminal condition to be reached and then go
f(g,x) in this way g becomes a copy of f making it like:
f(g,x) = {
expression, terminal condition
{
expression, terminal condition,
g(g,x), non-terminal codition
}, non-terminal condition
}
You need to do this because you can't access the the anonymous function itself upon being executed.
i.e
(lambda f,v: somehow call the function again inside itself )(_,_)
so in this example let A = the first function and B the second. We call A passing B as f and i as v. Now as B is essentially a copy of A and it's a parameter that has been passed you can now call B which is like calling A.
This generates the factorials in a list
print [ (lambda f,v: v == 0 and 1 or v*f(f,v-1))(lambda g,i: i == 0 and 1 or i*g(g,i-1),i) for i in [1,2,3,5,6,7] ]
[1, 2, 6, 120, 720, 5040]
>>>
Not sure if this is what you want, but you can write nested list comprehensions:
xs = [[i for i in range(1,10) if i % j == 0] for j in range(2,5)]
assert xs == [[2, 4, 6, 8], [3, 6, 9], [4, 8]]
From your code example, you seem to want to simply eliminate duplicates, which you can do with sets:
xs = sorted(set([1, 1, 2, 2, 3, 3, 4, 4]))
assert xs == [1, 2, 3, 4]
no. it won't work, there is no self to refer to while list comprehension is being executed.
And the main reason of course is that list comprehensions where not designed for this use.
No.
But it looks like you are trying to make a list of the unique elements in nums.
You could use a set:
unique_items = set(nums)
Note that items in nums need to be hashable.
You can also do the following. Which is a close as I can get to your original idea. But this is not as efficient as creating a set.
unique_items = []
for i in nums:
if i not in unique_items:
unique_items.append(i)
Do this:
nums = [1, 1, 2, 2, 3, 3, 4, 4]
set_of_nums = set(nums)
unique_num_list = list(set_of_nums)
or even this:
unique_num_list = sorted(set_of_nums)
Related
This question already has answers here:
Sort a part of a list in place
(3 answers)
Closed 3 years ago.
Suppose I have a list [2, 4, 1, 3, 5].
I want to sort the list just from index 1 to the end, which gives me [2, 1, 3, 4, 5]
How can I do it in Python?
(No extra spaces would be appreciated)
TL;DR:
Use sorted with a slicing assignment to keep the original list object without creating a new one:
l = [2, 4, 1, 3, 5]
l[1:] = sorted(l[1:])
print(l)
Output:
[2, 1, 3, 4, 5]
Longer Answer:
After the list is created, we will make a slicing assignment:
l[1:] =
Now you might be wondering what does [1:], it is slicing the list and starts from the second index, so the first index will be dropped. Python's indexing starts from zero, : means get everything after the index before, but if it was [1:3] it will only get values that are in between the indexes 1 and 3, let's say your list is:
l = [1, 2, 3, 4, 5]
If you use:
print(l[1:])
It will result in:
[2, 3, 4, 5]
And if you use:
print(l[1:3])
It will result in:
[2, 3]
About slicing, read more here if you want to.
And after slicing we have an equal sign =, that just simply changes what's before the = sign to what's after the = sign, so in this case, we use l[1:], and that gives [2, 3, 4, 5], it will change that to whatever is after the = sign.
If you use:
l[1:] = [100, 200, 300, 400]
print(l)
It will result in:
[1, 100, 200, 300, 400]
To learn more about it check out this.
After that, we got sorted, which is default builtin function, it simple sorts the list from small to big, let's say we have the below list:
l = [3, 2, 1, 4]
If you use:
print(sorted(l))
It will result in:
[1, 2, 3, 4]
To learn more about it check this.
After that we come back to our first topic about slicing, with l[1:], but from here you know that it isn't only used for assignments, you can apply functions to it and deal with it, like here we use sorted.
Maybe temporarily put something there that's smaller than the rest? Should be faster than the other solutions. And gets as close to your "No extra spaces" wish as you can get when using sort or sorted.
>>> tmp = l[0]
>>> l[0] = float('-inf')
>>> l.sort()
>>> l[0] = tmp
>>> l
[2, 1, 3, 4, 5]
Benchmarks
For the example list, 1,000,000 iterations (and mine of course preparing that special value only once):
sort_u10 0.8149 seconds
sort_chris 0.8569 seconds
sort_heap 0.7550 seconds
sort_heap2 0.5982 seconds # using -1 instead of -inf
For 50,000 lists like [int(x) for x in os.urandom(100)]:
sort_u10 0.4778 seconds
sort_chris 0.4786 seconds
sort_heap 0.8106 seconds
sort_heap2 0.4437 seconds # using -1 instead of -inf
Benchmark code:
import timeit, os
def sort_u10(l):
l[1:] = sorted(l[1:])
def sort_chris(l):
l = l[:1] + sorted(l[1:])
def sort_heap(l, smallest=float('-inf')):
tmp = l[0]
l[0] = smallest
l.sort()
l[0] = tmp
def sort_heap2(l):
tmp = l[0]
l[0] = -1
l.sort()
l[0] = tmp
for _ in range(3):
for sort in sort_u10, sort_chris, sort_heap, sort_heap2, sort_rev:
number, repeat = 1_000_000, 5
data = iter([[2, 4, 1, 3, 5] for _ in range(number * repeat)])
# number, repeat = 50_000, 5
# data = iter([[int(x) for x in os.urandom(100)] for _ in range(number * repeat)])
t = timeit.repeat(lambda: sort(next(data)), number=number, repeat=repeat)
print('%10s %.4f seconds' % (sort.__name__, min(t)))
print()
Use sorted with slicing:
l[:1] + sorted(l[1:])
Output:
[2, 1, 3, 4, 5]
For the special case that you actually have, according to our comments:
Q: I'm curious: Why do you want this? – Heap Overflow
A: I'm trying to make a next_permutation() in python – nwice13
Q: Do you really need to sort for that, though? Not just reverse? – Heap Overflow
A: Yup, reverse is ok, but I just curious to ask about sorting this way. – nwice13
I'd do that like this:
l[1:] = l[:0:-1]
You can define your own function in python using slicing and sorted and this function (your custom function) should take start and end index of the list.
Since list is mutable in python, I have written the function in such a way it doesn't modify the list passed. Feel free to modify the function. You can modify the list passed to this function to save memory if required.
def sortedList(li, start=0, end=None):
if end is None:
end = len(li)
fi = []
fi[:start] = li[:start]
fi[start:end] = sorted(li[start:end])
return fi
li = [2, 1, 4, 3, 0]
print(li)
print(sortedList(li, 1))
Output:
[2, 1, 4, 3, 0]
[2, 0, 1, 3, 4]
I'm making a program that basically calculates the missing values (x in this example) in multiple lists.
These are the lists:
L11=[1,3,5,'x',8,10]
L12=['x',3,3,'x',6,0]
L21=[6,1,1,9,2,2]
L22=[1,1,1,'x','x','x']
For example, I'm using this code block to find the x values in L22:
#How to find x:
#1--> a= calculate the sum of integers in the list
#2--> b=calculate the average of them
#3--> all values of x inside the list equal b
a22=L22.count('x')
for i in range(len(L22)):
if L22[i]=='x':
x_L22=round((sum([int(k) for k in L22 if type(k)==int]))/(len(L22)-a22))
So we find x_L22=1 and the new L22 is:
x_L22=1
L22=[1,1,1,1,1,1]
Now here is my question, I want to repeat this steps for all other lists without writing the same code. Is this possible?
Other answers focus on extracting your current code to a generic function which is useful but isn't neither sufficient nor necessary to apply the same piece of code on multiple input.
The only thing you need is to loop over your pieces of data :
L11=[1,3,5,'x',8,10]
L12=['x',3,3,'x',6,0]
L21=[6,1,1,9,2,2]
L22=[1,1,1,'x','x','x']
inputs = ( L11, L12, L21, L22 )
for input in inputs :
# your 4 previous lines on code, modified to work
# on the generic "input" instead of the specific "L22"
a=input.count('x')
for i in range(len(input)):
if input[i]=='x':
x=round((sum([int(k) for k in input if type(k)==int]))/(len(input)-a))
# or if you've extracted the above code to a function,
# just call it instead of using the above 4 lines of code.
try putting it in a function like this:
def list_foo(list_):
counted=list_.count('x')
for i in range(len(list_)):
if list_[i]=='x':
total=round((sum([int(k) for k in list_ if type(k)==int])) \
/(len(list_)-counted))
return total
use it in your main loop
x_L22 = list_foo(L22)
or x_L11 = list_foo(L11)
This is an excellent use case for functions in Python
def get_filled_list(list_of_numbers):
#How to find x:
#1--> a= calculate the sum of integers in the list
#2--> b=calculate the average of them
#3--> all values of x inside the list equal b
new_list=list_of_numbers.count('x')
for i in range(len(list_of_numbers)):
if list_of_numbers[i]=='x':
list_of_numbers = round(
(sum([int(k)
for k in list_of_numbers if type(k)==int]))/
(len(list_of_numbers)-new_list)
)
A11 = get_filled_list(L11)
# ,..
I'd write a function that receives a list as an input and returns the same list with the 'x' value replaced with a new value:
def calculateList(l):
nrX=l.count('x')
newList = []
for elem in l:
if elem == 'x':
x = int(round((sum([int(k) for k in l if type(k)==int]))/(len(l)-nrX)))
newList.append(x)
else:
newList.append(elem)
return newList
You can then call this function on all the list you have:
newL = calculateList(L22)
print(newL)
Output is:
[1, 1, 1, 1, 1, 1]
Or if you prefer you can create a list containing all the lists you want to evaluate:
allLists = [L11, L12, L21, L22]
And then you iterate over this list:
for l in allLists:
newL = calculateList(l)
print(newL)
Output is:
[1, 3, 5, 5, 8, 10]
[3, 3, 3, 3, 6, 0]
[6, 1, 1, 9, 2, 2]
[1, 1, 1, 1, 1, 1]
I have this problem on writing a python function which takes a bit list as input and prints the items represented by this bit list.
so the question is on Knapsack and it is a relatively simple and straightforward one as I'm new to the python language too.
so technically the items can be named in a list [1,2,3,4] which corresponds to Type 1, Type 2, Type 3 and etc but we won't be needing the "type". the problem is, i represented the solution in a bit list [0,1,1,1] where 0 means not taken and 1 means taken. in another words, item of type 1 is not taken but the rest are taken, as represented in the bit list i wrote.
now we are required to write a python function which takes the bit list as input and prints the item corresponding to it in which in this case i need the function to print out [2,3,4] leaving out the 1 since it is 0 by bit list. any help on this? it is a 2 mark question but i still couldn't figure it out.
def printItems(l):
for x in range(len(l)):
if x == 0:
return False
elif x == 1:
return l
i tried something like that but it is wrong. much appreciated for any help.
You can do this with the zip function that takes two tiers Lee and returns them in pairs:
for bit_item, item in zip(bit_list, item_list):
if bit_item:
print item
Or if you need a list rather than printing them, you can use a list comprehension:
[item for bit_item, item in zip(bit_list, item_list) if bit_item]
You can use itertools.compress for a quick solution:
>>> import itertools
>>> list(itertools.compress(itertools.count(1), [0, 1, 1, 1]))
[2, 3, 4]
The reason your solution doesn't work is because you are using return in your function, where you need to use print, and make sure you are iterating over your list correctly. In this case, enumerate simplifies things, but there are many similar approaches that would work:
>>> def print_items(l):
... for i,b in enumerate(l,1):
... if b:
... print(i)
...
>>> print_items([0,1,1,1])
2
3
4
>>>
You may do it using list comprehension with enumerate() as:
>>> my_list = [0, 1, 1, 1]
>>> taken_list = [i for i, item in enumerate(my_list, 1) if item]
>>> taken_list # by default start with 0 ^
[2, 3, 4]
Alternatively, in case you do not need any in-built function and want to create your own function, you may modify your code as:
def printItems(l):
new_list = []
for x in range(len(l)):
if l[x] == 1:
new_list.append(x+1) # "x+1" because index starts with `0` and you need position
return new_list
Sample run:
>>> printItems([0, 1, 1, 1])
[2, 3, 4]
I was trying to execute a for loop like:
a = [1,2,3,4,5,6,7]
for i in range(0, len(a), 1):
if a[i] == 4:
a.remove(a[i])
I end up having an index error since the length of the list becomes shorter but the iterator i does not become aware.
So, my question is, how can something like that be coded? Can the range of i be updated in each iterations of the loop based on current array condition?
For the .pop() that you mention for example you can use a list comprehension to create a second list or even modify the original one in place. Like so:
alist = [1, 2, 3, 4, 1, 2, 3, 5, 5, 4, 2]
alist = [x for x in alist if x != 4]
print(alist)
#[1, 2, 3, 1, 2, 3, 5, 5, 2]
As user2393256 more generally puts it, you can generalize and define a function my_filter() which will return a boolean based on some check you implement in it. And then you can do:
def my_filter(a_value):
return True if a_value != 4 else False
alist = [x for x in alist if my_filter(x)]
I would go with the function solution if the check was too complicated to type in the list comprehension, so mainly for readability. The example above is therefore not the best since the check is very simple but i just wanted to show you how it would be done.
If you want to delete elements from your list while iterating over it you should use list comprehension.
a = [1,2,3,4,5,6,7]
a = [x for x in a if not check(x)]
You would need to write a "check" function that returns wether or not you want to keep the element in the list.
I don't know where you are going with that but this would do what I beleive you want :
i=0
a = [1,2,3,4,5,6,7]
while boolean_should_i_stop_the_loop :
if i>=len(a) :
boolean_should_i_stop_the_loop = False
#here goes what you want to do in the for loop
print i;
a.append(4)
i += 1
I have a spectra of wavelengths as a list and some number of other lists I use in a formula (using tmm.tmm_core). Is there something more efficient than iterating through the wavelength if I'm just basically doing the same thing for all wavelengths?
Example
def go(n, thk, theta):
#do stuff
return(something)
wv = [1, 2, 3, 4]
a_vec = [3, 7, 3, 9]
b_vec = [6, 5, 9, 3]
c_vec = [0, 1, 8, 9]
theta = 0
th = [10, 1, 10]
final = []
for i in range(len(wv)):
n = [a[i], b[i], c[i]]
answer = go(n, th, theta)
final.append(answer)
in reality there are maybe 5000-10000 rows. It just seems to lag a bit when I press go and I assume it's because of the iteration. Pretty new to optimizing so I haven't used any benchmarking tools or anything.
I think you're looking for the map function in Python!
>>> list1 = [1,2,3,4]
>>> list2 = [5,6,7,8]
>>> map(lambda x,y: x+y, list1, list2)
[6, 8, 10, 12]
it takes in a function (in the above case, an anonymous lambda function), one or more lists and returns another list. At each iteration within the function, both lists are iterated and the result is added to the new list. You don't need to limit yourself to the expressive power of a lambda statement; you can also use globally defined functions as in the case below:
>>> def go(a,b,c):
... return a+b+c
...
>>> map(go, list1,list2, range(9,13))
[15, 18, 21, 24]
You can put all of your lists within a custom list like C_list and use map to create a new list all_len contain the length of all lists then use a list comprehension to create the list final :
all_len=map(len,C_list)
final =[[go([a[i], b[i], c[i]], th, theta) for i in range(li)] for li in all_len]
Also if the length of a and b and c are equal you can use zip function to zip then and refuse of multiple indexing :
all_len=map(len,C_list)
z=zip(a,b,c)
final =[[go(z[i], th, theta) for i in range(li)] for li in all_len]
If you have to perform an operation on every item in the list, then you're gonna have to go through every item in the list. However, you could gain speed through the use of list comprehensions: List Comprehensions