We Keep Coding

Python recursion memoization

I've had the following assignment: Given a list of n integers, each integer in the list is unique and larger than 0. I am also given a number K - which is an integer larger than 0.
List slicing of any kind is not allowed
I need to check whether there is a subset that sums up to K.
e.g: for the list [1,4,8], and k=5, I return True, because we have the subset {1,4}.
Now I need to implement this using a recursion:
So I did, however I needed to implement memoization:
And I wonder what is the difference between those functions' code:
I mean, both seem to implement memoization, however the second should work better but it doesn't. I'd really appreciate some help :)
def subsetsum_mem(L, k):
'''
fill-in your code below here according to the instructions
'''
sum_dict={}
return s_rec_mem(L,0,k,sum_dict)
def s_rec_mem(L, i, k, d):
'''
fill-in your code below here according to the instructions
'''
if(k==0):
return True
elif(k<0 or i==len(L)):
return False
else:
if k not in d:
res_k=s_rec_mem(L,i+1,k-L[i],d) or s_rec_mem(L,i+1,k,d)
d[k]=res_k
return res_k
def subsetsum_mem2(L, k):
'''
fill-in your code below here according to the instructions
'''
sum_dict={}
return s_rec_mem2(L,0,k,sum_dict)
def s_rec_mem2(L, i, k, d):
'''
fill-in your code below here according to the instructions
'''
if(k==0):
return True
elif(k<0 or i==len(L)):
return False
else:
if k not in d:
res_k=s_rec_mem2(L,i+1,k-L[i],d) or s_rec_mem2(L,i+1,k,d)
d[k]=res_k
return res_k
else:
return d[k]

You have two problems with your memoization.
First, you're using just k as the cache key. But the function does different things for different values of i, and you're ignoring i in the cache, so you're going to end up returning the value from L, 9, 1, d for L, 1, 1, d.
Second, only in s_rec_mem, you never return d[k]; if it's present, you just fall off the end and return None (which is falsey).
So, the second one does come closer to working—but it still doesn't actually work.
You could fix it like this:
def s_rec_mem2(L, i, k, d):
'''
fill-in your code below here according to the instructions
'''
if(k==0):
return True
elif(k<0 or i==len(L)):
return False
else:
if (i, k) not in d:
res_k=s_rec_mem2(L,i+1,k-L[i],d) or s_rec_mem2(L,i+1,k,d)
d[i, k]=res_k
return res_k
else:
return d[i, k]
… or by just using lru_cache, either by passing down tuple(L) instead of L (because tuples, unlike lists, can be hashed as keys, and your recursive function doesn't care what kind of sequence it gets), or by making it a local function that sees L via closure instead of getting passed it as a parameter.
Finally, from a quick glance at your code:
It looks like you're only ever going to evaluate s_rec_mem at most twice on each set of arguments (assuming you correctly cache on i, k rather than just k), which means memoization can only provide a 2x constant speedup at best. To get any more than that, you need to rethink your caching or your algorithm.
You're only memoizing within each separate top-level run, but switching to lru_cache on tuple(L), i, k means you're memoizing across all runs until you restart the program (or REPL session)—so the first test may take a long time, but subsequent runs on the same L (even with a different k) could benefit from previous caching.
You seem to be trying to solve a minor variation of the subset sum problem. That problem in the general case is provably NP-complete, which means it's guaranteed to take exponential time. And your variation seems to be if anything harder than the standard problem, not easier. If so, not only is your constant-factor speedup not going to have much benefit, there's really nothing you can do that will do better. In real life, people who solve equivalent problems usually use either optimization (e.g., via dynamic programming) or approximation to within an arbitrary specified limit, both of which allow for polynomial (or at least pseudo-polynomial) solutions in most cases, but can't solve all cases. Or, alternatively, there are subclasses of inputs for which you can solve the problem in polynomial time, and if you're lucky, you can prove your inputs fall within one of those subclasses. If I've guessed right on what you're doing, and you want to pursue any of those three options, you'll need to do a bit of research. (Maybe Wikipedia is good enough, or maybe there are good answers here or on the compsci or math SE sites to get you started.)

CVXPY: how to efficiently solve a series of similar problems

I have a large problem defined in CVXPY modelling language. I want to solve series of this problems - still the same format but with different parameters (constants).
I found out that after calling problem.solve() internal problem generating takes 20 s and main optimization runtime takes 0.2 s. It's a lot of time when I want to solve dozens of similar problems.
Is there any tool for CVXPY like YALMIP optimizer or any possibility to reduce problem generating time?

Yes there is. And it's even explained in the official docs.
Parameters
Parameters are symbolic representations of constants. The purpose of parameters is to change the value of a constant in a problem without reconstructing the entire problem.
Example straight from the docs (modified):
from cvxpy import *
import numpy
# Problem data.
n = 15
m = 10
numpy.random.seed(1)
A = numpy.random.randn(n, m)
b = numpy.random.randn(n, 1)
# gamma must be positive due to DCP rules.
gamma = Parameter(sign="positive") # !!!
# Construct the problem.
x = Variable(m)
error = sum_squares(A*x - b)
obj = Minimize(error + gamma*norm(x, 1))
prob = Problem(obj) # !!!
# Construct a trade-off curve of ||Ax-b||^2 vs. ||x||_1
sq_penalty = []
l1_penalty = []
x_values = []
gamma_vals = numpy.logspace(-4, 6)
for val in gamma_vals:
gamma.value = val # !!!
prob.solve() # !!!
# Use expr.value to get the numerical value of
# an expression in the problem.
sq_penalty.append(error.value)
l1_penalty.append(norm(x, 1).value)
x_values.append(x.value)
So what does it do
As you noticed, the setup of your optimization-problem might take some time, because it's following the DCP-approach (which proves convexity by construction).
Using parameter, this DCP-processing is only done one time! Every new solve willl only change some small parts within the problem. It's important to describe your parameter as precise as possible, so that DCP can work. Example: Parameter(sign="positive").
Is there more you can do
Maybe. Depending on the solver, you can also use warm-starting, if you think a special guess (e.g. solution-vector of your last iteration) is a good start for the new problem.
This would replace: prob.solve() with prob.solve(warm_start=True), resulting in reusing the previous solution as a start (explained here). Manually defining this vector does not seem to be possible (from cvxpy).
Sadly, the only solver supporting this (within cvxpy) is SCS as far as i know (and others will ignore it without crashing)!

Multiple solutions when doing ILP

Currently I'm using PuLP to solve a maximization problem. It works fine, but I'd like to be able to get the N-best solutions instead of just one. Is there a way to do this in PuLP or any other free/Python solution? I toyed with the idea of just randomly picking some of the variables from the optimal solution and throwing them out and re-running, but this seems like a total hack.

If your problem is fast to solve, you can try to limit the objective from above step by step. For examle, if the objective value of the optimal solution is X, try to re-run the problem with an additional constraint:
problem += objective <= X - eps, ""
where the reduction step eps depends on your knowledge of the problem.
Of course, if you just pick some eps blindly and get a solution, you don't know if the solution is the 2nd best, 10th best or 1000-th best ... But you can do some systematic search (binary, grid) on the eps parameter (if the problem is really fast to solve).

So I figured out how (by RTFM) to get multiple soutions. In my code I essentially have:
number_unique = 1 # The number of variables that should be unique between runs
model += objective
model += constraint1
model += constraint2
model += constraint3
for i in range(1,5):
model.solve()
selected_vars = []
for p in vars:
if p_vars[p].value() != 0:
selected_vars.append(p)
print_results()
# Add a new constraint that the sum of all of the variables should
# not total up to what I'm looking for (effectively making unique solutions)
model += sum([p_vars[p] for p in selected_vars]) <= 10 - number_unique
This works great, but I've realized that I really do need to go the random route. I've got 10 different variables and by only throwing out a couple of them, my solutions tend to have the same heavy weighted vars in all the permutations (which is to be expected).

seeking convergence with optimize.fmin on scipy

I have a function I want to minimize with scipy.optimize.fmin. Note that I force a print when my function is evaluated.
My problem is, when I start the minimization, the value printed decreases untill it reaches a certain point (the value 46700222.800). There it continues to decrease by very small bites, e.g., 46700222.797,46700222.765,46700222.745,46700222.699,46700222.688,46700222.678
So intuitively, I feel I have reached the minimum, since the length of each step are minus then 1. But the algorithm keeps running untill I get a "Maximum number of function evaluations has been exceeded" error.
My question is: how can I force my algorithm to accept the value of the parameter when the function evaluation reaches a value from where it does not really evolve anymore (let say, I don't gain more than 1 after an iteration). I read that the options ftol could be used but it has absolutely no effect on my code. In fact, I don't even know what value to put for ftol. I tried everything from 0.00001 to 10000 and there is still no convergence.

There is actually no need to see your code to explain what is happening. I will answer point by point quoting you.
My problem is, when I start the minimization, the value printed decreases
untill it reaches a certain point (the value 46700222.800). There it
continues to decrease by very small bites, e.g.,
46700222.797,46700222.765,46700222.745,46700222.699,46700222.688,46700222.678
Notice that the difference between the last 2 values is -0.009999997913837433, i.e. about 1e-2. In the convention of minimization algorithm, what you call values is usually labelled x. The algorithm stops if these 2 conditions are respected AT THE SAME TIME at the n-th iteration:
convergence on x: the absolute value of the difference between x[n] and the next iteration x[n+1] is smaller than xtol
convergence on f(x): the absolute value of the difference between f[n] and f[n+1] is smaller than ftol.
Moreover, the algorithm stops also if the maximum number of iterations is reached.
Now notice that xtol defaults to a value of 1e-4, about 100 times smaller than the value 1e-2 that appears for your case. The algorithm then does not stop, because the first condition on xtol is not respected, until it reaches the maximum number of iterations.
I read that the options ftol could be used but it has absolutely no
effect on my code. In fact, I don't even know what value to put for
ftol. I tried everything from 0.00001 to 10000 and there is still no
convergence.
This helped you respecting the second condition on ftol, but again the first condition was never reached.
To reach your aim, increase also xtol.
The following methods will also help you more in general when debugging the convergence of an optimization routine.
inside the function you want to minimize, print the value of x and the value of f(x) before returning it. Then run the optimization routine. From these prints you can decide sensible values for xtol and ftol.
consider nondimensionalizing the problem. There is a reason if ftol and xtol default both to 1e-4. They expect you to formulate the problem so that x and f(x) are of order O(1) or O(10), say numbers between -100 and +100. If you carry out the nondimensionalization you handle a simpler problem, in the way that you often know what values to expect and what tolerances you are after.
if you are interested just in a rough calculation and can't estimate typical values for xtol and ftol, and you know (or you hope) that your problem is well behaved, i.e. that it will converge, you can run fmin in a try block, pass to fmin only maxiter=20 (say), and catch the error regarding the Maximum number of function evaluations has been exceeded.

I just spent three hours digging into the source code of scipy.minimize. In it, the "while" loop in function "_minimize_neldermead" deals with the convergence rule:
if (numpy.max(numpy.ravel(numpy.abs(sim[1:] - sim[0]))) <= xtol and
numpy.max(numpy.abs(fsim[0] - fsim[1:])) <= ftol):
break"
"fsim" is the variable that stores results from functional evaluation. However, I found that fsim[0] = f(x0) which is the function evaluation of the initial value, and it never changes during the "while" loop. fsim[1:] updates itself all the time. The second condition of the while loop was never satisfied. It might be a bug. But my knowledge of mathematical optimization is far from enough to judge it.
My current solution: design your own system to control the convergence. Add this in your function:
global x_old, Q_old
if (np.absolute(x_old-x).sum() <= 1e-4) and (np.absolute(Q_old-Q).sum() <= 1e-4):
return None
x_old = x; Q_old = Q
Here Q=f(x). Don't forget to give them an initial value.
Update 01/30/15:
I got it! This should be the correct code for the second line of the if function (i.e. remove numpy.absolute):
numpy.max(fsim[0] - fsim[1:]) <= ftol)
btw, this is my first debugging of a open source software. I just created an issue on GitHub.
Update 01/31/15 - 1:
I don't think my previous update is correct. Nevertheless, this is the a screenshot of the iterations of a function using the original code.
It prints the values of sim and fsim variable for each iteration. As you can see, the changes of each iteration is less than both of xtol and ftol values, but it just kept going without stopping. The original code compares the difference between fsim[0] and the rest of fsim values, i.e. the value here is always 87.63228689 - 87.61312213 = .01916476, which is greater than ftol=1e-2.
Update 01/31/15 - 2:
Here is the data and code that I used to reproduce the previous results. It includes two data files and one iPython Notebook file.

From the documentation it looks like you DO want to change the ftol arg.
Post your code so we can look at your progress.
edit: Try increasing xtol as well.

Your question is a bit ambiguous. Are you printing the value of your function, or the point where it is evaluated?
My understanding of xtol and ftol is as follows. The iteration stops
when the change in the value of the function between iterations is less than ftol
AND
when the change in x between successive iterations is less than xtol
When you say "...accept the value of the parameter...", this suggests you should change xtol.

Two different approaches to writing a function (map vs loop)

SO busy with some code, and have a function which basically takes dictionary where each value is a list, and returns the key with the largest list.
I wrote the following:
def max_list(dic):
if dic:
l1 = dic.values()
l1 = map(len, l1)
l2 = dic.keys()
return l2[l1.index(max(l1))]
else:
return None
Someone else wrote the following:
def max_list(dic):
result = None
maxValue = 0
for key in dic.keys():
if len(dic[key]) >= maxValue:
result = key
maxValue = len(dic[key])
return result
Which would be the 'correct' way to do this, if there is one. I hope this is not regarded as community wiki (even though the code works), trying to figure which would be the best pattern in terms of the problem.

Another valid option:
maxkey,maxvalue = max(d.items(),key=lambda x: len(x[1]))
Of the two above, I would probably prefer the explicit for loop as you don't generate all sorts of intermediate objects just to throw them away.
As a side note, This solution doesn't work particularly well for empty dicts ... (it raises a ValueError). Since I expect that is an unusual case (rather than the norm), it shouldn't hurt to enclose in a try-except ValueError block.

the most pythonic would be max(dic,key=lambda x:len(dic[x])) ... at least I would think ...
maximizing readability and minimizing lines of code is pythonic ... usually

I think the question you should ask yourself is, what do you think the most important is: code maintainability or computation speed?
As the other answers point out, this problem has a very concise solution using a map. For most people this implementation would probably be more easy to read then the implementation with a loop.
In terms of computational speed, the map solution would be less efficient, but still be in the same Computational Magnitute.
Therefore, I think it is unlikely that the map method would ever have noticeably less performance. I would suggest you to use a profiler after your program is finished, so you can be sure where the real problem lies if your program turns out to run slower than desired.