We Keep Coding

Minimum Coin Change Leetcode problem (Dynamic Programming)

Here's the question: https://leetcode.com/problems/coin-change/
I'm having some trouble understanding two different methods of dynamic programming used to solve this problem. I'm currently going through the Grokking Dynamic Programming course from educative.io, and their approach is to use subsets to search for each combination. They go about testing if a coin is viable, if so, then try it in the DFS. If not, skip the coin and go to the next index and try the next coin.
Here's Grokking's approach with memoization:
def coinChange(self, coins: List[int], amount: int) -> int:
def dfs(i, total, memo):
key = (i, total)
if key in memo:
return memo[key]
if total == 0:
return 0
if len(coins) == 0 or i >= len(coins):
return inf
count = inf
if coins[i] <= total:
res = dfs(i, total - coins[i], memo)
if res != inf:
count = res + 1
memo[key] = min(count, dfs(i + 1, total, memo))
return memo[key]
return dfs(0, amount, {}) if dfs(0, amount, {}) != inf else -1
It doesn't do very well on Leetcode; it runs very slowly (but passes, nonetheless). The efficient algorithm that was in the discussions was this:
def coinChange(self, coins: List[int], amount: int) -> int:
#lru_cache(None)
def dp(sum):
if sum == 0: return 0
if sum < 0: return float("inf")
count = float('inf')
for coin in coins:
count = min(count, dp(sum - coin))
return count + 1
return dp(amount) if dp(amount) != float("inf") else -1
Does this second code have the same logic as "testing the subsets of coins?" What's the difference between the two? Is the for-loop a way of testing the different subsets, like with backtracking?
I tested the second algorithm with memoization in a dictionary, like the first, using sum as the key, and it tanked in efficiency. But then I tried using the #lru_cache with the first algorithm, and it didn't help.
Could anyone explain why the second algorithm is so much faster? Is it my memoization that sucks?

Does this second code have the same logic as "testing the subsets of coins?"
If with subset you mean the subset of the coins that is still available for selection, then: no. The second algorithm does not reduce the problem in terms of coins; it reasons that at any time any coin can be selected, irrespective of previous selections. Although this may seem inefficient as it tries to take the same combinations in all possible permutations, this downside is minimised by the effect of memoization.
What's the difference between the two?
The first one takes coins in the order they are given, never going back to take an earlier coin once it has decided to go to the next one. So doing, it tries to reduce the problem in terms of available coins. The second one doesn't care about the order and looks at any permutation, it only reduces the problem in terms of amount.
This first one has a larger memoization collection because the index is part of the key, whereas the second uses a memoization collection that is only keyed by the amount.
The first one makes a recursive call even when no coin is selected (the one at the end of the inner function), since that fits in the logic of reducing the problem to fewer coins. The second one only makes a recursive call when the amount is further reduced.
Is the for-loop a way of testing the different subsets, like with backtracking?
If with subset you mean that the problem is reduced to fewer coins, then no: the second algorithm doesn't attempt to apply that methodology.
The for loop is just a way to consider every coin. It doesn't reduce the problem size in terms of available coins, only in terms of remaining amount.
Could anyone explain why the second algorithm is so much faster?
It is faster because the memoization key is smaller, leading to more hits, leading to fewer recursive calls. You can experiment with this and add global counters that count the number of executions of both inner functions (dfs and dp) and you'll see a dramatic difference there.
Is it my memoization that sucks?
You could say that, but it is too harsh.

Big O of backtracking solution counts permutations with range

I have a problem and I've been struggling with my solution time and space complexity:
Given an array of integers (possible duplicates) A and min, low, high are integers.
Find the total number of combinations of items in A that:
low <= A[i] <= high
Each combination has at least min numbers.
Numbers in one combination can be duplicates as they're considered unique in A but combinations can not be duplicates. E.g.: [1,1,2] -> combinations: [1,1],[1,2],[1,1,2] are ok but [1,1],[1,1], [1,2], [2,1] ... are not.
Example: A=[4, 6, 3, 13, 5, 10], min = 2, low = 3, high = 5
There are 4 ways to combine valid integers in A: [4,3],[4,5],[4,3,5],[3,5]
Here's my solution and it works:
class Solution:
def __init__(self):
pass
def get_result(self, arr, min_size, low, high):
return self._count_ways(arr, min_size, low, high, 0, 0)
def _count_ways(self, arr, min_size, low, high, idx, comb_size):
if idx == len(arr):
return 0
count = 0
for i in range(idx, len(arr)):
if arr[i] >= low and arr[i] <= high:
comb_size += 1
if comb_size >= min_size:
count += 1
count += self._count_ways(arr, min_size, low, high, i + 1, comb_size)
comb_size -= 1
return count
I use backtracking so:
Time: O(n!) because for every single integer, I check with each and every single remaining one in worst case - when all integers can form combinations.
Space: O(n) for at most I need n calls on the call stack and I only use 2 variables to keep track of my combinations.
Is my analysis correct?
Also, a bit out of the scope but: Should I do some kind of memoization to improve it?

If I understand your requirements correctly, your algorithm is far too complicated. You can do it as follows:
Compute array B containing all elements in A between low and high.
Return sum of Choose(B.length, k) for k = min .. B.length, where Choose(n,k) is n(n-1)..(n-k+1)/k!.
Time and space complexities are O(n) if you use memoization to compute the numerators/denominators of the Choose function (e.g. if you have already computed 5*4*3, you only need one multiplication to compute 5*4*3*2 etc.).
In your example, you would get B = [4, 3, 5], so B.length = 3, and the result is
Choose(3, 2) + Choose(3, 3)
= (3 * 2)/(2 * 1) + (3 * 2 * 1)/(3 * 2 * 1)
= 3 + 1
= 4

Your analysis of the time complexity isn't quite right.
I understand where you're getting O(n!): the for i in range(idx, len(arr)): loop decreases in length with every recursive call, so it seems like you're doing n*(n-1)*(n-2)*....
However, the recursive calls from a loop of length m do not always contain a loop of size m-1. Suppose your outermost call has 3 elements. The loop iterates through 3 possible values, each spawning a new call. The first such call will have a loop that iterates over 2 values, but the next call iterates over only 1 value, and the last immediately hits your base case and stops. So instead of 3*2*1=((1+1)+(1+1)+(1+1)), you get ((1+0)+1+0).
A call to _count_ways with an array of size n takes twice as long as a call with size n-1. To see this, consider the first branch in the call of size n which is to choose the first element or not. First we choose that first element, which leads to a recursive call with size n-1. Second we do not choose that first element, which gives us n-1 elements left to iterate over, so it's as if we had a second recursive call with size n-1.
Each increase in n increase time complexity by a factor of 2, so the time complexity of your solution is O(2^n). This makes sense: you're checking every combination, and there are 2^n combinations in a set of size n.
However, as you're only trying to count the combinations and not do something with them, this is highly inefficient. See #Mo B.'s answer for a better solution.

Which algorithm is faster to sort these number pairs?

I wrote two solutions in python.. It is supposed to take a list of numbers and sort the ones that add up to a sum, both these return the same pairs, but which one is more efficient? I'm not sure if using python's count method does more work behind the scene making the second one longer
numbers = [1, 2, 4, 4, 4, 4, 5, 7, 7, 8, 8, 8, 9]
match = []
for i in range(len(numbers)):
for j in range(len(numbers)):
if (i!=j):
if(numbers[i] + numbers[j] == sum):
match.append([numbers[i], numbers[j]])
match2 = []
for i in range(len(numbers)):
counterPart = abs(numbers[i] - sum)
numberOfCounterParts = numbers.count(counterPart)
if(numberOfCounterParts >= 1):
if(counterPart == numbers[i]):
for j in range(numbers.count(counterPart)-1):
match2.append([numbers[i], counterPart])
else:
for j in range(numbers.count(counterPart)):
match2.append([numbers[i], counterPart])
Is there an even better solution that I'm missing?

When comparing algorithms, you should compare their time complexities. Measuring the time is also a good idea, but heavily dependent on the input, which now is tiny.
The first algorithm takes:
O(N2)
because of the double for loop.
For the second algorithm, you should take into account that count() has a time complexity of O(N). You have on for loop, and in its body count() will be called twice, once after abs() and once in whichever body of the if-else statement you go into. As a result the time complexity is O(N) * 2 * O(N) = 2*O(N<sup>2</sup>), which yields:
O(N2)
That means that both algorithm have the same time complexity. As a result it now has meaning to measure the performance, by running many experiments and taking the average of the time measurements, with big enough input to reflect performance.

It's almost always useful to measure complexity of your algorithms.
Both of your algorithms has O(N^2) complexity, so there are almost interchangeable in terms of performance.
You may improve your algorithm by keeping a mapping of value-index pairs. It will reduce complexity to O(N), basically you'll have one loop.

you can run test yourself using the timeit module:
t1 = timeit(setup='from __main__ import sort1, numbers',
stmt='sort1(numbers)',
number=1)
t2 = timeit(setup='from __main__ import sort2, numbers',
stmt='sort2(numbers)',
number=1)
print(t1)
print(t2)
also note that sum is a built-in and therefore not a good name for a variable...
there are way better algorithms for that! especially considering you have duplicates in your list.
here is a faster version that will only give you the matches but not the multiplicity of the matches:
def sum_target(lst, target):
# make list unique
unique_set = set(lst)
unique_list = list(unique_set)
remainders = {number: target-number for number in unique_list}
print(remainders)
match = set()
for a, b in remainders.items():
if a == b and lst.count(a) >= 2:
match.add((a, b))
else:
if b in remainders:
match.add(frozenset((a, b)))
return match

Yes there is better algorithm which can be used if you know lower_bound and upper_bound of the data. Counting Sort which takes O(N) time and space is not constant (which depends on the range of upper bound and lower bound).
Refer Counting Sort
PS: Counting Sort is not a comparison based sort algorithm.
Refer below sample code:
def counting_sort(numbers, k):
counter = [0] * (k + 1)
for i in numbers:
counter[i] += 1
ndx = 0
for i in range(len(counter)):
while 0 < counter[i]:
numbers[ndx] = i
ndx += 1
counter[i] -= 1

Calculations on sliding windows and memoization

I am working on Project Euler Problem 50, which states:
The prime 41, can be written as the sum of six consecutive primes:
41 = 2 + 3 + 5 + 7 + 11 + 13
This is the longest sum of consecutive primes that adds to a prime below one-hundred.
The longest sum of consecutive primes below one-thousand that adds to a prime, contains 21 terms, and is equal to 953.
Which prime, below one-million, can be written as the sum of the most consecutive primes?
For determining the terms in prime P (if it at all can be written as a sum of primes) I use a sliding window of all the primes (in increasing order) up to (but not including) P, and calculate the sum of all these windows, if the sum is equal to the prime considered, I count the length of the window...
This works fine for all primes up to 1000, but for primes up to 10**6 it is very slow, so I was hoping memozation would help; when calculating the sum of sliding windows, a lot of double work is done...(right?)
So I found the standard memoizaton implemention on the net and just pasted it in my code, is this correct? (I have no idea how it is supposed to work here...)
primes = tuple(n for n in range(1, 10**6) if is_prime(n)==True)
count_best = 0
##http://docs.python.org/release/2.3.5/lib/itertools-example.html:
## Slightly modified (first for loop)
from itertools import islice
def window(seq):
for n in range(2, len(seq) + 1):
it = iter(seq)
result = tuple(islice(it, n))
if len(result) == n:
yield result
for elem in it:
result = result[1:] + (elem,)
yield result
def memoize(function):
cache = {}
def decorated_function(*args):
if args in cache:
return cache[args]
else:
val = function(*args)
cache[args] = val
return val
return decorated_function
#memoize
def find_lin_comb(prime):
global count_best
for windows in window(primes[0 : primes.index(prime)]):
if sum(windows) == prime and len(windows) > count_best:
count_best = len(windows)
print('Prime: ', prime, 'Terms: ', count_best)
##Find them:
for x in primes[::-1]: find_lin_comb(x)
(btw, the tuple of prime numbers is generated "decently" fast)
All input is appreciated, I am just a hobby programmer, so please don´t get to advanced on me.
Thank you!
Edit: here is a working code paste that doesn´t have ruined indentations:
http://pastebin.com/R1NpMqgb

This works fine for all primes up to 1000, but for primes up to 10**6 it is very slow, so I was hoping memozation would help; when calculating the sum of sliding windows, a lot of double work is done...(right?)
Yes, right. And of course it's slow for the primes up to 106.
Say you have n primes up to N, numbered in increasing order, p_1 = 2, p_2 = 3, .... When considering whether prime no. k is the sum of consecutive primes, you consider all windows [p_i, ..., p_j], for pairs (i,j) with i < j < k. There are (k-1)*(k-2)/2 of them. Going through all k to n, you examine about n³/6 windows in total (counting multiplicity, you're examining w(i.j) in total n-j times). Even ignoring the cost of creating the window and summing it, you can see how it scales badly:
For N = 1000, there are n = 168 primes and about 790000 windows to examine (counting multiplicity).
For N = 10**6, there are n = 78498 primes and about 8.3*10**13 windows to examine.
Now factor in the work for creating and summing the windows, estimate it low at j-i+1 for summing the j-i+1 primes in w(i,j), the work for p_k is about k³/6, and the total work becomes roughly k**4/24. Something like 33 million steps for N = 1000, peanuts, but nearly 1.6*10**18 for N = 1000000.
A year contains about 3.1*10**7 seconds, with a ~3GHz CPU, that's roughly 1017 clock cycles. So we're talking of an operation needing something like 100 CPU-years (may be a factor of 10 off or so).
You aren't willing to wait that long, I suppose;)
Now, with memoisation, you still look at each window multiple times, but you do the computation of each window only once. That means you need about n³/6 work for the computation of the windows, and look about n³/6 times at any window.
Problem 1: You still need to look at windows about 8.3*10**13 times, that's several hours even if looking cost only one cycle.
Problem 2: There are about 8.3*10**13 windows to memoise. You don't have that much memory, unless you can use a really large HD.
You can circumvent the memory problem by throwing away data you don't need anymore and only calculating data for the windows when it is needed, but once you know which data you may throw away when, you should be able to see a much better approach.
The longest sum of consecutive primes below one-thousand that adds to a prime, contains 21 terms, and is equal to 953.
What does this tell you about the window generating that sum? Where can it start, where can it stop? How can you use that information to create an efficient algorithm to solve the problem?

The memoize decorator adds a wrapper to a function to cache the return value for each value of the argument (each combination of values in case of multiple arguments). It is useful when the function is called multiple times with the same arguments. You can only use it with a pure function, i.e.
The function has no side effects. Changing a global variable and doing output are examples of side effects.
The return value depends only on the values of the arguments, not on some global variables that may change values between calls.
Your find_lin_comb function does not satisfy the above criteria. For one thing, it is called with a different argument every time, for another, the function does not return a value.

Python Time Complexity (run-time)

def f2(L):
sum = 0
i = 1
while i < len(L):
sum = sum + L[i]
i = i * 2
return sum
Let n be the size of the list L passed to this function. Which of the following most accurately describes how the runtime of this function grow as n grows?
(a) It grows linearly, like n does.
(b) It grows quadratically, like n^2 does.
(c) It grows less than linearly.
(d) It grows more than quadratically.
I don't understand how you figure out the relationship between the runtime of the function and the growth of n. Can someone please explain this to me?

ok, since this is homework:
this is the code:
def f2(L):
sum = 0
i = 1
while i < len(L):
sum = sum + L[i]
i = i * 2
return sum
it is obviously dependant on len(L).
So lets see for each line, what it costs:
sum = 0
i = 1
# [...]
return sum
those are obviously constant time, independant of L.
In the loop we have:
sum = sum + L[i] # time to lookup L[i] (`timelookup(L)`) plus time to add to the sum (obviously constant time)
i = i * 2 # obviously constant time
and how many times is the loop executed?
it's obvously dependant on the size of L.
Lets call that loops(L)
so we got an overall complexity of
loops(L) * (timelookup(L) + const)
Being the nice guy I am, I'll tell you that list lookup is constant in python, so it boils down to
O(loops(L)) (constant factors ignored, as big-O convention implies)
And how often do you loop, based on the len() of L?
(a) as often as there are items in the list (b) quadratically as often as there are items in the list?
(c) less often as there are items in the list (d) more often than (b) ?

I am not a computer science major and I don't claim to have a strong grasp of this kind of theory, but I thought it might be relevant for someone from my perspective to try and contribute an answer.
Your function will always take time to execute, and if it is operating on a list argument of varying length, then the time it takes to run that function will be relative to how many elements are in that list.
Lets assume it takes 1 unit of time to process a list of length == 1. What the question is asking, is the relationship between the size of the list getting bigger vs the increase in time for this function to execute.
This link breaks down some basics of Big O notation: http://rob-bell.net/2009/06/a-beginners-guide-to-big-o-notation/
If it were O(1) complexity (which is not actually one of your A-D options) then it would mean the complexity never grows regardless of the size of L. Obviously in your example it is doing a while loop dependent on growing a counter i in relation to the length of L. I would focus on the fact that i is being multiplied, to indicate the relationship between how long it will take to get through that while loop vs the length of L. Basically, try to compare how many loops the while loop will need to perform at various values of len(L), and then that will determine your complexity. 1 unit of time can be 1 iteration through the while loop.
Hopefully I have made some form of contribution here, with my own lack of expertise on the subject.
Update
To clarify based on the comment from ch3ka, if you were doing more than what you currently have inside your with loop, then you would also have to consider the added complexity for each loop. But because your list lookup L[i] is constant complexity, as is the math that follows it, we can ignore those in terms of the complexity.

Here's a quick-and-dirty way to find out:
import matplotlib.pyplot as plt
def f2(L):
sum = 0
i = 1
times = 0
while i < len(L):
sum = sum + L[i]
i = i * 2
times += 1 # track how many times the loop gets called
return times
def main():
i = range(1200)
f_i = [f2([1]*n) for n in i]
plt.plot(i, f_i)
if __name__=="__main__":
main()
... which results in
Horizontal axis is size of L, vertical axis is how many times the function loops; big-O should be pretty obvious from this.

Consider what happens with an input of length n=10. Now consider what happens if the input size is doubled to 20. Will the runtime double as well? Then it's linear. If the runtime grows by factor 4, then it's quadratic. Etc.

When you look at the function, you have to determine how the size of the list will affect the number of loops that will occur.
In your specific situation, lets increment n and see how many times the while loop will run.
n = 0, loop = 0 times
n = 1, loop = 1 time
n = 2, loop = 1 time
n = 3, loop = 2 times
n = 4, loop = 2 times
See the pattern? Now answer your question, does it:
(a) It grows linearly, like n does. (b) It grows quadratically, like n^2 does.
(c) It grows less than linearly. (d) It grows more than quadratically.
Checkout Hugh's answer for an empirical result :)

it's O(log(len(L))), as list lookup is a constant time operation, independant of the size of the list.