Here's the question: https://leetcode.com/problems/coin-change/
I'm having some trouble understanding two different methods of dynamic programming used to solve this problem. I'm currently going through the Grokking Dynamic Programming course from educative.io, and their approach is to use subsets to search for each combination. They go about testing if a coin is viable, if so, then try it in the DFS. If not, skip the coin and go to the next index and try the next coin.
Here's Grokking's approach with memoization:
def coinChange(self, coins: List[int], amount: int) -> int:
def dfs(i, total, memo):
key = (i, total)
if key in memo:
return memo[key]
if total == 0:
return 0
if len(coins) == 0 or i >= len(coins):
return inf
count = inf
if coins[i] <= total:
res = dfs(i, total - coins[i], memo)
if res != inf:
count = res + 1
memo[key] = min(count, dfs(i + 1, total, memo))
return memo[key]
return dfs(0, amount, {}) if dfs(0, amount, {}) != inf else -1
It doesn't do very well on Leetcode; it runs very slowly (but passes, nonetheless). The efficient algorithm that was in the discussions was this:
def coinChange(self, coins: List[int], amount: int) -> int:
#lru_cache(None)
def dp(sum):
if sum == 0: return 0
if sum < 0: return float("inf")
count = float('inf')
for coin in coins:
count = min(count, dp(sum - coin))
return count + 1
return dp(amount) if dp(amount) != float("inf") else -1
Does this second code have the same logic as "testing the subsets of coins?" What's the difference between the two? Is the for-loop a way of testing the different subsets, like with backtracking?
I tested the second algorithm with memoization in a dictionary, like the first, using sum as the key, and it tanked in efficiency. But then I tried using the #lru_cache with the first algorithm, and it didn't help.
Could anyone explain why the second algorithm is so much faster? Is it my memoization that sucks?
Does this second code have the same logic as "testing the subsets of coins?"
If with subset you mean the subset of the coins that is still available for selection, then: no. The second algorithm does not reduce the problem in terms of coins; it reasons that at any time any coin can be selected, irrespective of previous selections. Although this may seem inefficient as it tries to take the same combinations in all possible permutations, this downside is minimised by the effect of memoization.
What's the difference between the two?
The first one takes coins in the order they are given, never going back to take an earlier coin once it has decided to go to the next one. So doing, it tries to reduce the problem in terms of available coins. The second one doesn't care about the order and looks at any permutation, it only reduces the problem in terms of amount.
This first one has a larger memoization collection because the index is part of the key, whereas the second uses a memoization collection that is only keyed by the amount.
The first one makes a recursive call even when no coin is selected (the one at the end of the inner function), since that fits in the logic of reducing the problem to fewer coins. The second one only makes a recursive call when the amount is further reduced.
Is the for-loop a way of testing the different subsets, like with backtracking?
If with subset you mean that the problem is reduced to fewer coins, then no: the second algorithm doesn't attempt to apply that methodology.
The for loop is just a way to consider every coin. It doesn't reduce the problem size in terms of available coins, only in terms of remaining amount.
Could anyone explain why the second algorithm is so much faster?
It is faster because the memoization key is smaller, leading to more hits, leading to fewer recursive calls. You can experiment with this and add global counters that count the number of executions of both inner functions (dfs and dp) and you'll see a dramatic difference there.
Is it my memoization that sucks?
You could say that, but it is too harsh.
Related
def maxN(n:list[int])-> int:
if len(n)==2:
if n[0] > n[1]:
return n[0]
else:
return n[1]
if len(n) ==1:
return n[0]
else:
mid = len(n)//2
first = maxN(n[0:mid])
second= maxN(n[mid:])
if first>second:
return first
else:
return second
I'm getting struggled with my emplementation, because I don't know if this is better than simply using a for or a while loop.
At every level, every function calls two more functions until there are no more numbers. Roughly, the total number of functions calls will be 1 + 2 + 4 + 8 .... n. The total length of the series would be approximately logn as the array is halved at every level.
To get the total number of function calls, we could sum the specified GP series, which would give us the total as n.
We see that there n number of function calls in total and every function does constant amount of work. Thus, the time complexity would be O(n).
The time complexity is similar to the iterative version. However, we know that recursion consumes space in the call stack. Since there could be at most logn functions in the call stack (the depth of the recursive tree), the space complexity will be O(logn).
Thus, the iterative version would be a better choice as it has the same time complexity but a better, O(1), space complexity.
Edit: since the list being splitted into sublists in every function, the splitting cost would add up too. To avoid that, you could pass two variables in the function to the track the list boundary.
for leetcode 322 coin change, the code is below:
class Solution:
def change(self, amount: int, coins: List[int]) -> int:
dp = [0] * (amount + 1)
dp[0] = 1
for i in range(len(coins)):
for j in range(1, amount + 1):
if coins[i] <= j:
dp[j] = dp[j] + dp[j-coins[i]]
return dp[-1]
for leetcode 1049 last stone weight II, the code is below:
class Solution:
def lastStoneWeightII(self, stones: List[int]) -> int:
total = sum(stones)
Max_weight = int(total/2)
print(Max_weight)
current = (Max_weight+1)*[0]
for stone in stones:
for wgt in range(Max_weight, -1, -1):
if wgt-stone>=0:
current[wgt] = max(stone + current[wgt-stone], current[wgt])
print(stone, wgt, current)
return total-2*current[-1]
I like to understand the logic of knapsack more. My biggest question mark is how do we decide whether the second "for loop" is in ascending order or descending order. For example, the first coin change code [for j in range(1, amount + 1):], it is in ascending order but the second stone weigh code[ for wgt in range(Max_weight, -1, -1):] is in descending order. I kind of understand why it works after tracing the code but is there a more intuitive way of really explaining why we use ascending or descending for loop when dealing with knapsack problems. I really not able to grasp the concept of when to use which one.... Thanks!
They are fundamentally two different problems.
The 322 Coin Change problem is an instance of the Unbounded Knapsack problem -- you have infinite quantities of the items that you want to fit in your knapsack.
The 1049 LastStoneWeightII problem is an instance of the Bounded Knapsack problem -- you have only one instance of each item that you want to fit in your knapsack.
The reason you need to run the for loop in descending order in the second problem is because if you don't you will accidentally add the stones more than once. The optimal solution of a subproblem might have already considered the current stone if you run the loop incrementally. But if you run the loop in reverse, you can be sure that the optimal subproblem solution, that you are using to build the solution considering the current stone, does not consist of the current stone.
If you want to use the incrementing for loop for the bounded knapsack problem, you can either use the 2D array version of the algorithm OR you need to keep track of the subset that is part of the solution to each subproblem.
You can find more discussion here: https://leetcode.com/discuss/study-guide/1200320/Thief-with-a-knapsack-a-series-of-crimes.
I am studying Python by the book "a beginner guide to python 3" written by Mr.John Hunt. In chapter 8, which is about recursion, there is an exercise, that demands a code in which a prime number is found by recursion. I wrote first code below independently, but the answer key is written in different structure. Because I am very doubtful about recursion, What is your analysis about these two? Which is more recursive?
My code:
def is_prime(n, holder = 1):
if n == 2:
return True
else:
if (n-1 + holder)%(n-1) == 0:
return False
else:
return is_prime(n-1, holder+1)
print('is_prime(9):', is_prime(9))
print('is_prime(31):', is_prime(31))
Answer key:
def is_prime(n, i=2):
# Base cases
if n <= 2:
return True if (n == 2) else False
if n % i == 0:
return False
if i * i > n:
return True
# Check for next divisor
return is_prime(n, i + 1)
print('is_prime(9):', is_prime(9))
print('is_prime(31):', is_prime(31))
My suggestion in this case would be not to use recursion at all. Whilst I understand that you want to use this as a learning example of how to use recursion, it is also important to learn when to use recursion.
Recursion has a maximum allowed depth, because the deeper the recursion, the more items need to be put on the call stack. As such, this is not a good example to use recursion for, because it is easy to reach the maximum in this case. Even the "model" example code suffers from this. The exact maximum recursion depth may be implementation-dependent, but for example, if I try to use it to compute is_prime(1046527) then I get an error:
RecursionError: maximum recursion depth exceeded while calling a Python object
and inserting a print(i) statement shows that it is encountered when i=998.
A simple non-recursive equivalent of the "model" example will not have this problem. (There are more efficient solutions, but this one is trying to stay close to the model solution apart from not using recursion.)
def is_prime(n):
if n == 2:
return True
i = 2
while i * i <= n:
if n % i == 0:
return False
i += 1
return True
(In practice you would probably also want to handle n<2 cases.)
If you want a better example of a problem to practise recursive programming, check out the Tower of Hanoi problem. In this case, you will find that using recursion allows you to make a simpler and cleaner solution than is possible without it, while being unlikely to involve exceeding the maximum recursion depth (you are unlikely to need to consider a tower 1000 disks high, because the solution would require a vast number of moves, 2^1000-1 or about 10^301).
As another good example of where recursion can be usefully employed, try using turtle graphics to draw a Koch snowflake.
I'd say the Answer Key needs improvement. We can make it faster and handle the base cases more cleanly:
def is_prime(n, i=3):
# Base cases
if n < 2:
return False
if n % 2 == 0:
return n == 2
if i * i > n:
return True
if n % i == 0:
return False
# Check for next divisor
return is_prime(n, i + 2)
The original answer key starts at 2 and counts up by 1 -- here we start at 3 and count up by 2.
As far as your answer goes, there's a different flaw to consider. Python's default stack depth is 1,000 frames, and your function fails shortly above input of 1,000. The solution above uses recursion more sparingly and can handle input of up to nearly 4,000,000 before hitting up against Python's default stack limit.
Yes your example seems to work correctly. Note However, that by the nature of the implementation, the answer key is more efficient. To verify that a number n is a prime number, your algorithm uses a maximum of n-1 function calls, while the provided answer stops after reaching the iteration count of sqrt(n). Checking higher numbers makes generally no sense since if n is dividable without remainder by a value a > sqrt(n) it has to also be dividable by b = n % a.
Furthermore, your code raises an exception for evaluating at n = 1 since the modulo of 0 is not defined.
I have coded a greedy recursive algorithm to Find minimum number of coins that make a given change. Now I need to estimate its time complexity. As the algorithm has nested "ifs" depending on the same i (n * n), with the inner block halving the recursive call (log(2)n), I believe the correct answer could be O(n*log(n)), resulting from the following calculation:
n * log2(n) * O(1)
Please, give me your thoughts on whether my analysis is correct and feel free to also suggest improvements on my greedy recursive algorithm.
This is my recursive algorithm:
coins = [1, 5, 10, 21, 25]
coinsArraySize = len(coins)
change = 63
pickedCoins = []
def findMin(change, i, pickedCoins):
if (i>=0):
if (change >= coins[i]):
pickedCoins.append(coins[i])
findMin(change - coins[i], i, pickedCoins)
else:
findMin(change, i-1, pickedCoins)
findMin(change, coinsArraySize-1, pickedCoins)
Each recursive call decreases change by at least 1, and there is no branching (that is, your recursion tree is actually a straight line, so no recursion is actually necessary). Your running time is O(n).
What is n? The runtime depends on both the amount and the specific coins. For example, suppose you have a million coins, 1 through 1,000,000, and try to make change for 1. The code will go a million recursive levels deep before it finally finds the largest coin it can use (1). Same thing in the end if you have only one coin (1) and try to make change for 1,000,000 - then you find the coin at once, but go a million levels deep picking that coin a million times.
Here's a non-recursive version that improves on both of those: use binary search to find the next usable coin, and once a suitable coin is found use it as often as possible.
def makechange(amount, coins):
from bisect import bisect_right
# assumes `coins` is sorted. and that coins[0] > 0
right_bound = len(coins)
result = []
while amount > 0:
# Find largest coin <= amount.
i = bisect_right(coins, amount, 0, right_bound)
if not i:
raise ValueError("don't have a coin <=", amount)
coin = coins[i-1]
# How many of those can we use?
n, amount = divmod(amount, coin)
assert n >= 1
result.extend([coin] * n)
right_bound = i - 1
return result
It still takes O(amount) time if asked to make change for a million with the only coin being 1, but because it has to build a result list with a million copies of 1. If there are a million coins and you ask for change for 1, though, it's O(log2(len(coins))) time. The first could be slashed by changing the output format to a dict, mapping a coin to the number of times that coin is used. Then the first case would be cut to O(1) time.
As is, the time it takes is proportional to the length of the result list, plus some (usually trivial) time for a number of binary searches equal to the number of distinct coins used. So "a bad case" is one where every coin needs to be used; e.g.,
>>> coins = [2**i for i in range(10)]
>>> makechange(sum(coins), coins)
[512, 256, 128, 64, 32, 16, 8, 4, 2, 1]
That's essentially O(n + n log n) where n is len(coins).
Finding an optimal solution
As #Stef noted in a comment, the greedy algorithm doesn't always find a minimal number of coins. That's substantially harder. The usual approach is via dynamic programming, with worst case O(amt * len(coins)) time. But that's also best case: it works "bottom up", finding the smallest number of coins to reach 1, then 2, then 3, then 4, ..., and finally amt.
So I'm going to suggest a different approach, using breadth-first tree search, working down from the initial amount until reaching 0. Worst-case O() behavior is the same, but best-case time is far better. For the comment's:
mincoins(10000, [1, 2000, 3000])
case it looks at less than 20 nodes before finding the optimal 4-coin solution. Because it's a breadth-first search, it knows there's no possible shorter path to the root, so can stop right away then.
For a worst-case example, try
mincoins(1000001, range(2, 200, 2))
All the coins are even numbers, so it's impossible for any collection of them to sum to the odd target. The tree has to be expanded half a million levels deep before it realizes 0 is unreachable. But while the branching factor at high levels is O(len(coins)), the total number of nodes in the entire expanded tree is bounded above by amt + 1 (pigeonhole principle: the dict can't have more than amt + 1 keys, so any number of nodes beyond that are necessarily duplicate targets and so are discarded as soon as they're generated). So, in reality, the tree in this case grows wide very quickly, but then quickly becomes very narrow and very deep.
Also note that this approach makes it very easy to reconstruct the minimal collection of coins that sum to amt.
def mincoins(amt, coins):
from collections import deque
coins = sorted(set(coins)) # increasing, no duplicates
# Map amount to coin that was subtracted to reach it.
a2c = {amt : None}
d = deque([amt])
while d:
x = d.popleft()
for c in coins:
y = x - c
if y < 0:
break # all remaining coins too large
if y in a2c:
continue # already found cheapest way to get y
a2c[y] = c
d.append(y)
if not y: # done!
d.clear()
break
if 0 not in a2c:
raise ValueError("not possible", amt, coins)
picks = []
a = 0
while True:
c = a2c[a]
if c is None:
break
picks.append(c)
a += c
assert a == amt
return sorted(picks)
I created this program for an assignment in which we were required to create an implementation of Quichesort. This is a hybrid sorting algorithm that uses Quicksort until it reaches a certain recursion depth (log2(N), where N is the length of the list), then switches to Heapsort, to avoid exceeding the maximum recursion depth.
While testing my implementation, I discovered that although it generally performed better than regular Quicksort, Heapsort consistently outperformed both. Can anyone explain why Heapsort performs better, and under what circumstances Quichesort would be better than both Quicksort and Heapsort?
Note that for some reason, the assignment referred to the algorithm as "Quipsort".
Edit: Apparently, "Quichesort" is actually identical to
Introsort.
I also noticed that a logic error in my medianOf3() function was
causing it to return the wrong value for certain inputs. Here is an improved
version of the function:
def medianOf3(lst):
"""
From a lst of unordered data, find and return the the median value from
the first, middle and last values.
"""
first, last = lst[0], lst[-1]
if len(lst) <= 2:
return min(first, last)
middle = lst[(len(lst) - 1) // 2]
return sorted((first, middle, last))[1]
Would this explain the algorithm's relatively poor performance?
Code for Quichesort:
import heapSort # heapSort
import math # log2 (for quicksort depth limit)
def medianOf3(lst):
"""
From a lst of unordered data, find and return the the median value from
the first, middle and last values.
"""
first, last = lst[0], lst[-1]
if len(lst) <= 2:
return min(first, last)
median = lst[len(lst) // 2]
return max(min(first, median), min(median, last))
def partition(pivot, lst):
"""
partition: pivot (element in lst) * List(lst) ->
tuple(List(less), List(same, List(more))).
Where:
List(Less) has values less than the pivot
List(same) has pivot value/s, and
List(more) has values greater than the pivot
e.g. partition(5, [11,4,7,2,5,9,3]) == [4,2,3], [5], [11,7,9]
"""
less, same, more = [], [], []
for val in lst:
if val < pivot:
less.append(val)
elif val > pivot:
more.append(val)
else:
same.append(val)
return less, same, more
def quipSortRec(lst, limit):
"""
A non in-place, depth limited quickSort, using median-of-3 pivot.
Once the limit drops to 0, it uses heapSort instead.
"""
if lst == []:
return []
if limit == 0:
return heapSort.heapSort(lst)
limit -= 1
pivot = medianOf3(lst)
less, same, more = partition(pivot, lst)
return quipSortRec(less, limit) + same + quipSortRec(more, limit)
def quipSort(lst):
"""
The main routine called to do the sort. It should call the
recursive routine with the correct values in order to perform
the sort
"""
depthLim = int(math.log2(len(lst)))
return quipSortRec(lst, depthLim)
Code for Heapsort:
import heapq # mkHeap (for adding/removing from heap)
def heapSort(lst):
"""
heapSort(List(Orderable)) -> List(Ordered)
performs a heapsort on 'lst' returning a new sorted list
Postcondition: the argument lst is not modified
"""
heap = list(lst)
heapq.heapify(heap)
result = []
while len(heap) > 0:
result.append(heapq.heappop(heap))
return result
The basic facts are as follows:
Heapsort has worst-case O(n log(n)) performance but tends to be slow in practice.
Quicksort has O(n log(n)) performance on average, but O(n^2) in the worst case but is fast in practice.
Introsort is intended to harness the fast-in-practice performance of quicksort, while still guaranteeing the worst-case O(n log(n)) behavior of heapsort.
One question to ask is, why is quicksort faster "in practice" than heapsort? This is a tough one to answer, but most answers point to how quicksort has better spatial locality, leading to fewer cache misses. However, I'm not sure how applicable this is to Python, as it is running in an interpreter and has a lot more junk going on under the hood than other languages (e.g. C) that could interfere with cache performance.
As to why your particular introsort implementation is slower than Python's heapsort - again, this is difficult to determine. First of all, note that the heapq module is written in Python, so it's on a relatively even footing with your implementation. It may be that creating and concatenating many smaller lists is costly, so you could try rewriting your quicksort to act in-place and see if that helps. You could also try tweaking various aspects of the implementation to see how that affects performance, or run the code through a profiler and see if there are any hot spots. But in the end I think it's unlikely you'll find a definite answer. It may just boil down to which operations are particularly fast or slow in the Python interpreter.