I wrote a simple algorithm to return a list of all possible permutations of a string, as follows:
def get_permutations(sequence):
'''
Enumerate all permutations of a given string
sequence (string): an arbitrary string to permute. Assume that it is a
non-empty string.
Returns: a list of all permutations of sequence
'''
if len(sequence) <= 1:
return list(sequence)
else:
return_list = get_permutations(sequence[1:])
new_list = []
for e in return_list:
for pos in range(len(e) + 1):
new_list.append(e[:pos] + sequence[0] + e[pos:])
return new_list
From this code I'm seeing a time complexity of O(n* n!), O(n!) is the increasing tendency for the number of elements "e" in the "return_list", and there's a nested loop that increases linearly with each new recursion, so from my understanding, O(n). The conclusion is that the algorithm as a whole has O(n*n!) complexity.
However, when searching for similar solutions I found many threads saying the optimal case for this type of algorithm should be only O(n!), so my question is:
Am I missing something on my complexity analysis or is my code not optimal? And if it isn't, how can I properly correct it?
Take any algorithm that generates and then prints out all permutations of a sequence of n different elements. Then, since there are n! different permutations and each one has n elements, simply printing out all the permutations will take time Θ(n · n!). That's worth keeping in mind as you evaluate the cost of generating permutations - even if you could generate all permutations in time O(n!), you couldn't then visit all those permutations without doing O(n · n!) work to view them all.
That being said - the recursive permutation-generating code you have up above does indeed run in time Θ(n · n!). There are some other algorithms for generating permutations that can generate but not print the permutations in time Θ(n!), but they work on different principles.
I have found, empirically, that unless you see a careful runtime analysis of a permutation-generating algorithm, you should be skeptical that the runtime is Θ(n!). Most algorithms don't hit this runtime, and in the cases of the ones that do, the analysis is somewhat subtle. Stated differently - you're not missing anything; there's just lots of "on the right track but incorrect" claims made out there. :-)
I think your algorithm is O(n * n!) because to calculate a permutation of a string x of length n, your algorithm will use the permutations of a sub string of x, which is x without the first character. I'll call this sub string y. But to calculate the permutations of y, the permutations of a sub string of y will need to be calculated. This will continue until the sub string to have its permutations calculated is of length 1. This means that to calculate the permutations of x you will need to calculate the permutations of n - 1 other strings.
Here is an example. Let's say the input string was "pie". Then what your algorithm does is it takes "pie" and calls its self again with "ie", after which it calls itself with "e". Because "e" is of length 1 it returns and all the permutations for "ie" are found which are "ie" and ei". Then that function call will return the permutations of "ie" and it is only at this point that the permutations of "pie" are calculated which it does using the permutations of "ie".
I looked up a permutation generating algorithm called Heap's algorithm that has a time complexity of O(n!). The reason it has a time complexity of n! is because it generates permutations using swaps and each swap that it does on an array generates a unique permutation for the input string. Your algorithm however, generates permutations of the n-1 sub strings of the input string which is where the time complexity of O(n * n!) comes from.
I hope this helps and sorry if I'm being overly verbose.
Related
I had this question on a test and i'm trying to understand it:
What is the time complexity of this function (in the worst case) assuming that Bubblesort() is the most optimized version of the Bubble Sort algorithm?
def MultSort(a,n):
for i in range(n):
BubbleSort(a)
The options were:
Linear
Quadratic
Cubic
I was thinking that the first sort (because it's the worst case) would be O(len(a)^2) and then n*O(len(a)) once it's ordered.But i can't really get to a result by myself
The annoying thing in this exercise is that the answers to choose from do not indicate what the dimension of variation is by which the complexity would be linear, quadratic or cubic. There are two obvious candidates: len(a) and n. Since no distinction is made in the exercise, and no information is given as to whether one of the two is to be assumed constant, your best guess would be to assume they are equal (n = len(a)).
We could guess that n is given as a separate argument because the same function signature would be used in languages where there is no equivalent for the len() function, like for C arrays.
Anyway, with the assumption that n = len(a), here is what we can say:
the first sort (because it's the worst case) would be O(len(a)^2)
Yes, or in other words: O(𝑛²)
and then n*O(len(a)) once it's ordered.
Yes, or in other words: 𝑛O(𝑛) = O(𝑛²)
But i can't really get to a result...
The final step is to add these two phases that you have analysed, so here is a spoiler in case you don't see how those can be added:
O(𝑛²) + O(𝑛²) = O(𝑛²)
...which means the anser is:
Quadratic
first of all, can we even measure it since we don't know how many times random.shuffle will shuffle the array until it reaches the desired outcome
def sort(numbers):
import random
while not sort(numbers)==numbers:
random.shuffle(numbers)
return numbers
First I assume the function name to not be sort as this would be trivial and would lead to unconditional infinite recursion. I am assuming this function
import random
def random_sort(numbers):
while not sorted(numbers) == numbers:
random.shuffle(numbers)
return numbers
Without looking at the implementation to much I would assume O(n) for the inner shuffle random.shuffle(numbers). Where n is the number of elements in numbers.
Then we have the while loop. It stops when the array is sorted. Now shuffle returns us one of all possible permutations of numbers. The loop aborts when its sorted. This is for just one of those. (if we don't assume a small number space).
This stopping is statistical. So we need technically define which complexity we are speaking of. This is where best case, worst case, amortized case comes in.
Best case
The numbers we get are already sorted. Then we have the cost of sort(numbers) and the comparison .. == numbers. Sorting a sorted array is O(n). So our best case complexity is O(n).
Worst case
The shuffle never gives us the right permutation. This is definitely possible. The algorithm would never terminate. So its O(∞).
Average case
This is probably the most interesting case. First we need to establish how many permutations shuffle is giving us. Here is a link which discusses that. An approximation is given as e ⋅ n!. Which is O(n!) (please check).
Now the question is on average when does our loop stop. This is answered in this link. They say its the geometric distribution (please check). The result is 1/ p, where p is the probablity of getting it. In our case this is p = 1 / (e ⋅ n!). So we need on average e ⋅ n! tries.
Now for each try we need to sort O(n log(n)), compare O(n) and compute the shuffle O(n). For the shuffle we can say it uses the Fisher Yates algorithm which has a complexity of O(n), as shown here.
So we have O(n! n log(n)) for the average complexity.
I've been trying to understand complexities, but all the online material has left me confused. Especially the part where they create actual mathematical functions. I have a for loop and a while loop. My confusion arises from the while loop. I know the complexity of the for loop is O(n), but the while loop is based on randomness. A random number if picked, and if this number is not in the list, it is added and the while loop broken. But my confusion arises here, the while loop may run in the worst case (in my thoughts) for an m number of times, until it is done. So I was thinking the complexity would then be O(n*m)?
I'm just really lost, and need some help.
Technically worst-case complexity is O(inf): random.randint if we consider it real random generator (it isn't, of course) can produce arbitrary long sequence with equal elements. However, we can estimate "average-case" complexity. It isn't the real average-case complexity (best, worst and average cases must be defined by input, not randomly), but it can show how many iterations program will do, if we run it for fixed n multiple times and take average of results.
Let's note that the list works as set here (you never add repeated number), so I'd stick with not in set comparison instead which is O(1) (while not in list is O(i)) to remove that complexity source and simplify things a bit: now count of iterations and complexity can be estimated with same big O limits. Single trial here is choosing from uniform integer distribution on [1; n]. Success is choosing number that is not in the list yet.
Then what's expected value of number of trials before getting item that is not in the set? Set size before each step is i in your code. We can pick any of n-i numbers. Thus probability of success is p_i = (n-i)/n (as the distribution is uniform). Every outer iteration is an example of geometrical distribution: count of trials before first success. So estimated count of while iterations is n_i = 1 / p_i = n / (n-i). To get final complexity we should sum this counts for each for iteration: sum(n_i for i in range(n)). This is obviously equal to n * Harmonic(n), where Harmonic(n) is n-th harmonic number (sum of first n reciprocals to natural numbers). Harmonic(n) ~ O(log n), thus "average-case" complexity of this code is O(n log n).
For list it will be sum(i*n / (n-i) for i in range(n)) ~ O(n^2 log(n)) (proof of this equality will be a little longer).
Big 'O' notation is used for worst case scenarios only.
figure out what could be he worst case for given loop.
make a function in 'n' , and take highest power of 'n' and ignore constant if any, you will get time complexity.
Given two strings s & t, determine if s is divisible by t. For
example: "abab" is divisible by "ab" But "ababab" is not divisible by
"abab". If it isn't divisible, return -1. If it is, return the length
of the smallest common divisor: So, for "abababab" and "abab", return
2 as s is divisible by t and the smallest common divisor is "ab" with
length 2.
The way I thought it through was: I define the lengths of these two strings, find the greatest common divisor of these two. If t divides s, then the smallest common divisor is just the smallest divisor of t. And then one can use this algorithm: https://en.wikipedia.org/wiki/Knuth%E2%80%93Morris%E2%80%93Pratt_algorithm.
Is there any simpler solution?
To test for divisibility:
test that the length of s is a multiple of the length of t (otherwise not divisible);
divide s into chunks of length t; and
check that all the chunks are the same.
To find the smallest common divisor, you need to find the shortest repeating substring of t that makes up the whole of t. One approach is:
Find the factors of the length of t (the crude approach of searching from 1 up to sqrt(len(t)) should be fine for strings of any reasonable length);
For each factor (start with the smallest):
i. divide t into chunks of length factor;
ii. check if all the chunks are the same, and return factor if they are.
Using a Python set is a neat way to check if all the chunks in a list are equal. len(set(chunks)) == 1 tells you they are.
The GCD of the string lengths doesn't help you.
A string t "divides" a string s if len(s) is a multiple of len(t) and s == t * (len(s)//len(t)).
As for finding the length of the smallest divisor of t, the classic trick for that is (t+t).index(t, 1): find the first nonzero index of t in t+t. I've used the built-in index method here, but depending on why you're doing this and what runtime properties you want, KMP string search may be a better choice. Manually implementing KMP in Python is going to have a massive constant-factor overhead, but it'll have a much better worst-case runtime than index.
KMP isn't particularly hard to implement, as far as string searches go. You're not going to get anything significantly "simpler" without delegating to a library routine or sacrificing asymptotic complexity properties (or both).
If you want to avoid a search table and keep asymptotic complexity guarantees, you can use something like two-way string matching, but it's not going to be any easier to implement than KMP.
Can anyone help me solve this question in pseudocode?
Consider the function SumKSmallest(A[0..n − 1], k) that returns the sum of the k smallest elements in an unsorted integer array A of size n. For example, given the array A =[6,-6,3,2,1,2,0,4,3,5] and k =3, the function should return -5.
a. Write an algorithm in pseudocode for SumKSmallest using the brute force paradigm. Indicate and justify (within a few sentences) the time complexity of your algorithm.b. Write another algorithm in pseudocode for SumKSmalleast using the transform & conquer paradigm. Your algorithm should strictly run in O(n log n) time. Justify the time complexity of your algorithm. c. Explain with details, how we could implement another SumKSmalleast that strictly runs in less than O(n log n) time, considering k<
Brute force
O(n^2) You got it right.
Second Approach:
Sort the array and sum the first k numbers to get the sum of the smallest K numbers. O(nlog(n)) Merge Sort.
Third Approach:
Heap: Take the first K elements and Max-heapify it. Now iterate over remaining elements taking them one by one and comparing it with the root of max-heap (arr[0]), if (arr[0] > element) remove arr[0] from the heap and add an element into heap. At last, you will be left with K smallest numbers of the array.
O(k + (n-k)log(k))
Read about Min-heap and Max-heap