Given two sequences A and B of the same length: one is strictly increasing, the other is strictly decreasing.
It is required to find an index i such that the absolute value of the difference between A[i] and B[i] is minimal. If there are several such indices, the answer is the smallest of them. The input sequences are standard Python arrays. It is guaranteed that they are of the same length. Efficiency requirements: Asymptotic complexity: no more than the power of the logarithm of the length of the input sequences.
I have implemented index lookup using the golden section method, but I am confused by the use of floating point arithmetic. Is it possible to somehow improve this algorithm so as not to use it, or can you come up with a more concise solution?
import random
import math
def peak(A,B):
def f(x):
return abs(A[x]-B[x])
phi_inv = 1 / ((math.sqrt(5) + 1) / 2)
def cal_x1(left,right):
return right - (round((right-left) * phi_inv))
def cal_x2(left,right):
return left + (round((right-left) * phi_inv))
left, right = 0, len(A)-1
x1, x2 = cal_x1(left, right), cal_x2(left,right)
while x1 < x2:
if f(x1) > f(x2):
left = x1
x1 = x2
x2 = cal_x1(x1,right)
else:
right = x2
x2 = x1
x1 = cal_x2(left,x2)
if x1 > 1 and f(x1-2) <= f(x1-1): return x1-2
if x1+2 < len(A) and f(x1+2) < f(x1+1): return x1+2
if x1 > 0 and f(x1-1) <= f(x1): return x1-1
if x1+1 < len(A) and f(x1+1) < f(x1): return x1+1
return x1
#value check
def make_arr(inv):
x = set()
while len(x) != 1000:
x.add(random.randint(-10000,10000))
x = sorted(list(x),reverse = inv)
return x
x = make_arr(0)
y = make_arr(1)
needle = 1000000
c = 0
for i in range(1000):
if abs(x[i]-y[i]) < needle:
c = i
needle = abs(x[i]-y[i])
print(c)
print(peak(x,y))
Approach
The poster asks about alternative, simpler solutions to posted code.
The problem is a variant of Leetcode Problem 852, where the goal is to find the peak index in a moutain array. We convert to a peak, rather than min, by computing the negative of the abolute difference. Our aproach is to modify this Python solution to the Leetcode problem.
Code
def binary_search(x, y):
''' Mod of https://walkccc.me/LeetCode/problems/0852/ to use function'''
def f(m):
' Absoute value of difference at index m of two arrays '
return -abs(x[m] - y[m]) # Make negative so we are looking for a peak
# peak using binary search
l = 0
r = len(arr) - 1
while l < r:
m = (l + r) // 2
if f(m) < f(m + 1): # check if increasing
l = m + 1
else:
r = m # was decreasing
return l
Test
def linear_search(A, B):
' Linear Search Method '
values = [abs(ai-bi) for ai, bi in zip(A, B)]
return values.index(min(values)) # linear search
def make_arr(inv):
random.seed(10) # added so we can repeat with the same data
x = set()
while len(x) != 1000:
x.add(random.randint(-10000,10000))
x = sorted(list(x),reverse = inv)
return x
# Create data
x = make_arr(0)
y = make_arr(1)
# Run search methods
print(f'Linear Search Solution {linear_search(x, y)}')
print(f'Golden Section Search Solution {peak(x, y)}') # posted code
print(f'Binary Search Solution {binary_search(x, y)}')
Output
Linear Search Solution 499
Golden Section Search Solution 499
Binary Search Solution 499
so im trying to make a program to solve various normal distribution questions with pure python (no modules other than math) to 4 decimal places only for A Levels, and there is this problem that occurs in the function get_z_less_than_a_equal(0.75):. Apparently, without the assert statement in the except clause, the variables all get messed up, and change. The error, i'm catching is the recursion error. Anyways, if there is an easier and more efficient way to do things, it'd be appreciated.
import math
mean = 0
standard_dev = 1
percentage_points = {0.5000: 0.0000, 0.4000: 0.2533, 0.3000: 0.5244, 0.2000: 0.8416, 0.1000: 1.2816, 0.0500: 1.6440, 0.0250: 1.9600, 0.0100: 2.3263, 0.0050: 2.5758, 0.0010: 3.0902, 0.0005: 3.2905}
def get_z_less_than(x):
"""
P(Z < x)
"""
return round(0.5 * (1 + math.erf((x - mean)/math.sqrt(2 * standard_dev**2))), 4)
def get_z_greater_than(x):
"""
P(Z > x)
"""
return round(1 - get_z_less_than(x), 4)
def get_z_in_range(lower_bound, upper_bound):
"""
P(lower_bound < Z < upper_bound)
"""
return round(get_z_less_than(upper_bound) - get_z_less_than(lower_bound), 4)
def get_z_less_than_a_equal(x):
"""
P(Z < a) = x
acquires a, given x
"""
# first trial: brute forcing
for i in range(401):
a = i/100
p = get_z_less_than(a)
if x == p:
return a
elif p > x:
break
# second trial: using symmetry
try:
res = -get_z_less_than_a_equal(1 - x)
except:
# third trial: using estimation
assert a, "error"
prev = get_z_less_than(a-0.01)
p = get_z_less_than(a)
if abs(x - prev) > abs(x - p):
res = a
else:
res = a - 0.01
return res
def get_z_greater_than_a_equal(x):
"""
P(Z > a) = x
"""
if x in percentage_points:
return percentage_points[x]
else:
return get_z_less_than_a_equal(1-x)
print(get_z_in_range(-1.20, 1.40))
print(get_z_less_than_a_equal(0.7517))
print(get_z_greater_than_a_equal(0.1000))
print(get_z_greater_than_a_equal(0.0322))
print(get_z_less_than_a_equal(0.1075))
print(get_z_less_than_a_equal(0.75))
Since python3.8, the statistics module in the standard library has a NormalDist class, so we could use that to implement our functions "with pure python" or at least for testing:
import math
from statistics import NormalDist
normal_dist = NormalDist(mu=0, sigma=1)
for i in range(-2000, 2000):
test_val = i / 1000
assert get_z_less_than(test_val) == round(normal_dist.cdf(test_val), 4)
Doesn't throw an error, so that part probably works fine
Your get_z_less_than_a_equal seems to be the equivalent of NormalDist.inv_cdf
There are very efficient ways to compute it accurately using the inverse of the error function (see Wikipedia and Python implementation), but we don't have that in the standard library
Since you only care about the first few digits and get_z_less_than is monotonic, we can use a simple bisection method to find our solution
Newton's method would be much faster, and not too hard to implement since we know that the derivative of the cdf is just the pdf, but still probably more complex than what we need
def get_z_less_than_a_equal(x):
"""
P(Z < a) = x
acquires a, given x
"""
if x <= 0.0 or x >= 1.0:
raise ValueError("x must be >0.0 and <1.0")
min_res, max_res = -10, 10
while max_res - min_res > 1e-7:
mid = (max_res + min_res) / 2
if get_z_less_than(mid) < x:
min_res = mid
else:
max_res = mid
return round((max_res + min_res) / 2, 4)
Let's test this:
for i in range(1, 2000):
test_val = i / 2000
left_val = get_z_less_than_a_equal(test_val)
right_val = round(normal_dist.inv_cdf(test_val), 4)
assert left_val == right_val, f"{left_val} != {right_val}"
# AssertionError: -3.3201 != -3.2905
We see that we are losing some precision, that's because the error introduced by get_z_less_than (which rounds to 4 digits) gets propagated and amplified when we use it to estimate its inverse (see Wikipedia - error propagation for details)
So let's add a "digits" parameter to get_z_less_than and change our functions slightly:
def get_z_less_than(x, digits=4):
"""
P(Z < x)
"""
res = 0.5 * (1 + math.erf((x - mean) / math.sqrt(2 * standard_dev ** 2)))
return round(res, digits)
def get_z_less_than_a_equal(x, digits=4):
"""
P(Z < a) = x
acquires a, given x
"""
if x <= 0.0 or x >= 1.0:
raise ValueError("x must be >0.0 and <1.0")
min_res, max_res = -10, 10
while max_res - min_res > 10 ** -(digits * 2):
mid = (max_res + min_res) / 2
if get_z_less_than(mid, digits * 2) < x:
min_res = mid
else:
max_res = mid
return round((max_res + min_res) / 2, digits)
And now we can try the same test again and see it passes
I'm new to Python, and I'm trying to get familiar with it by solving problems on CodeChef. I'm attempting to solve the Easy problem Number Game. The issue is that the execution time is too long for my code.
I have translated the Python solution I wrote into C++, and the submission was accepted, so I know I have a correct answer, and it's just off by a constant multiple.
Is it possible to solve this problem in Python 3 in the allotted time? Can you help me speed up my code to accomplish this?
import time
def getStartValues(A, M):
startVals = [0]*M
b = [0]*len(A)
for i in range(len(A)-1):
b[i+1] = (10*b[i] + A[i]) % M
f = 0
power = 1
for i in range(len(A)-1,0,-1):
startVals[(b[i]*power + f) % M] += 1
f = (A[i]*power + f) % M
power = (power*10 % M)
startVals[f] += 1
return startVals, power
def checkValues(i, startVals, M, powNm1, checked, chklst):
if checked[i] == 1:
return startVals[i]
q = [i]
chk = [0]*M
chk[i] = 1
while len(q) > 0:
val = q.pop(0)
for j in chklst:
val2 = (powNm1*val + j) % M
if checked[val2] > 0:
checked[i] = 1
return startVals[i]
elif chk[val2] == 0:
q.append(val2)
chk[val2] = 1
return 0
def compute(A, M):
startVals, power = getStartValues(A, M)
checked = [0]*M
checked[0] = 1
chklst = [j for j in range(M) if startVals[j] > 0]
total = 0
for i in chklst:
c = checkValues(i, startVals, M, power, checked, chklst)
total += c
return total
start = time.time()
file = open('numbgame.in', 'r')
#T = int(input())
T = int(file.readline())
for i in range(T):
#A, M = input().split()
A, M = file.readline().split()
A = list(map(int,A))
M = int(M)
print(compute(A, M))
tDiff = time.time() - start
print('Total time: %s' % tDiff)
Note that I have modified the code to read from a file and to display execution time, as a convenience, and some small alterations are needed before it can be submitted.
getStartValues takes in the (big) list of digits of the input A and the (small) integer M and returns the values modulo M that can be generated from A by removing a single digit.
checkValues takes an index i, the list startValues, the integer M, the integer powNm1 (which is the value 10^(n-1) mod M, where n is the number of digits in A, a list checked that keeps track of whether a value has already been determined to be solvable, and the list chklst (which contains the indices i such that startValues[i] > 0).
The majority of the time is spent in the function getStartValues, since A could be up to 10^6 digits long. On my desktop, the getStartValues function call takes about 1.2s, while the rest of the compute function takes about 0.04s (for worst case inputs).
I am trying this problem for a while but getting wrong answer again and again.
number can be very large <=2^2014.
22086. Prime Power Test
Explanation about my algorithm:
For a Given number I am checking if the number can be represented as form of prime power or not.
So the the maximum limit to check for prime power is log n base 2.
Finally problem reduced to finding nth root of a number and if it is prime we have our answer else check for all i till log (n base 2) and exit.
I have used all sort of optimizations and have tested enormous test-cases and for all my algorithm gives correct answer
but Judge says wrong answer.
Spoj have another similar problem with small constraints n<=10^18 for which I already got accepted with Python and C++(Best solver in c++)
Here is My python code Please suggest me if I am doing something wrong I am not very proficient in python so my algorithm is a bit lengthy. Thanks in advance.
My Algorithm:
import math
import sys
import fractions
import random
import decimal
write = sys.stdout.write
def sieve(n):
sqrtn = int(n**0.5)
sieve = [True] * (n+1)
sieve[0] = False
sieve[1] = False
for i in range(2, sqrtn+1):
if sieve[i]:
m = n//i - i
sieve[i*i:n+1:i] = [False] * (m+1)
return sieve
def gcd(a, b):
while b:
a, b = b, a%b
return a
def mr_pass(a, s, d, n):
a_to_power = pow(a, d, n)
if a_to_power == 1:
return True
for i in range(s-1):
if a_to_power == n - 1:
return True
a_to_power = (a_to_power * a_to_power) % n
return a_to_power == n - 1
isprime=sieve(1000000)
sprime= [2,3,5,7,11,13,17,19,23,29,31,37,41,43,47,53,59,61,67,71,73,79,83,89,97,101,103,107,109,113,127,131,137,139,149,151,157,163,167,173,179,181,191,193,197,199,211,223,227,229,233,239,241,251,257,263,269,271,277,281,283,293,307,311,313,317,331,337,347,349,353,359,367,373,379,383,389,397,401,409,419,421,431,433,439,443,449,457,461,463,467,479,487,491,499,503,509,521,523,541,547,557,563,569,571,577,587,593,599,601,607,613,617,619,631,641,643,647,653,659,661,673,677,683,691,701,709,719,727,733,739,743,751,757,761,769,773,787,797,809,811,821,823,827,829,839,853,857,859,863,877,881,883,887,907,911,919,929,937,941,947,953,967,971,977,983,991,997]
def smooth_num(n):
c=0
for a in sprime:
if(n%a==0):
c+=1
if(c>=2):
return True;
return False
def is_prime(n):
if(n<1000000):
return isprime[n]
if any((n % p) == 0 for p in sprime):
return False
if n==2:
return True
d = n - 1
s = 0
while d % 2 == 0:
d >>= 1
s += 1
for repeat in range(10):
a=random.randint(1,n-1)
if not mr_pass(a, s, d, n):
return False
return True
def iroot(n,k):
hi = 1
while pow(hi, k) < n:
hi *= 2
lo = hi // 2
while hi - lo > 1:
mid = (lo + hi) // 2
midToK = (mid**k)
if midToK < n:
lo = mid
elif n < midToK:
hi = mid
else:
return mid
if (hi**k) == n:
return hi
else:
return lo
def isqrt(x):
n = int(x)
if n == 0:
return 0
a, b = divmod(n.bit_length(), 2)
x = pow(2,(a+b))
while True:
y = (x + n//x)>>1
if y >= x:
return x
x = y
maxx=2**1024;minn=2**64
def nth_rootp(n,k):
return int(round(math.exp(math.log(n)/k),0))
def main():
for cs in range(int(input())):
n=int(sys.stdin.readline().strip())
if(smooth_num(n)):
write("Invalid order\n")
continue;
order = 0;m=0
power =int(math.log(n,2))
for i in range(1,power+1):
if(n<=maxx):
if i==1:m=n
elif(i==2):m=isqrt(n)
elif(i==4):m=isqrt(isqrt(n))
elif(i==8):m=isqrt(isqrt(isqrt(n)))
elif(i==16):m=isqrt(isqrt(isqrt(isqrt(n))))
elif(i==32):m=isqrt(isqrt(isqrt(isqrt(isqrt(n)))))
elif(i==64):m=isqrt(isqrt(isqrt(isqrt(isqrt(isqrt(n))))))
elif(i==128):m=isqrt(isqrt(isqrt(isqrt(isqrt(isqrt(isqrt(n)))))))
elif(i==256):m=isqrt(isqrt(isqrt(isqrt(isqrt(isqrt(isqrt(isqrt(n))))))))
else:m=int(nth_rootp(n,i))
else:
if i==1:m=n
elif i==2:m=isqrt(n)
elif(i==4):m=isqrt(isqrt(n))
elif(i==8):m=isqrt(isqrt(isqrt(n)))
elif(i==16):m=isqrt(isqrt(isqrt(isqrt(n))))
elif(i==32):m=isqrt(isqrt(isqrt(isqrt(isqrt(n)))))
elif(i==64):m=isqrt(isqrt(isqrt(isqrt(isqrt(isqrt(n))))))
elif(i==128):m=isqrt(isqrt(isqrt(isqrt(isqrt(isqrt(isqrt(n)))))))
elif(i==256):m=isqrt(isqrt(isqrt(isqrt(isqrt(isqrt(isqrt(isqrt(n))))))))
else:m=iroot(n,i)
if m<2:
order=0
break
if(is_prime(m) and n==(m**i)):
write("%d %d\n"%(m,i))
order = 1
break
if(order==0):
write("Invalid order\n")
main()
I'm not going to read all that code, though I suspect the problem is floating-point inaccuracy. Here is my program to determine if a number n is a prime power; it returns the prime p and the power k:
# prime power predicate
from random import randint
from fractions import gcd
def findWitness(n, k=5): # miller-rabin
s, d = 0, n-1
while d % 2 == 0:
s, d = s+1, d/2
for i in range(k):
a = randint(2, n-1)
x = pow(a, d, n)
if x == 1 or x == n-1: continue
for r in range(1, s):
x = (x * x) % n
if x == 1: return a
if x == n-1: break
else: return a
return 0
# returns p,k such that n=p**k, or 0,0
# assumes n is an integer greater than 1
def primePower(n):
def checkP(n, p):
k = 0
while n > 1 and n % p == 0:
n, k = n / p, k + 1
if n == 1: return p, k
else: return 0, 0
if n % 2 == 0: return checkP(n, 2)
q = n
while True:
a = findWitness(q)
if a == 0: return checkP(n, q)
d = gcd(pow(a,q,n)-a, q)
if d == 1 or d == q: return 0, 0
q = d
The program uses Fermat's Little Theorem and exploits the witness a to the compositeness of n that is found by the Miller-Rabin algorithm. It is given as Algorithm 1.7.5 in Henri Cohen's book A Course in Computational Algebraic Number Theory. You can see the program in action at http://ideone.com/cNzQYr.
this is not really an answer, but I don't have enough space to write it as a comment.
So, if the problem still not solved, you may try the following function for nth_rootp, though it is a bit ugly (it is just a binary search to find the precise value of the function):
def nth_rootp(n,k):
r = int(round(math.log(n,2)/k))
left = 2**(r-1)
right = 2**(r+1)
if left**k == n:
return left
if right**k == n:
return right
while left**k < n and right**k > n:
tmp = (left + right)/2
if tmp**k == n:
return tmp
if tmp == left or tmp == right:
return tmp
if tmp**k < n:
left = tmp
else:
if tmp**k > n:
right = tmp
your code look like a little overcomplicated for this task, I will not bother to check it, but the thing you need are the following
is_prime, naturally
a prime generator, optional
calculate the nth root of a number in a precise way
for the first one I recommend the deterministic form of the Miller-Rabin test with a appropriate set of witness to guaranty a exact result until 1543267864443420616877677640751301 (1.543 x 1033) for even bigger numbers you can use the probabilistic one or use a bigger list of witness chosen at your criteria
with all that a template for the solution is as follow
import math
def is_prime(n):
...
def sieve(n):
"list of all primes p such that p<n"
...
def inthroot(x,n):
"calculate floor(x**(1/n))"
...
def is_a_power(n):
"return (a,b) if n=a**b otherwise throw ValueError"
for b in sieve( math.log2(n) +1 ):
a = inthroot(n,b)
if a**b == n:
return a,b
raise ValueError("is not a power")
def smooth_factorization(n):
"return (p,e) where p is prime and n = p**e if such value exists, otherwise throw ValueError"
e=1
p=n
while True:
try:
p,n = is_a_power(p)
e = e*n
except ValueError:
break
if is_prime(p):
return p,e
raise ValueError
def main():
for test in range( int(input()) ):
try:
p,e = smooth_factorization( int(input()) )
print(p,e)
except ValueError:
print("Invalid order")
main()
And the code above should be self explanatory
Filling the blacks
As you are familiar with Miller-Rabin test, I will only mention that if you are interested you can find a implementation of the determinist version here just update the list of witness and you are ready to go.
For the sieve, just change the one you are using to return a list with primes number like this for instance [ p for p,is_p in enumerate(sieve) if is_p ]
With those out of the way, the only thing left is calculate the nth root of the number and to do that in a precise way we need to get rip of that pesky floating point arithmetic that only produce headaches, and the answer is implement the Nth root algorithm using only integer arithmetic, which is pretty similar to the one of isqrt that you already use, I guide myself with the one made by Mark Dickinson for cube root and generalize it and I get this
def inthroot(A, n) :
"calculate floor( A**(1/n) )"
#https://en.wikipedia.org/wiki/Nth_root_algorithm
#https://en.wikipedia.org/wiki/Nth_root#nth_root_algorithm
#https://stackoverflow.com/questions/35254566/wrong-answer-in-spoj-cubert/35276426#35276426
#https://stackoverflow.com/questions/39560902/imprecise-results-of-logarithm-and-power-functions-in-python/39561633#39561633
if A<0:
if n%2 == 0:
raise ValueError
return - inthroot(-A,n)
if A==0:
return 0
n1 = n-1
if A.bit_length() < 1024: # float(n) safe from overflow
xk = int( round( pow(A,1.0/n) ) )
xk = ( n1*xk + A//pow(xk,n1) )//n # Ensure xk >= floor(nthroot(A)).
else:
xk = 1 << -(-A.bit_length()//n) # 1 << sum(divmod(A.bit_length(),n))
# power of 2 closer but greater than the nth root of A
while True:
sig = A // pow(xk,n1)
if xk <= sig:
return xk
xk = ( n1*xk + sig )//n
and with all the above you can solve the problem without inconvenient
from sympy.ntheory import factorint
q=int(input("Give me the number q="))
fact=factorint(q) #We factor the number q=p_1^{n_1}*p_2^{n_2}*...
p_1=list(fact.keys()) #We create a list from keys to be the the numbers p_1,p_2,...
n_1=list(fact.values()) #We create a list from values to be the the numbers n_1,n_2,...
p=int(p_1[0])
n=int(n_1[0])
if q!=p**n: #Check if the number q=p_{1}[0]**n_{1}[0]=p**n.
print("The number "+str(q)+" is not a prime power")
else:
print("The number "+str(q)+" is a prime power")
print("The prime number p="+str(p))
print("The natural number n="+str(n))