Calculating a^b mod p for a large prime p - python

I'm trying to write a python code that calculates a^b mod p, where p = 10^9+7 for a list of pairs (a,b). The challenge is that the code has to finish the calculation and output the result in < 1 second. I've implemented successive squaring to calculate a^b mod p quickly. Please see my code below:
from sys import stdin, stdout
rl = stdin.readline
wo = stdout.write
m = 10**9+7
def fn(a,n):
t = 1
while n > 0:
if n%2 != 0: #exponent is odd
t = t*a %m
a = a*a %m
n = int(n/2)
return t%m
t = int(rl()) # number of pairs
I = stdin.read().split() # reading all pairs
I = list(map(int,I)) # making pairs a list of integers
# calculating a^b mod p. I used map because I read its faster than a for loop
s = list(map(fn,I[0:2*t:2],I[1:2*t:2]))
stdout.write('\n'.join(map(str,s))) # printing output
for 200000 pairs (a,b) with a,b<10^9, my code takes > 1 second. I'm new to python and was hoping someone could help me identify the time bottle neck in my code. Is it reading input and printing output or the calculation itself? Thanks for the help!

I don't see something wrong with your code from an efficiency standpoint, it's just unnecessarily complicated.
Here's what I'd call the straight-forward solution:
n = int(input())
for _ in range(n):
a, b = map(int, input().split())
print(pow(a, b, 10**9 + 7))
That did get accepted with PyPy3 but not with CPython3. And with PyPy3 it still took 0.93 seconds.
I'd say their time limit is inappropriate for Python. But try yours with PyPy3 if you haven't yet.
In case someone's wondering whether the map wastes time, the following got accepted in 0.92 seconds:
n = int(input())
for _ in range(n):
a, b = input().split()
print(pow(int(a), int(b), 10**9 + 7))

Related

Primitive Calculator - Dynamic & Recursive approach

I have tried to solve the Primitive calculator problem with dynamic and recursive approach , works fine for smaller inputs but taking long time for larger inputs (eg: 96234) .
You are given a primitive calculator that can perform the following three operations with
the current number 𝑥: multiply 𝑥 by 2, multiply 𝑥 by 3, or add 1 to 𝑥. Your goal is given a
positive integer 𝑛, find the minimum number of operations needed to obtain the number 𝑛
starting from the number 1.
import sys
def optimal_sequence(n,memo={}):
if n in memo:
return memo[n]
if (n==1):
return 0
c1 = 1+optimal_sequence(n-1,memo)
c2 = float('inf')
if n % 2 == 0 :
c2 = 1+optimal_sequence(n // 2,memo)
c3 = float('inf')
if n % 3 == 0 :
c3 = 1+optimal_sequence(n // 3,memo)
c = min(c1,c2,c3)
memo[n] = c
return c
input = sys.stdin.read()
n = int(input)
sequence = optimal_sequence(n)
print(sequence) # Only printing optimal no. of operations
Can anyone point out what is wrong in recursive solution as it works fine by using for loop.
There are a few things to consider here. The first is that you always check if you can subtract 1 away from n. This is always going to be true until n is 1. therefor with a number like 12. You will end up taking 1 away first, then calling the function again with n=11, then n=10, then n=9 etc......only once you have resolved how many steps it will take to resolve using the -1 method (in this case c1 will be 11) you then try for c2.
So for c2 you then half 12, then call the function which will start with the -1 again so you end up with n=12, n=6, n=5, n=4...etc. Even though you have n in the memo you still spend a lot of wasted time on function calls.
Instead you probably want to just shrink the problem space as fast as possible. So start with the rule that will reduce n the most. I.E divide by 3, if that doesnt work then divide by 2, only if neither of the first two worked then subtract 1.
With this method you dont even need to track n as n will always be getting smaller so there is no need to have a memo dict tracking the results.
from time import time
def optimal_sequence(n):
if n == 1:
return 0
elif n % 3 == 0:
c = optimal_sequence(n // 3)
elif n % 2 == 0:
c = optimal_sequence(n // 2)
else:
c = optimal_sequence(n - 1)
return 1 + c
n = int(input("Enter a value for N: "))
start = time()
sequence = optimal_sequence(n)
end = time()
print(f"{sequence=} took {end - start} seconds")
also input is a python function to read from teh terminal you dont need to uses stdin
OUTPUT
Enter a value for N: 96234
sequence=15 took 0.0 seconds

Optimizing Totient function

I'm trying to maximize the Euler Totient function on Python given it can use large arbitrary numbers. The problem is that the program gets killed after some time so it doesn't reach the desired ratio. I have thought of increasing the starting number into a larger number, but I don't think it's prudent to do so. I'm trying to get a number when divided by the totient gets higher than 10. Essentially I'm trying to find a sparsely totient number that fits this criteria.
Here's my phi function:
def phi(n):
amount = 0
for k in range(1, n + 1):
if fractions.gcd(n, k) == 1:
amount += 1
return amount
The most likely candidates for high ratios of N/phi(N) are products of prime numbers. If you're just looking for one number with a ratio > 10, then you can generate primes and only check the product of primes up to the point where you get the desired ratio
def totientRatio(maxN,ratio=10):
primes = []
primeProd = 1
isPrime = [1]*(maxN+1)
p = 2
while p*p<=maxN:
if isPrime[p]:
isPrime[p*p::p] = [0]*len(range(p*p,maxN+1,p))
primes.append(p)
primeProd *= p
tot = primeProd
for f in primes:
tot -= tot//f
if primeProd/tot >= ratio:
return primeProd,primeProd/tot,len(primes)
p += 1 + (p&1)
output:
totientRatio(10**6)
16516447045902521732188973253623425320896207954043566485360902980990824644545340710198976591011245999110,
10.00371973209101,
55
This gives you the smallest number with that ratio. Multiples of that number will have the same ratio.
n = 16516447045902521732188973253623425320896207954043566485360902980990824644545340710198976591011245999110
n*2/totient(n*2) = 10.00371973209101
n*11*13/totient(n*11*13) = 10.00371973209101
No number will have a higher ratio until you reach the next product of primes (i.e. that number multiplied by the next prime).
n*263/totient(n*263) = 10.041901868473037
Removing a prime from the product affects the ratio by a proportion of (1-1/P).
For example if m = n/109, then m/phi(m) = n/phi(n) * (1-1/109)
(n//109) / totient(n//109) = 9.91194248684247
10.00371973209101 * (1-1/109) = 9.91194248684247
This should allow you to navigate the ratios efficiently and find the numbers that meed your need.
For example, to get a number with a ratio that is >= 10 but closer to 10, you can go to the next prime product(s) and remove one or more of the smaller primes to reduce the ratio. This can be done using combinations (from itertools) and will allow you to find very specific ratios:
m = n*263/241
m/totient(m) = 10.000234225865265
m = n*(263...839) / (7 * 61 * 109 * 137) # 839 is 146th prime
m/totient(m) = 10.000000079805726
I have a partial solution for you, but the results don't look good.. (this solution may not give you an answer with modern computer hardware (amount of ram is limiting currently)) I took an answer from this pcg challenge and modified it to spit out ratios of n/phi(n) up to a particular n
import numba as nb
import numpy as np
import time
n = int(2**31)
#nb.njit("i4[:](i4[:])", locals=dict(
n=nb.int32, i=nb.int32, j=nb.int32, q=nb.int32, f=nb.int32))
def summarum(phi):
#calculate phi(i) for i: 1 - n
#taken from <a>https://codegolf.stackexchange.com/a/26753/42652</a>
phi[1] = 1
i = 2
while i < n:
if phi[i] == 0:
phi[i] = i - 1
j = 2
while j * i < n:
if phi[j] != 0:
q = j
f = i - 1
while q % i == 0:
f *= i
q //= i
phi[i * j] = f * phi[q]
j += 1
i += 1
#divide each by n to get ratio n/phi(n)
i = 1
while i < n: #jit compiled while loop is faster than: for i in range(): blah blah blah
phi[i] = i//phi[i]
i += 1
return phi
if __name__ == "__main__":
s1 = time.time()
a = summarum(np.zeros(n, np.int32))
locations = np.where(a >= 10)
print(len(locations))
I only have enough ram on my work comp. to test about 0 < n < 10^8 and the largest ratio was about 6. You may or may not have any luck going up to larger n, although 10^8 already took several seconds (not sure what the overhead was... spyder's been acting strange lately)
p55# is a sparsely totient number satisfying the desired condition.
Furthermore, all subsequent primorial numbers are as well, because pn# / phi(pn#) is a strictly increasing sequence:
p1# / phi(p1#) is 2, which is positive. For n > 1, pn# / phi(pn#) is equal to pn-1#pn / phi(pn-1#pn), which, since pn and pn-1# are coprime, is equal to (pn-1# / phi(pn-1#)) * (pn/phi(pn)). We know pn > phi(pn) > 0 for all n, so pn/phi(pn) > 1. So we have that the sequence pn# / phi(pn#) is strictly increasing.
I do not believe these to be the only sparsely totient numbers satisfying your request, but I don't have an efficient way of generating the others coming to mind. Generating primorials, by comparison, amounts to generating the first n primes and multiplying the list together (whether by using functools.reduce(), math.prod() in 3.8+, or ye old for loop).
As for the general question of writing a phi(n) function, I would probably first find the prime factors of n, then use Euler's product formula for phi(n). As an aside, make sure to NOT use floating-point division. Even finding the prime factors of n by trial division should outperform computing gcd n times, but when working with large n, replacing this with an efficient prime factorization algorithm will pay dividends. Unless you want a good cross to die on, don't write your own. There's one in sympy that I'm aware of, and given the ubiquity of the problem, probably plenty of others around. Time as needed.
Speaking of timing, if this is still relevant enough to you (or a future reader) to want to time... definitely throw the previous answer in the mix as well.

Python: Streamlining Code for Brown Numbers

I was curious if any of you could come up with a more streamline version of code to calculate Brown numbers. as of the moment, this code can do ~650! before it moves to a crawl. Brown Numbers are calculated thought the equation n! + 1 = m**(2) Where M is an integer
brownNum = 8
import math
def squareNum(n):
x = n // 2
seen = set([x])
while x * x != n:
x = (x + (n // x)) // 2
if x in seen: return False
seen.add(x)
return True
while True:
for i in range(math.factorial(brownNum)+1,math.factorial(brownNum)+2):
if squareNum(i) is True:
print("pass")
print(brownNum)
print(math.factorial(brownNum)+1)
break
else:
print(brownNum)
print(math.factorial(brownNum)+1)
brownNum = brownNum + 1
continue
break
print(input(" "))
Sorry, I don't understand the logic behind your code.
I don't understand why you calculate math.factorial(brownNum) 4 times with the same value of brownNum each time through the while True loop. And in the for loop:
for i in range(math.factorial(brownNum)+1,math.factorial(brownNum)+2):
i will only take on the value of math.factorial(brownNum)+1
Anyway, here's my Python 3 code for a brute force search of Brown numbers. It quickly finds the only 3 known pairs, and then proceeds to test all the other numbers under 1000 in around 1.8 seconds on this 2GHz 32 bit machine. After that point you can see it slowing down (it hits 2000 around the 20 second mark) but it will chug along happily until the factorials get too large for your machine to hold.
I print progress information to stderr so that it can be separated from the Brown_number pair output. Also, stderr doesn't require flushing when you don't print a newline, unlike stdout (at least, it doesn't on Linux).
import sys
# Calculate the integer square root of `m` using Newton's method.
# Returns r: r**2 <= m < (r+1)**2
def int_sqrt(m):
if m <= 0:
return 0
n = m << 2
r = n >> (n.bit_length() // 2)
while True:
d = (n // r - r) >> 1
r += d
if -1 <= d <= 1:
break
return r >> 1
# Search for Browns numbers
fac = i = 1
while True:
if i % 100 == 0:
print('\r', i, file=sys.stderr, end='')
fac *= i
n = fac + 1
r = int_sqrt(n)
if r*r == n:
print('\nFound', i, r)
i += 1
You might want to:
pre calculate your square numbers, instead of testing for them on the fly
pre calculate your factorial for each loop iteration num_fac = math.factorial(brownNum) instead of multiple calls
implement your own, memoized, factorial
that should let you run to the hard limits of your machine
one optimization i would make would be to implement a 'wrapper' function around math.factorial that caches previous values of factorial so that as your brownNum increases, factorial doesn't have as much work to do. this is known as 'memoization' in computer science.
edit: found another SO answer with similar intention: Python: Is math.factorial memoized?
You should also initialize the square root more closely to the root.
e = int(math.log(n,4))
x = n//2**e
Because of 4**e <= n <= 4**(e+1) the square root will be between x/2 and x which should yield quadratic convergence of the Heron formula from the first iteration on.

Optimise the solution to Project Euler 12 (Python)

I have the following code for Project Euler Problem 12. However, it takes a very long time to execute. Does anyone have any suggestions for speeding it up?
n = input("Enter number: ")
def genfact(n):
t = []
for i in xrange(1, n+1):
if n%i == 0:
t.append(i)
return t
print "Numbers of divisors: ", len(genfact(n))
print
m = input("Enter the number of triangle numbers to check: ")
print
for i in xrange (2, m+2):
a = sum(xrange(i))
b = len(genfact(a))
if b > 500:
print a
For n, I enter an arbitrary number such as 6 just to check whether it indeed returns the length of the list of the number of factors.
For m, I enter entered 80 000 000
It works relatively quickly for small numbers. If I enter b > 50 ; it returns 28 for a, which is correct.
My answer here isn't pretty or elegant, it is still brute force. But, it simplifies the problem space a little and terminates successfully in less than 10 seconds.
Getting factors of n:
Like #usethedeathstar mentioned, it is possible to test for factors only up to n/2. However, we can do better by testing only up to the square root of n:
let n = 36
=> factors(n) : (1x36, 2x18, 3x12, 4x9, 6x6, 9x4, 12x3, 18x2, 36x1)
As you can see, it loops around after 6 (the square root of 36). We also don't need to explicitly return the factors, just find out how many there are... so just count them off with a generator inside of sum():
import math
def get_factors(n):
return sum(2 for i in range(1, round(math.sqrt(n)+1)) if not n % i)
Testing the triangular numbers
I have used a generator function to yield the triangular numbers:
def generate_triangles(limit):
l = 1
while l <= limit:
yield sum(range(l + 1))
l += 1
And finally, start testing:
def test_triangles():
triangles = generate_triangles(100000)
for i in triangles:
if get_factors(i) > 499:
return i
Running this with the profiler, it completes in less than 10 seconds:
$ python3 -m cProfile euler12.py
361986 function calls in 8.006 seconds
The BIGGEST time saving here is get_factors(n) testing only up to the square root of n - this makes it heeeaps quicker and you save heaps of memory overhead by not generating a list of factors.
As I said, it still isn't pretty - I am sure there are more elegant solutions. But, it fits the bill of being faster :)
I got my answer to run in 1.8 seconds with Python.
import time
from math import sqrt
def count_divisors(n):
d = {}
count = 1
while n % 2 == 0:
n = n / 2
try:
d[2] += 1
except KeyError:
d[2] = 1
for i in range(3, int(sqrt(n+1)), 2):
while n % i == 0 and i != n:
n = n / i
try:
d[i] += 1
except KeyError:
d[i] = 1
d[n] = 1
for _,v in d.items():
count = count * (v + 1)
return count
def tri_number(num):
next = 1 + int(sqrt(1+(8 * num)))
return num + (next/2)
def main():
i = 1
while count_divisors(i) < 500:
i = tri_number(i)
return i
start = time.time()
answer = main()
elapsed = (time.time() - start)
print("result %s returned in %s seconds." % (answer, elapsed))
Here is the output showing the timedelta and correct answer:
$ python ./project012.py
result 76576500 returned in 1.82238006592 seconds.
Factoring
For counting the divisors, I start by initializing an empty dictionary and a counter. For each factor found, I create key of d[factor] with value of 1 if it does not exist, otherwise, I increment the value d[factor].
For example, if we counted the factors 100, we would see d = {25: 1, 2: 2}
The first while loop, I factor out all 2's, dividing n by 2 each time. Next, I begin factoring at 3, skipping two each time (since we factored all even numbers already), and stopping once I get to the square root of n+1.
We stop at the square_root of n because if there's a pair of factors with one of the numbers bigger than square_root of n, the other of the pair has to be less than 10. If the smaller one doesn't exist, there is no matching larger factor.
https://math.stackexchange.com/questions/1343171/why-only-square-root-approach-to-check-number-is-prime
while n % 2 == 0:
n = n / 2
try:
d[2] += 1
except KeyError:
d[2] = 1
for i in range(3, int(sqrt(n+1)), 2):
while n % i == 0 and i != n:
n = n / i
try:
d[i] += 1
except KeyError:
d[i] = 1
d[n] = 1
Now that I have gotten each factor, and added it to the dictionary, we have to add the last factor (which is just n).
Counting Divisors
Now that the dictionary is complete, we loop through each of the items, and apply the following formula: d(n)=(a+1)(b+1)(c+1)...
https://www.wikihow.com/Determine-the-Number-of-Divisors-of-an-Integer
All this formula means is taking all of the counts of each factor, adding 1, then multiplying them together. Take 100 for example, which has factors 25, 2, and 2. We would calculate d(n)=(a+1)(b+1) = (1+1)(2+1) = (2)(3) = 6 total divisors
for _,v in d.items():
count = count * (v + 1)
return count
Calculate Triangle Numbers
Now, taking a look at tri_number(), you can see that I opted to calculate the next triangle number in a sequence without manually adding each whole number together (saving me millions of operations). Instead I used T(n) = n (n+1) / 2
http://www.maths.surrey.ac.uk/hosted-sites/R.Knott/runsums/triNbProof.html
We are providing a whole number to the function as an argument, so we need to solve for n, which is going to be the whole number to add next. Once we have the next number (n), we simply add that single number to num and return
S=n(n+1)2
S=n2+n2
2S=n2+n
n2+n−2S=0
At this point, we use the quadratic formula for : ax2+bx+c=0.
n=−b±√b2−4ac / 2a
n=−1±√1−4(1)(−2S) / 2
n=−1±√1+8S / 2
https://socratic.org/questions/how-do-you-solve-for-n-in-s-n-n-1-2
So all tri_number() does is evaluate n=1+√1+8S / 2 (we ignore the negative equation here). The answer that is returned is the next triangle number in the sequence.
def tri_number(num):
next = 1 + int(sqrt(1+(8 * num)))
return num + (next/2)
Main Loop
Finally, we can look at main(). We start at whole number 1. We count the divisor of 1. If it is less than 500, we get the next triangle number, then try again and again until we get a number with > 500 divisors.
def main():
i = 1
while count_divisors(i) < 500:
i = tri_number(i)
return i
I am sure there are additional ways to optimize but I am not smart enough to understand those ways. If you find any better ways to optimize python, let me know! I originally solved project 12 in Golang, and that run in 25 milliseconds!
$ go run project012.go
76576500
2018/07/12 01:56:31 TIME: main() took 23.581558ms
one of the hints i can give is
def genfact(n):
t = []
for i in xrange(1, n+1):
if n%i == 0:
t.append(i)
return t
change that to
def genfact(n):
t=[]
for i in xrange(1,numpy.sqrt(n)+1):
if(n%i==0):
t.append(i)
t.apend(n/i)
since if a is a divisor than so is b=n/a, since a*b=a*n/b=n, That should help a part already (not sure if in your case a square is possible, but if so, add another case to exclude adding the same number twice)
You could devise a recursive thing too, (like if it is something like for 28, you get 1,28,2,14 and at the moment you are at knowing 14, you put in something to actually remember the divisors of 14 (memoize), than check if they are alraedy in the list, and if not, add them to the list, together with 28/d for each of the divisors of 14, and at the end just take out the duplicates
If you think my first answer is still not fast enough, ask for more, and i will check how it would be done to solve it faster with some more tricks (could probably make use of erastothenes sieve or so too, and some other tricks could be thought up as well if you would wish to really blow up the problem to huge proportions, like to check the first one with over 10k divisors or so)
while True:
c=0
n=1
m=1
for i in range(1,n+1):
if n%i==0:
c=c+1
m=m+1
n=m*(m+1)/2
if c>500:
break
print n
this is not my code but it is so optimized.
source: http://code.jasonbhill.com/sage/project-euler-problem-12/
import time
def num_divisors(n):
if n % 2 == 0: n = n / 2
divisors = 1
count = 0
while n % 2 == 0:
count += 1
n = n / 2
divisors = divisors * (count + 1)
p = 3
while n != 1:
count = 0
while n % p == 0:
count += 1
n = n / p
divisors = divisors * (count + 1)
p += 2
return divisors
def find_triangular_index(factor_limit):
n = 1
lnum, rnum = num_divisors(n), num_divisors(n + 1)
while lnum * rnum < 500:
n += 1
lnum, rnum = rnum, num_divisors(n + 1)
return n
start = time.time()
index = find_triangular_index(500)
triangle = (index * (index + 1)) / 2
elapsed = (time.time() - start)
print("result %s returned in %s seconds." % (triangle, elapsed))

Natural Binary Code - finding first n numbers

Outputting/finding first n numbers of Natural Binary Code:
import math
def binary_print(n):
m = int(math.ceil(math.log(n, 2)))
for i in range(n):
b = str(bin(i)[2:])
print((m - len(b)) * '0' + b)
My question is:
Do you know any other way to do this in Python? Maybe faster? Or shorter (less code)?
Well this is shorter, not sure about faster:
def binary_print(n):
print '\n'.join('{:0{}b}'.format(x, (n-1).bit_length()) for x in range(n))
Example usage:
>>> binary_print(6)
000
001
010
011
100
101
def binary_values(n):
fmt = "{0:0"+str((n-1).bit_length())+"b}"
for i in range(n):
print fmt.format(i)
Note: (n-1).bit_length() fixes a fencepost error (otherwise if n is a power of two it prints 1 too many leading zeros).
Might also be able to speed it up a bit more by unrolling a lookup,
def binary_values(n):
fmt = ("{0:0"+str((n-1).bit_length())+"b}").format
for i in range(n):
print fmt(i)

Categories