We Keep Coding

Having trouble with implementeing the Miller-Rabin compositeness in Python

I'm not sure if this is the right place to post this question so if it isn't let me know! I'm trying to implement the Miller Rabin test in python. The test is to find the first composite number that is a witness to N, an odd number. My code works for numbers that are somewhat smaller in length but stops working when I enter a huge number. (The "challenge" wants to find the witness of N := 14779897919793955962530084256322859998604150108176966387469447864639173396414229372284183833167 in which my code returns that it is prime when it isn't) The first part of the test is to convert N into the form 2^k + q, where q is a prime number.
Is there some limit with python that doesn't allow huge numbers for this?
Here is my code for that portion of the test.
def convertN(n): #this turns n into 2^x * q
placeholder = False
list = []
#this will be x in the equation
count = 1
while placeholder == False:
#x = result of division of 2^count
x = (n / (2**count))
#y tells if we can divide by 2 again or not
y = x%2
#if y != 0, it means that we cannot divide by 2, loop exits
if y != 0:
placeholder = True
list.append(count) #x
list.append(x) #q
else:
count += 1
#makes list to return
#print(list)
return list
The code for the actual test:
def test(N):
#if even return false
if N == 2 | N%2 == 0:
return "even"
#convert number to 2^k+q and put into said variables
n = N - 1
nArray = convertN(n)
k = nArray[0]
q = int(nArray[1])
#this is the upper limit a witness can be
limit = int(math.floor(2 * (math.log(N))**2))
#Checks when 2^q*k = 1 mod N
for a in range(2,limit):
modu = pow(a,q,N)
for i in range(k):
print(a,i,modu)
if i==0:
if modu == 1:
break
elif modu == -1:
break
elif i != 0:
if modu == 1:
#print(i)
return a
#instead of recalculating 2^q*k+1, can square old result and modN that.
modu = pow(modu,2,N)
Any feedback is appreciated!

I don't like unanswered questions so I decided to give a small update.
So as it turns out I was entering the wrong number from the start. Along with that my code should have tested not for when it equaled to 1 but if it equaled -1 from the 2nd part.
The fixed code for the checking
#Checks when 2^q*k = 1 mod N
for a in range(2,limit):
modu = pow(a,q,N)
witness = True #I couldn't think of a better way of doing this so I decided to go with a boolean value. So if any of values of -1 or 1 when i = 0 pop up, we know it's not a witness.
for i in range(k):
print(a,i,modu)
if i==0:
if modu == 1:
witness = False
break
elif modu == -1:
witness = False
break
#instead of recalculating 2^q*k+1, can square old result and modN that.
modu = pow(modu,2,N)
if(witness == True):
return a

Mei, i wrote a Miller Rabin Test in python, the Miller Rabin part is threaded so it's very fast, faster than sympy, for larger numbers:
import math
def strailing(N):
return N>>lars_last_powers_of_two_trailing(N)
def lars_last_powers_of_two_trailing(N):
""" This utilizes a bit trick to find the trailing zeros in a number
Finding the trailing number of zeros is simply a lookup for most
numbers and only in the case of 1 do you have to shift to find the
number of zeros, so there is no need to bit shift in 7 of 8 cases.
In those 7 cases, it's simply a lookup to find the amount of zeros.
"""
p,y=1,2
orign = N
N = N&15
if N == 1:
if ((orign -1) & (orign -2)) == 0: return orign.bit_length()-1
while orign&y == 0:
p+=1
y<<=1
return p
if N in [3, 7, 11, 15]: return 1
if N in [5, 13]: return 2
if N == 9: return 3
return 0
def primes_sieve2(limit):
a = [True] * limit
a[0] = a[1] = False
for (i, isprime) in enumerate(a):
if isprime:
yield i
for n in range(i*i, limit, i):
a[n] = False
def llinear_diophantinex(a, b, divmodx=1, x=1, y=0, offset=0, withstats=False, pow_mod_p2=False):
""" For the case we use here, using a
llinear_diophantinex(num, 1<<num.bit_length()) returns the
same result as a
pow(num, 1<<num.bit_length()-1, 1<<num.bit_length()). This
is 100 to 1000x times faster so we use this instead of a pow.
The extra code is worth it for the time savings.
"""
origa, origb = a, b
r=a
q = a//b
prevq=1
#k = powp2x(a)
if a == 1:
return 1
if withstats == True:
print(f"a = {a}, b = {b}, q = {q}, r = {r}")
while r != 0:
prevr = r
a,r,b = b, b, r
q,r = divmod(a,b)
x, y = y, x - q * y
if withstats == True:
print(f"a = {a}, b = {b}, q = {q}, r = {r}, x = {x}, y = {y}")
y = 1 - origb*x // origa - 1
if withstats == True:
print(f"x = {x}, y = {y}")
x,y=y,x
modx = (-abs(x)*divmodx)%origb
if withstats == True:
print(f"x = {x}, y = {y}, modx = {modx}")
if pow_mod_p2==False:
return (x*divmodx)%origb, y, modx, (origa)%origb
else:
if x < 0: return (modx*divmodx)%origb
else: return (x*divmodx)%origb
def MillerRabin(arglist):
""" This is a standard MillerRabin Test, but refactored so it can be
used with multi threading, so you can run a pool of MillerRabin
tests at the same time.
"""
N = arglist[0]
primetest = arglist[1]
iterx = arglist[2]
powx = arglist[3]
withstats = arglist[4]
primetest = pow(primetest, powx, N)
if withstats == True:
print("first: ",primetest)
if primetest == 1 or primetest == N - 1:
return True
else:
for x in range(0, iterx-1):
primetest = pow(primetest, 2, N)
if withstats == True:
print("else: ", primetest)
if primetest == N - 1: return True
if primetest == 1: return False
return False
# For trial division, we setup this global variable to hold primes
# up to 1,000,000
SFACTORINT_PRIMES=list(primes_sieve2(100000))
# Uses MillerRabin in a unique algorithimically deterministic way and
# also uses multithreading so all MillerRabin Tests are performed at
# the same time, speeding up the isprime test by a factor of 5 or more.
# More k tests can be performed than 5, but in my testing i've found
# that's all you need.
def sfactorint_isprime(N, kn=5, trialdivision=True, withstats=False):
from multiprocessing import Pool
if N == 2:
return True
if N % 2 == 0:
return False
if N < 2:
return False
# Trial Division Factoring
if trialdivision == True:
for xx in SFACTORINT_PRIMES:
if N%xx == 0 and N != xx:
return False
iterx = lars_last_powers_of_two_trailing(N)
""" This k test is a deterministic algorithmic test builder instead of
using random numbers. The offset of k, from -2 to +2 produces pow
tests that fail or pass instead of having to use random numbers
and more iterations. All you need are those 5 numbers from k to
get a primality answer. I've tested this against all numbers in
https://oeis.org/A001262/b001262.txt and all fail, plus other
exhaustive testing comparing to other isprimes to confirm it's
accuracy.
"""
k = llinear_diophantinex(N, 1<<N.bit_length(), pow_mod_p2=True) - 1
t = N >> iterx
tests = []
if kn % 2 == 0: offset = 0
else: offset = 1
for ktest in range(-(kn//2), (kn//2)+offset):
tests.append(k+ktest)
for primetest in range(len(tests)):
if tests[primetest] >= N:
tests[primetest] %= N
arglist = []
for primetest in range(len(tests)):
if tests[primetest] >= 2:
arglist.append([N, tests[primetest], iterx, t, withstats])
with Pool(kn) as p:
s=p.map(MillerRabin, arglist)
if s.count(True) == len(arglist): return True
else: return False
sinn=14779897919793955962530084256322859998604150108176966387469447864639173396414229372284183833167
print(sfactorint_isprime(sinn))

How to find Lagest sum consecutive increasing digits in a number

How can i find the Largest sum consecutive increasing digits and its position in a Number using python ?
Here is my approach :
def findLIS(A, n):
hash = dict()
LIS_size, LIS_index = 1, 0
hash[A[0]] = 1
for i in range(1, n):
if A[i] - 1 not in hash:
hash[A[i] - 1] = 0
hash[A[i]] = hash[A[i] - 1] + 1
if LIS_size < hash[A[i]]:
LIS_size = hash[A[i]]
LIS_index = A[i]
start = LIS_index - LIS_size + 1
while start <= LIS_index:
# print(start, end = " ")
z.insert(i - 1,start)
start += 1
print(z[~0])
print(sum(z))
# Driver Code
if __name__ == "__main__":
num = input()
A = [int(x) for x in str(num)]
n = len(A)
z=[]
findLIS(A, n)

Solution:
n = input()+"0"
max_sum = 0
max_pos = ""
start = 0
sum = int(n[0])
for i in range(1, len(n)):
if n[i-1] >= n[i]:
if max_sum < sum:
max_sum = sum
max_pos = "{}-{}".format(start+1, i)
start = i
sum = int(n[i])
else:
sum += int(n[i])
print("{}:{}".format(max_sum, max_pos))
There is numbers should strictly increase. If sequence 333 also match your condition that you need to change condition: if n[i-1] > n[i]:

Python 3: Optimizing Project Euler Problem #14

I'm trying to solve the Hackerrank Project Euler Problem #14 (Longest Collatz sequence) using Python 3. Following is my implementation.
cache_limit = 5000001
lookup = [0] * cache_limit
lookup[1] = 1
def collatz(num):
if num == 1:
return 1
elif num % 2 == 0:
return num >> 1
else:
return (3 * num) + 1
def compute(start):
global cache_limit
global lookup
cur = start
count = 1
while cur > 1:
count += 1
if cur < cache_limit:
retrieved_count = lookup[cur]
if retrieved_count > 0:
count = count + retrieved_count - 2
break
else:
cur = collatz(cur)
else:
cur = collatz(cur)
if start < cache_limit:
lookup[start] = count
return count
def main(tc):
test_cases = [int(input()) for _ in range(tc)]
bound = max(test_cases)
results = [0] * (bound + 1)
start = 1
maxCount = 1
for i in range(1, bound + 1):
count = compute(i)
if count >= maxCount:
maxCount = count
start = i
results[i] = start
for tc in test_cases:
print(results[tc])
if __name__ == "__main__":
tc = int(input())
main(tc)
There are 12 test cases. The above implementation passes till test case #8 but fails for test cases #9 through #12 with the following reason.
Terminated due to timeout
I'm stuck with this for a while now. Not sure what else can be done here.
What else can be optimized here so that I stop getting timed out?
Any help will be appreciated :)
Note: Using the above implementation, I'm able to solve the actual Project Euler Problem #14. It is giving timeout only for those 4 test cases in hackerrank.

Yes, there are things you can do to your code to optimize it. But I think, more importantly, there is a mathematical observation you need to consider which is at the heart of the problem:
whenever n is odd, then 3 * n + 1 is always even.
Given this, one can always divide (3 * n + 1) by 2. And that saves one a fair bit of time...

Here is an improvement (it takes 1.6 seconds): there is no need to compute the sequence of every number. You can create a dictionary and store the number of the elements of a sequence. If a number that has appeared already comes up, the sequence is computed as dic[original_number] = dic[n] + count - 1. This saves a lot of time.
import time
start = time.time()
def main(n,dic):
'''Counts the elements of the sequence starting at n and finishing at 1'''
count = 1
original_number = n
while True:
if n < original_number:
dic[original_number] = dic[n] + count - 1 #-1 because when n < original_number, n is counted twice otherwise
break
if n == 1:
dic[original_number] = count
break
if (n % 2 == 0):
n = n/2
else:
n = 3*n + 1
count += 1
return dic
limit = 10**6
dic = {n:0 for n in range(1,limit+1)}
if __name__ == '__main__':
n = 1
while n < limit:
dic=main(n,dic)
n += 1
print('Longest chain: ', max(dic.values()))
print('Number that gives the longest chain: ', max(dic, key=dic.get))
end = time.time()
print('Time taken:', end-start)

The trick to solve this question is to compute the answers for only largest input and save the result as lookup for all smaller inputs rather than calculating for extreme upper bound.
Here is my implementation which passes all the Test Cases.(Python3)
MAX = int(5 * 1e6)
ans = [0]
steps = [0]*(MAX+1)
def solve(N):
if N < MAX+1:
if steps[N] != 0:
return steps[N]
if N == 1:
return 0
else:
if N % 2 != 0:
result = 1+ solve(3*N + 1) # This is recursion
else:
result = 1 + solve(N>>1) # This is recursion
if N < MAX+1:
steps[N]=result # This is memoization
return result
inputs = [int(input()) for _ in range(int(input()))]
largest = max(inputs)
mx = 0
collatz=1
for i in range(1,largest+1):
curr_count=solve(i)
if curr_count >= mx:
mx = curr_count
collatz = i
ans.append(collatz)
for _ in inputs:
print(ans[_])

this is my brute force take:
'
#counter
C = 0
N = 0
for i in range(1,1000001):
n = i
c = 0
while n != 1:
if n % 2 == 0:
_next = n/2
else:
_next= 3*n+1
c = c + 1
n = _next
if c > C:
C = c
N = i
print(N,C)

Here's my implementation(for the question specifically on Project Euler website):
num = 1
limit = int(input())
seq_list = []
while num < limit:
sequence_num = 0
n = num
if n == 1:
sequence_num = 1
else:
while n != 1:
if n % 2 == 0:
n = n / 2
sequence_num += 1
else:
n = 3 * n + 1
sequence_num += 1
sequence_num += 1
seq_list.append(sequence_num)
num += 1
k = seq_list.index(max(seq_list))
print(k + 1)

Generating gray codes.

I tried generating gray codes in Python. This code works correctly. The issue is that I am initialising the base case (n=1,[0,1]) in the main function and passing it to gray_code function to compute the rest. I want to generate all the gray codes inside the function itself including the base case. How do I do that?
def gray_code(g,n):
k=len(g)
if n<=0:
return
else:
for i in range (k-1,-1,-1):
char='1'+g[i]
g.append(char)
for i in range (k-1,-1,-1):
g[i]='0'+g[i]
gray_code(g,n-1)
def main():
n=int(raw_input())
g=['0','1']
gray_code(g,n-1)
if n>=1:
for i in range (len(g)):
print g[i],
main()
Is the recurrence relation of this algorithm T(n)=T(n-1)+n ?

Generating Gray codes is easier than you think. The secret is that the Nth gray code is in the bits of N^(N>>1)
So:
def main():
n=int(raw_input())
for i in range(0, 1<<n):
gray=i^(i>>1)
print "{0:0{1}b}".format(gray,n),
main()

def gray_code(n):
def gray_code_recurse (g,n):
k=len(g)
if n<=0:
return
else:
for i in range (k-1,-1,-1):
char='1'+g[i]
g.append(char)
for i in range (k-1,-1,-1):
g[i]='0'+g[i]
gray_code_recurse (g,n-1)
g=['0','1']
gray_code_recurse(g,n-1)
return g
def main():
n=int(raw_input())
g = gray_code (n)
if n>=1:
for i in range (len(g)):
print g[i],
main()

It's relatively easy to do if you implement the function iteratively (even if it's defined recursively). This will often execute more quickly as it generally requires fewer function calls.
def gray_code(n):
if n < 1:
g = []
else:
g = ['0', '1']
n -= 1
while n > 0:
k = len(g)
for i in range(k-1, -1, -1):
char = '1' + g[i]
g.append(char)
for i in range(k-1, -1, -1):
g[i] = '0' + g[i]
n -= 1
return g
def main():
n = int(raw_input())
g = gray_code(n)
print ' '.join(g)
main()

What about this:
#! /usr/bin/python3
def hipow(n):
''' Return the highest power of 2 within n. '''
exp = 0
while 2**exp <= n:
exp += 1
return 2**(exp-1)
def code(n):
''' Return nth gray code. '''
if n>0:
return hipow(n) + code(2*hipow(n) - n - 1)
return 0
# main:
for n in range(30):
print(bin(code(n)))

Here's how I did it. state array need to hold some n-bit gray code for some value of n, from which the next gray-code will be generated and state array will contain the generated gray-code, and so on. Although the state is initialized here to be a n-bit '0' code it can be any other n-bit gray code as well.
Time Complexity: O(2^n) For iteratively listing out each 2^n gray codes.
Space Complexity: O(n) For having n-length state and powers array.
def get_bit(line, bit_pos, state, powers):
k = powers[bit_pos-1]
if line % (k // 2):
return str(state[bit_pos-1])
else:
bit = 1 - state[bit_pos - 1]
state[bit_pos - 1] = bit
if line % k == 0:
state[bit_pos - 1] = 1 - bit
bit = 1 - bit
return str(bit)
def gray_codes(n):
lines = 1 << n
state = [0] * n
powers = [1 << i for i in range(1, n + 1)]
for line in range(lines):
gray_code = ''
for bit_pos in range(n, 0, -1):
gray_code += get_bit(line, bit_pos, state, powers)
print(gray_code)
n = int(input())
gray_codes(n)

Clearly this horse has been beaten to death already, but I'll add that if you aren't going to use the cool and time-honored n ^ (n >> 1) trick, the recursion can be stated rather more succinctly:
def gc(n):
if n == 1:
return ['0', '1']
r = gc(n - 1)
return ['0' + e for e in r] + ['1' + e for e in reversed(r)]
... and the iteration, too:
def gc(n):
r = ['0', '1']
for i in range(2, n + 1):
r = ['0' + e for e in r] + ['1' + e for e in reversed(r)]
return r

Converting phone number range list to prefix list

I have a phone number range, for example:
3331234-3332345
I need to write a function that converts it to list of prefixes:
3331234
...
3331239
333124
...
333129
33313
...
33319
33320
...
33322
333231
333232
333233
3332341
...
3332345
Question is not so easy. I don't need to get a list of numbers between range start and end.

My working code. It not very quick, too. Optimizations welcome.
def diap_to_prefix(a, b):
lst = ['%0*d'%(max(len(str(a)), len(str(b))), x) for x in range(int(a), int(b)+1)]
new_lst = []
while len(lst) != len(new_lst):
lst = new_lst or lst
new_lst = []
c = lst[0]
tmp_lst = [c]
for i in lst[1:]:
if c[:-1] == i[:-1]:
c = i
tmp_lst.append(c)
else:
if len(tmp_lst) == 10:
new_lst.append(c[:-1])
else:
new_lst.extend(tmp_lst)
c = i
tmp_lst = [c]
if len(tmp_lst) == 10:
new_lst.append(c[:-1])
else:
new_lst.extend(tmp_lst)
return lst

My new more optimal solution (py3.4)
def diap_to_prefix(a, b):
def inner(aa, bb, p):
if p == 1:
if a <= aa <= b:
yield aa
return
for d in range(aa, bb + 1, p):
if a <= d and d + p - 1 <= b:
yield d // p
elif not (bb < a or aa > b):
for i in range(10):
yield from inner(d + i * p // 10, d + (i + 1) * p // 10 - 1, p // 10)
a, b = int(a), int(b)
p = 10**(max(len(str(x)) for x in (a, b)) - 1)
yield from inner(a // p * p, b // p * p + p - 1, p)

You need to get the common prefix of the values separated by "-", so:
Use .split to get these and iterate through them until you find a difference
Complete the first value with zeros (to get the least number) until you get phone_len digits and do the same for the maximum (with nines)
Then, you have a simple range of numbers
Iterate through them and convert them to strings
Here it is:
phone_len = 7
R = "33312345-3332345".split("-")
prefix = ""
for i in range(len(R[0])):
if R[0][i] == R[1][i]:
prefix += R[0][i]
else:
break
m = int(R[0]+"0"*(phone_len-len(R[0])))
M = int(R[1]+"9"*(phone_len-len(R[0])))
phones = [str(n) for n in range(m, M+1)]

Here's a sketch of one way to handle this problem. I've used ellipses to mark the spots where you'll need to fill in the details explained in the comments. I'd write a function to derive the initial value of 'maxpower', everything else is simple enough to be written inline.
firstnumber = 3331234
lastnumber = 3332345
current = firstnumber
while current <= lastnumber:
# Find the largest power of 10 that exactly divides 'current'.
# Call this value 'maxpower'. 'maxpower' is a candidate for the
# size of the block of numbers that will be represented by the
# next output value.
maxpower = ... # 1, 10, 100, 1000, 10000, and so on
# If a block of size 'maxpower' would take us past the
# 'lastnumber', we can't use that block size. We must try a
# smaller block. Divide 'maxpower' by 10 until the block size
# becomes acceptable.
while (current + maxpower) > ... :
maxpower /= 10
# Now 'maxpower' is the largest acceptable size for the next
# block, so the desired prefix is 'current' divided by 'maxpower'.
# Emit that value, then add 'maxpower' to 'current' to get the new
# 'current' value for the next iteration.
print ...
current += maxpower

My working code. It not very quick, but working. Optimizations welcome.
def fill(root, prefix, value, parent, pkey):
if len(prefix) > 1:
if prefix[0] in root:
fill(root[prefix[0]], prefix[1:], value, root, prefix[0])
if pkey:
if len(parent[pkey]) == 10:
parent[pkey] = value
elif type(root) == type({}):
root[prefix[0]] = {}
fill(root[prefix[0]], prefix[1:], value, root, prefix[0])
if pkey:
if len(parent[pkey]) == 10:
parent[pkey] = value
elif type(root) == type({}):
root[prefix[0]] = value
if pkey:
if len(parent[pkey]) == 10:
parent[pkey] = value
return root
def compact(prefixes, current):
if not type(prefixes) == type({}):
return [current]
else:
rlist = []
for k, v in prefixes.iteritems():
rlist.extend(compact(v, current + k))
continue
return rlist
if __name__ == '__main__':
plist = {}
for x in range(4440000, 4490000):
fill(plist, str(x), 'value', plist, None)
#print plist
print compact(plist, '')