We Keep Coding

Optimizing algorithm for finding palindromic primes in Python

I've been trying to solve a LeetCode problem which takes an input number (less than 10^8) and returns the next palindromic prime. Also, the answer is guaranteed to exist and is less than 2 * 10^8. My approach seems to work fine for most numbers, but the runtime increases significantly and LeetCode tells me I've exceeded the time limit when a specific number is entered (like 9989900). Is it because the gap between palindromic primes is large in that range?
This is the code I've written.
import time
start = time.time()
def is_prime(num: int) -> bool:
if num < 2:
return False
elif num == 2 or num == 3:
return True
if num % 6 != 1 and num % 6 != 5:
return False
else:
for i in range(3, int(num ** 0.5) + 1, 2):
if num % i == 0:
return False
else:
return True
def is_palindrome(num: int) -> bool:
return str(num) == str(num)[::-1]
class Solution:
def primePalindrome(self, N: int):
if N == 1:
return 2
elif 8 <= N < 11:
return 11
elif is_prime(N) and is_palindrome(N):
return N
# To skip even numbers, take 2 cases, i.e., when N is even and when N is odd
elif N % 2 == 0:
for i in range(N + 1, 2 * 10 ** 8, 2):
if len(str(i)) % 2 == 0: # Because even length palindromes are divisible by 11
i += 2
elif is_palindrome(i):
if is_prime(i):
return i
else:
continue
else:
for i in range(N, 2 * 10 ** 8, 2):
if len(str(i)) % 2 == 0:
i += 2
elif is_palindrome(i):
if is_prime(i):
return i
else:
continue
obj = Solution()
print(obj.primePalindrome(9989900)) # 100030001
print(time.time() - start) # 9 seconds
Is my solution slow because of too many loops and conditional statements? How do I reduce the runtime? Solving this without using any external libraries/packages would be preferable. Thank you.

Given that checking primes/palindromes sequentially isn't fast enough, I thought of this "number assembly" approach:
Given that prime numbers can only end with digits 1, 3, 7 or 9. The palindrome numbers also can only begin with these digits. So, if we generate palindrom digits in between the first and last we will get a lot fewer numbers to chck for "primality".
For example: 1xxxxxx1, 3xxxxxx3, 7xxxxxx7 and 9xxxxxx9
These middle parts must also be palindromes so we only have half the digits to consider: 1xxxyyy1 where yyy is a mirror of xxx. For odd sized middle we will have xxzyy where zyy is a mirror of xxz.
Combining this with a sequential generation of the first/last digits and digits in the middle, we can get the next number after N. By generating the most significant digits sequentially (i.e. the xxx part) we are certain that the composed numbers will be generated in an increasing sequence.
def isPrime(n):
return n==2 if n<3 or n%2==0 else all(n%d for d in range(3,int(n**0.5)+2,2))
def nextPalPrime(N):
digits = list(map(int,str(N)))
while True:
if digits[0] not in (1,3,7,9): # advance first/last digits
digits[0] = [1,1,3,3,7,7,7,7,9,9][digits[0]]
digits[1:] = [0]*(len(digits)-1)
digits[-1] = digits[0]
midSize = (len(digits)-1)//2
midStart = int("".join(map(str,digits[1:1+midSize] or [0])))
for middle in range(midStart,10**midSize): # generate middle digits
if midSize:
midDigits = list(map(int,f"{middle:0{midSize}}")) # left half of middle
digits[1:1+midSize] = midDigits # set left half
digits[-midSize-1:-1] = midDigits[::-1] # set mirrored right half
number = int("".join(map(str,digits)))
if number>N and isPrime(number): # check for prime
return number
digits[0] += 1 # next first digit
if digits[0] > 9: digits = [1]+[0]*len(digits) # need more digits
output:
pp = 1000
for _ in range(20):
pp = nextPalPrime(pp)
print(pp)
10301
10501
10601
11311
11411
12421
12721
12821
13331
13831
13931
14341
14741
15451
15551
16061
16361
16561
16661
17471
Performance:
from time import time
start=time()
print(nextPalPrime(9989900),time()-start)
100030001 0.023847103118896484
No even number of digits
Initially I was surprised that the solutions never produced a prime number with an even number of digits. but analyzing the composition of palindrome numbers I realized that those would always be multiples of 11 (so not prime):
abba = a*1001 + b*110
= a*11*91 + b*11*10
= 11*(a*91 + b*10)
abccba = a*100001 + b*10010 + c*1100
= a*11*9091 + b*11*910 + c*11*100
= 11*(a*9091 + b*910 + c*100)
abcddcba = a*10000001 + b*1000010 + c*100100 + d*110000
= a*11*909091 + b*11*90910 + c*11*9100 + d*11*10000
= 11*(a*909091 + b*90910 + c*9100 + d*10000)
abcdeedcba = a*1000000001 + b*100000010 + c*10000100 + d*10010000 + e*11000000
= a*11*90909091 + b*11*9090910 + c*11*909100 + d*11*910000 + e*11*1000000
= 11*(a*90909091 + b*9090910 + c*909100 + d*910000 + e*1000000)
Using this observation and a more numerical approach, we get a nice performance boost:
def nextPalPrime(N):
for width in range(len(str(N)),10):
if width%2==0: continue
size = width//2
factors = [(100**(size-n)+1)*10**n for n in range(size)]+[10**size]
for firstDigit in (1,3,7,9):
if (firstDigit+1)*factors[0]<N: continue
for middle in range(10**size):
digits = [firstDigit]+[*map(int,f"{middle:0{size}}")]
number = sum(f*d for f,d in zip(factors,digits))
if number>N and isPrime(number):
return number
from time import time
start=time()
print(nextPalPrime(9989900),time()-start)
100030001 0.004210948944091797

PYTHON RECURSION Behavior when adding Elements and Counting Elements in a List

I studied Python Recursion Recently. There are few things that i am finding difficult to understand.
looks like Counting and Adding Elements of a list Through Recursion in python is almost done in a similar way. Can someone Please explain me why the calls are acting differently just by adding '1' and 'num[0]' for the below return calls:
return "*1*"+sum(num[1:]) and return "*num[0]*"+sum(num[1:])
Code for Counting Numbers in a list:
def count(num):
if len(num) == 0:
return 0
return 1+count(num[1:])
print(count([1,2,3,4,5,6,11]))
Output: 7
Code for Adding Elements in a List:
def sum(num):
if len(num) == 0:
return 0
return num[0]+sum(num[1:])
print(sum([1,2,3,4,5,6,11]))
Output: 32
Can someone please explain the return statement and in both of above recursion programs.
Any information that makes sense to me will be of a great help. Thanks a lot.

First, let’s write this with a loop, which is more pythonic and also probably easier for a novice to understand:
def count(nums):
res = 0
for num in nums:
res = 1 + res
return res
def sum(nums):
res = 0
for num in nums:
res = num + res
return res
The same one-word difference in the + operation. But now, hopefully you can understand why it’s different. The first one starts with 0, and adds 1 to that, once per element. The second one starts with 0, and adds the element to that, once per element.
The recursive code is doing basically the same thing, except a bit more complicated.
What it’s actually doing is pushing each element onto a stack, then starting with 0, then popping the stack until it’s empty, adding 1 or the element (respectively) for each element popped off the stack.1
But pushing a bunch of elements onto a stack and popping them off gives you the same elements, just in reverse. And adding 1 once per element, or adding up all of the elements, does the same thing counting backward as counting forward.
If you still don't understand, you should try stepping through the calls. Normally I'd suggest using an interactive visualizer, like Python Tutor, but since this problem seems specifically designed to make you think functionally, let's do it that way.
Imagine that function calls in Python were just a matter of substituting in the arguments for the parameters.2
So:
count([1,2,3])
= 0 if len([1,2,3]) == 0 else 1 + count([2,3])
= 1 + count([2,3])
= 1 + (0 if len([2,3]) == 0 else 1 + count([3]))
= 1 + (1 + count([3]))
= 1 + (1 + (0 if len([3]) == 0 else 1 + count([])))
= 1 + (1 + (1 + count([])))
= 1 + (1 + (1 + (0 if len([]) == 0 else 1 + count([])))))
= 1 + (1 + (1 + (0)))
= 1 + (1 + (1))
= 1 + (2)
= 3
sum([5,10,1])
= 0 if len([5,10,1]) == 0 else 5 + count([10,1])
= 5 + count([10,1])
= 5 + (0 if len([10,1]) == 0 else 10 + count([1]))
= 5 + (10 + count([3]))
= 5 + (10 + (0 if len([3]) == 0 else 1 + count([])))
= 5 + (10 + (1 + count([])))
= 5 + (10 + (1 + (0 if len([]) == 0 else ??? + count([])))))
= 5 + (10 + (1 + (0)))
= 5 + (10 + (1))
= 5 + (11)
= 16
Also, notice that ??? above. What is num[0] when num is empty? That would raise an exception—but, because we never use that branch, it doesn't matter what it would have done.
1. It’s also making a whole lot of useless list copies for no reason but to let the author pretend he’s programming in Scheme rather than Python, but let’s ignore that.
2. They aren't—but as long as you aren't doing assignments or any other kind of mutation, you can't actually tell the difference. Which is why people who love pure functional languages—where you can't do assignments, etc.—love them.

How to added up a variable with multiple values together in Python Recursion Function?

So I was studying recursion function online. And the one question asks me to write a function to add up a number's digits together. For example (1023) -> 1 + 0 + 2 + 3 = 6. I used % and // get get rid of a digit each time. However, I don't know how to add them up together. The closest I can get is to print out each digit. Can anyone help me solve it or give me a hint please?
def digitalSum(n):
if n < 10:
sum_total = n
print(sum_total)
else:
sum_total = n % 10
digitalSum((n - (n % 10))//10)
print(sum_total)
digitalSum(1213)

Your function should return the current digit plus the sum of the rest of the digits:
def digitalSum(n):
if n < 10: return n
return n % 10 + digitalSum(n // 10)
print digitalSum(1213)
For completeness, you can also handle negative numbers:
def digitalSum(n):
if n < 0: sign = -1
else: sign = 1
n = abs(n)
if n < 10: return n
return sign * (n % 10 + digitalSum(n // 10))
print digitalSum(1213)

A correct version of your function is as follows:
from math import log10
def sum_digits(n, i=None):
if i is None:
i = int(log10(abs(n)))
e = float(10**i)
a, b = (n / e), (abs(n) % e)
if i == 0:
return int(a)
else:
return int(a) + sum_digits(b, (i - 1))
print sum_digits(1234)
print sum_digits(-1234)
Example:
$ python -i foo.py
10
8
>>>
Updated: Updated to properly (IHMO) cope with negative numbers. e.g: -1234 == -1 + 2 + 3 + 4 == 8
NB: Whilst this answer has been accepted (Thank you) I really think that perreal's answer should have been accepted for simplicity and clarity.
Also note: that whilst my solution handles negative numbers and summing their respective digits, perreal clearly points out in our comments that there are ate least three different ways to interpret the summing of digits of a negative number.

Digital sum, Python

I need to write a code that counts the sum of the digits of a number, these is the exact text of the problem:The digital sum of a number n is the sum of its digits. Write a recursive function digitalSum(n) that takes a positive integer n and returns its digital sum. For example, digitalSum(2019) should return 12 because 2+0+1+9=12. These is the code I wrote :
def digitalSum(n):
L=[]
if n < 10:
return n
else:
S=str(n)
for i in S:
L.append(int(i))
return sum(L)
These code works fine, but it's not a recursive function, and I'm not allowed to change any int to str. May you help me?

Try this:
def digitalSum(n):
if n < 10 :
return n
return n % 10 + digitalSum( n // 10 )
Edit: The logic behind this algorithm is that for every call of the recursive function, we chop off the number's last digit and add it to the sum. First we obtain the last digit with n % 10 and then we call the function again, passing the number with the last digit truncated: n // 10. We only stop when we reach a one-digit number. After we stop, the sum of the digits is computed in reverse order, as the recursive calls return.
Example for the number 12345 :
5 + digitalSum( 1234 )
5 + 4 + digitalSum( 123 )
5 + 4 + 3 + digitalSum( 12 )
5 + 4 + 3 + 2 + 1 <- done recursing
5 + 4 + 3 + 3
5 + 4 + 6
5 + 10
15

It's homework, so I'm not writing much code. Recursion can be used in the following way:
get the first (or last) digit
format the rest as a shorter number
add the digit and the digital sum of the shorter number (recursion!)

This is more of a question related to algorithms.
Here is your answer:
def digit_sum(a):
if a == 0:
return 0
return a % 10 + digit_sum(a/10)
Let me know if you don't understand why it works and I'll provide an explanation.

Some hints:
You can define inner functions in Python
You can use the modulus operator (look up its syntax and usage) to good effect, here
There's no need to build up an explicit list representation with a proper recursive solution
EDIT The above is a bit "bad" as a general answer, what if someone else has this problem in a non-homework context? Then Stack Overflow fails ...
So, here's how I would implement it, and you need to decide whether or not you should continue reading. :)
def digitalSum(n):
def process(n, sum):
if n < 10:
return sum + n
return process(n / 10, sum + n % 10)
return process(n, 0)
This might be a bit too much, but even in a learning situation having access to one answer can be instructive.
My solution is more a verbose than some, but it's also more friendly towards a tail call optimizing compiler, which I think is a feature.

def digital_sum(number):
if number < 10:
return number
else:
return number % 10 + digital_sum(number / 10)

def sumofdigits(a):
a = str(a)
a = list(a)
b = []
for i in a:
b.append(int(i))
b = sum(b)
if b > 9:
return sumofdigits(b)
else:
return b
print sumofdigits(5487123456789087654)

For people looking non-recursive ways,
Solution 1:
Using formula,
digits = int(input())
res = (digits * (digits + 1) // 2)
Solution 2:
Using basic syntax
numbers = [6, 5, 3, 8, 4, 2, 5, 4, 11]
total = numbers[0]
print(f'{total}')
for val in numbers[1:]:
print(f'{total} + {val} = {total + val}')
total += val
gives
6
6 + 5 = 11
11 + 3 = 14
14 + 8 = 22
22 + 4 = 26
26 + 2 = 28
28 + 5 = 33
33 + 4 = 37
37 + 11 = 48

Still you can do it in O(log10 n)...cancel out all the digits that adds to 9 then if no numbers left,9 is the answer else sum up all the left out digits...
def rec_sum_Reduce(n) :
ans = 0
for i in map(int,str(n)) :
ans = 1+(ans+i-1)%9
return ans

def drs_f(p):
drs = sum([int (q) for q in str(p)])
while drs >= 10:
drs = sum([int(q) for q in str(drs)])
return drs

def digitalSum(n):
if n < 10:
return n
else:
return ???
The 1st part is from your existing code.
The ??? is the part you need to work out. It could take one digit off n and add it to the digitalSum of the remaining digits.
You don't really need the else, but I left it there so the code structure looks the same

Optimizing python code

Any tips on optimizing this python code for finding next palindrome:
Input number can be of 1000000 digits
COMMENTS ADDED
#! /usr/bin/python
def inc(lst,lng):#this function first extract the left half of the string then
#convert it to int then increment it then reconvert it to string
#then reverse it and finally append it to the left half.
#lst is input number and lng is its length
if(lng%2==0):
olst=lst[:lng/2]
l=int(lng/2)
olst=int(olst)
olst+=1
olst=str(olst)
p=len(olst)
if l<p:
olst2=olst[p-2::-1]
else:
olst2=olst[::-1]
lst=olst+olst2
return lst
else:
olst=lst[:lng/2+1]
l=int(lng/2+1)
olst=int(olst)
olst+=1
olst=str(olst)
p=len(olst)
if l<p:
olst2=olst[p-3::-1]
else:
olst2=olst[p-2::-1]
lst=olst+olst2
return lst
t=raw_input()
t=int(t)
while True:
if t>0:
t-=1
else:
break
num=raw_input()#this is input number
lng=len(num)
lst=num[:]
if(lng%2==0):#this if find next palindrome to num variable
#without incrementing the middle digit and store it in lst.
olst=lst[:lng/2]
olst2=olst[::-1]
lst=olst+olst2
else:
olst=lst[:lng/2+1]
olst2=olst[len(olst)-2::-1]
lst=olst+olst2
if int(num)>=int(lst):#chk if lst satisfies criteria for next palindrome
num=inc(num,lng)#otherwise call inc function
print num
else:
print lst

I think most of the time in this code is spent converting strings to integers and back. The rest is slicing strings and bouncing around in the Python interpreter. What can be done about these three things? There are a few unnecessary conversions in the code, which we can remove. I see no way to avoid the string slicing. To minimize your time in the interpreter you just have to write as little code as possible :-) and it also helps to put all your code inside functions.
The code at the bottom of your program, which takes a quick guess to try and avoid calling inc(), has a bug or two. Here's how I might write that part:
def nextPal(num):
lng = len(num)
guess = num[:lng//2] + num[(lng-1)//2::-1] # works whether lng is even or odd
if guess > num: # don't bother converting to int
return guess
else:
return inc(numstr, n)
This simple change makes your code about 100x faster for numbers where inc doesn't need to be called, and about 3x faster for numbers where it does need to be called.
To do better than that, I think you need to avoid converting to int entirely. That means incrementing the left half of the number without using ordinary Python integer addition. You can use an array and carry out the addition algorithm "by hand":
import array
def nextPal(numstr):
# If we don't need to increment, just reflect the left half and return.
n = len(numstr)
h = n//2
guess = numstr[:n-h] + numstr[h-1::-1]
if guess > numstr:
return guess
# Increment the left half of the number without converting to int.
a = array.array('b', numstr)
zero = ord('0')
ten = ord('9') + 1
for i in range(n - h - 1, -1, -1):
d = a[i] + 1
if d == ten:
a[i] = zero
else:
a[i] = d
break
else:
# The left half was all nines. Carry the 1.
# Update n and h since the length changed.
a.insert(0, ord('1'))
n += 1
h = n//2
# Reflect the left half onto the right half.
a[n-h:] = a[h-1::-1]
return a.tostring()
This is another 9x faster or so for numbers that require incrementing.
You can make this a touch faster by using a while loop instead of for i in range(n - h - 1, -1, -1), and about twice as fast again by having the loop update both halves of the array rather than just updating the left-hand half and then reflecting it at the end.

You don't have to find the palindrome, you can just generate it.
Split the input number, and reflect it. If the generated number is too small, then increment the left hand side and reflect it again:
def nextPal(n):
ns = str(n)
oddoffset = 0
if len(ns) % 2 != 0:
oddoffset = 1
leftlen = len(ns) / 2 + oddoffset
lefts = ns[0:leftlen]
right = lefts[::-1][oddoffset:]
p = int(lefts + right)
if p < n:
## Need to increment middle digit
left = int(lefts)
left += 1
lefts = str(left)
right = lefts[::-1][oddoffset:]
p = int(lefts + right)
return p
def test(n):
print n
p = nextPal(n)
assert p >= n
print p
test(1234567890)
test(123456789)
test(999999)
test(999998)
test(888889)
test(8999999)

EDIT
NVM, just look at this page: http://thetaoishere.blogspot.com/2009/04/finding-next-palindrome-given-number.html

Using strings. n >= 0
from math import floor, ceil, log10
def next_pal(n):
# returns next palindrome, param is an int
n10 = str(n)
m = len(n10) / 2.0
s, e = int(floor(m - 0.5)), int(ceil(m + 0.5))
start, middle, end = n10[:s], n10[s:e], n10[e:]
assert (start, middle[0]) == (end[-1::-1], middle[-1]) #check that n is actually a palindrome
r = int(start + middle[0]) + 1 #where the actual increment occurs (i.e. add 1)
r10 = str(r)
i = 3 - len(middle)
if len(r10) > len(start) + 1:
i += 1
return int(r10 + r10[-i::-1])
Using log, more optized. n > 9
def next_pal2(n):
k = log10(n + 1)
l = ceil(k)
s, e = int(floor(l/2.0 - 0.5)), int(ceil(l/2.0 + 0.5))
mmod, emod = 10**(e - s), int(10**(l - e))
start, end = divmod(n, emod)
start, middle = divmod(start, mmod)
r1 = 10*start + middle%10 + 1
i = middle > 9 and 1 or 2
j = s - i + 2
if k == l:
i += 1
r2 = int(str(r1)[-i::-1])
return r1*10**j + r2