We Keep Coding

Let n be a square number. Using Python, how we can efficiently calculate natural numbers y up to a limit l such that n+y^2 is again a square number?

Using Python, I would like to implement a function that takes a natural number n as input and outputs a list of natural numbers [y1, y2, y3, ...] such that n + y1*y1 and n + y2*y2 and n + y3*y3 and so forth is again a square.
What I tried so far is to obtain one y-value using the following function:
def find_square(n:int) -> tuple[int, int]:
if n%2 == 1:
y = (n-1)//2
x = n+y*y
return (y,x)
return None
It works fine, eg. find_square(13689) gives me a correct solution y=6844. It would be great to have an algorithm that yields all possible y-values such as y=44 or y=156.

Simplest slow approach is of course for given N just to iterate all possible Y and check if N + Y^2 is square.
But there is a much faster approach using integer Factorization technique:
Lets notice that to solve equation N + Y^2 = X^2, that is to find all integer pairs (X, Y) for given fixed integer N, we can rewrite this equation to N = X^2 - Y^2 = (X + Y) * (X - Y) which follows from famous school formula of difference of squares.
Now lets rename two factors as A, B i.e. N = (X + Y) * (X - Y) = A * B, which means that X = (A + B) / 2 and Y = (A - B) / 2.
Notice that A and B should be of same odditiy, either both odd or both even, otherwise in last formulas above we can't have whole division by 2.
We will factorize N into all possible pairs of two factors (A, B) of same oddity. For fast factorization in code below I used simple to implement but yet quite fast algorithm Pollard Rho, also two extra algorithms were needed as a helper to Pollard Rho, one is Fermat Primality Test (which allows fast checking if number is probably prime) and second is Trial Division Factorization (which helps Pollard Rho to factor out small factors, which could cause Pollard Rho to fail).
Pollard Rho for composite number has time complexity O(N^(1/4)) which is very fast even for 64-bit numbers. Any faster factorization algorithm can be chosen if needed a bigger space to be searched. My fast algorithm time is dominated by speed of factorization, remaining part of algorithm is blazingly fast, just few iterations of loop with simple formulas.
If your N is a square itself (hence we know its root easily), then Pollard Rho can factor N even much faster, within O(N^(1/8)) time. Even for 128-bit numbers it means very small time, 2^16 operations, and I hope you're solving your task for less than 128 bit numbers.
If you want to process a range of possible N values then fastest way to factorize them is to use techniques similar to Sieve of Erathosthenes, using set of prime numbers, it allows to compute all factors for all N numbers within some range. Using Sieve of Erathosthenes for the case of range of Ns is much faster than factorizing each N with Pollard Rho.
After factoring N into pairs (A, B) we compute (X, Y) based on (A, B) by formulas above. And output resulting Y as a solution of fast algorithm.
Following code as an example is implemented in pure Python. Of course one can use Numba to speed it up, Numba usually gives 30-200 times speedup, for Python it achieves same speed as optimized C++. But I thought that main thing here is to implement fast algorithm, Numba optimizations can be done easily afterwards.
I added time measurement into following code. Although it is pure Python still my fast algorithm achieves 8500x times speedup compared to regular brute force approach for limit of 1 000 000.
You can change limit variable to tweak amount of searched space, or num_tests variable to tweak amount of different tests.
Following code implements both solutions - fast solution find_fast() described above plus very tiny brute force solution find_slow() which is very slow as it scans all possible candidates. This slow solution is only used to compare correctness in tests and compare speedup.
Code below uses nothing except few standard Python library modules, no external modules were used.
Try it online!
def find_slow(N):
import math
def is_square(x):
root = int(math.sqrt(float(x)) + 0.5)
return root * root == x, root
l = []
for y in range(N):
if is_square(N + y ** 2)[0]:
l.append(y)
return l
def find_fast(N):
import itertools, functools
Prod = lambda it: functools.reduce(lambda a, b: a * b, it, 1)
fs = factor(N)
mfs = {}
for e in fs:
mfs[e] = mfs.get(e, 0) + 1
fs = sorted(mfs.items())
del mfs
Ys = set()
for take_a in itertools.product(*[
(range(v + 1) if k != 2 else range(1, v)) for k, v in fs]):
A = Prod([p ** t for (p, _), t in zip(fs, take_a)])
B = N // A
assert A * B == N, (N, A, B, take_a)
if A < B:
continue
X = (A + B) // 2
Y = (A - B) // 2
assert N + Y ** 2 == X ** 2, (N, A, B, X, Y)
Ys.add(Y)
return sorted(Ys)
def trial_div_factor(n, limit = None):
# https://en.wikipedia.org/wiki/Trial_division
fs = []
while n & 1 == 0:
fs.append(2)
n >>= 1
all_checked = False
for d in range(3, (limit or n) + 1, 2):
if d * d > n:
all_checked = True
break
while True:
q, r = divmod(n, d)
if r != 0:
break
fs.append(d)
n = q
if n > 1 and all_checked:
fs.append(n)
n = 1
return fs, n
def fermat_prp(n, trials = 32):
# https://en.wikipedia.org/wiki/Fermat_primality_test
import random
if n <= 16:
return n in (2, 3, 5, 7, 11, 13)
for i in range(trials):
if pow(random.randint(2, n - 2), n - 1, n) != 1:
return False
return True
def pollard_rho_factor(n):
# https://en.wikipedia.org/wiki/Pollard%27s_rho_algorithm
import math, random
fs, n = trial_div_factor(n, 1 << 7)
if n <= 1:
return fs
if fermat_prp(n):
return sorted(fs + [n])
for itry in range(8):
failed = False
x = random.randint(2, n - 2)
for cycle in range(1, 1 << 60):
y = x
for i in range(1 << cycle):
x = (x * x + 1) % n
d = math.gcd(x - y, n)
if d == 1:
continue
if d == n:
failed = True
break
return sorted(fs + pollard_rho_factor(d) + pollard_rho_factor(n // d))
if failed:
break
assert False, f'Pollard Rho failed! n = {n}'
def factor(N):
import functools
Prod = lambda it: functools.reduce(lambda a, b: a * b, it, 1)
fs = pollard_rho_factor(N)
assert N == Prod(fs), (N, fs)
return sorted(fs)
def test():
import random, time
limit = 1 << 20
num_tests = 20
t0, t1 = 0, 0
for i in range(num_tests):
if (round(i / num_tests * 1000)) % 100 == 0 or i + 1 >= num_tests:
print(f'test {i}, ', end = '', flush = True)
N = random.randrange(limit)
tb = time.time()
r0 = find_slow(N)
t0 += time.time() - tb
tb = time.time()
r1 = find_fast(N)
t1 += time.time() - tb
assert r0 == r1, (N, r0, r1, t0, t1)
print(f'\nTime slow {t0:.05f} sec, fast {t1:.05f} sec, speedup {round(t0 / max(1e-6, t1))} times')
if __name__ == '__main__':
test()
Output:
test 0, test 2, test 4, test 6, test 8, test 10, test 12, test 14, test 16, test 18, test 19,
Time slow 26.28198 sec, fast 0.00301 sec, speedup 8732 times

For the easiest solution, you can try this:
import math
n=13689 #or we can ask user to input a square number.
for i in range(1,9999):
if math.sqrt(n+i**2).is_integer():
print(i)

Simpson's Rule Takes Forever to Run in Python

I've written the following function for estimating the definite integral of a function with Simpson's Rule:
def fnInt(func, a, b):
if callable(func) and type(a) in [float] and type(b) in [float]:
if a > b:
return -1 * fnInt(func, b, a)
else:
y1 = nDeriv(func)
y2 = nDeriv(y1)
y3 = nDeriv(y2)
y4 = nDeriv(y3)
f = lambda t: abs(y4(t))
k = f(max(f, a, b))
n = ((1 / 0.00001) * k * (b - a) ** 5 / 180) ** 0.25
if n > 0:
n = math.ceil(n) if math.ceil(n) % 2 == 0 else math.ceil(n) + 1
else:
n = 2
x = (b - a) / n
ans = 0
for i in range(int((n - 4) / 2 + 1)):
ans += (x / 3) * (4 * func(a + x * (2 * i + 1)) + 2 * func(a + x * (2 * i + 2)))
ans += (x / 3) * (func(a) + 4 * func(a + x * (n - 1)) + func(b))
return ans
else:
raise TypeError('Data Type Error')
It seems, however, that whenever I try to use this function, it takes forever to produce an output. Is there a way that I can rewrite this code in order to take up less time?

As one of the comments mentioned, profiling the code will show you the slowdowns. Perhaps nDeriv is slow. If you don't have a profiling tool, you can put time() calls around each section of code and print the results. More info here: Measure time elapsed in Python?
So, if the slowdown ends up being in your for loop, here are a few things you can try:
Python might be computing the loop condition every iteration:
for i in range(int((n - 4) / 2 + 1)):
calculate int((n - 4) / 2 + 1) once before the loop.
Don't recalculate stuff inside the loops that doesn't change. For example, x / 3 is going to be recalculated every loop iteration, but it never changes. Do it before the loop starts.
Likewise, you're doing 2 * i twice every loop iteration.
Addition is faster than multiplication. The func arguments could be re-written as:
xi = x * i
a1 = a + xi + xi + x
a2 = a1 + x
and then taking it a step further, you could also re-do xi as an accumulator. That is, start with x = 0, then every iteration simply x += x
This is probably obvious, but if func() is difficult to calculate, this function will be exponentially slow.
Python may be doing a lot of simpler optimizations for you, so these may not help, but just wanted to share some ideas.

python programming help for equation

I am new to python and trying to learn some codes. This is my first programming attempt with python. I have a sequence S and a sequence T(which is also a relation of a couples recurrence relationship equation)where
Sn= 2S(n-1)+S(n-2)+4T(n-1)
and T=S(n-1)+T(n-1).
S0=1, S1=2, T0=0 AND T1=1.
How can i write a function that returns nth value of S and T sequence where the function takes n as a parameter and returns Sn,Tn as a tuple as result of calling the function?

Here are the recursive functions:
def T(n):
if n == 0:
return 0
if n == 1:
return 1
return S(n - 1) + T(n - 1)
def S(n):
if n == 0:
return 1
if n == 1:
return 2
return 2 * S(n - 1) + S(n - 2) + 4 * T(n - 1)
def tuple_func(n):
return(S(n), T(n))
Somewhere between n == 20 and n == 30 this becomes ridiculously slow, depending on your threshold for ridiculousness.
"For fun" I've converted the recursive functions to an iterative version. On my computer it can do up to n == 50,000 in about a second.
def tuple_func(n):
S = [1, 2]
T = [0, 1]
if n < 0:
return(None, None)
if 0 >= n < 2:
return(S[n], T[n])
for n in range(2, n + 1):
S.append(2 * S[n - 1] + S[n - 2] + 4 * T[n - 1])
T.append(S[n - 1] + T[n - 1])
return(S[n], T[n])

Dynamic binding using dictionaries in python?

I know how to do Fibonacci series recursively, that's pretty simple:
def F(n):
if n == 1 or n == 2:
return 1
else:
return k * F(n-2) + F(n - 1)
I do know however that this is extremely inefficient since it must calculate the same values again and again. A way around this is to somehow store values as you go along. Say you want the 20th value. Once you've calculated what F(13) is, its value can be stored and called directly rather than going through all the recursion levels to get the same answer.
Dictionaries are an obvious solution to this issue. However, my answer involving a dictionary does not work.
U = {1:1, 2:1}
def F(n):
global U
if n in U:
return U[n]
else:
U[n] = F(n - 2) + F(n - 1)
Once this code is run just print U[n].
I've run through the logic multiple times and everything seems fine, but I keep getting this error:
TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'
whenever I try a number greater than 3. I don't see how a None should ever be returned, but I concede that I may be missing some sort of nuance about dictionaries that's causing the issue.

You're missing a return statement in the else clause.
def F(n):
global U
if n in U:
return U[n]
else:
U[n] = F(n - 2) + F(n - 1)
return U[n]
Or simplified:
def F(n):
if n not in U:
U[n] = F(n - 2) + F(n - 1)
return U[n]
(global is not needed.)

Simple way using the fact that mutable defaults are only initialized once
def F(n, U={1: 1, 2: 1}):
if n not in U:
U[n] = F(n - 2) + F(n - 1)
return U[n]
This avoid having to have U in the global namespace

When n is not in U, you should also return U[n]
Just:
U = {1:1, 2:1}
def F(n):
global U
if n in U:
return U[n]
else:
U[n] = F(n - 2) + F(n - 1)
return U[n]
For the efficient:
if you want do it once, your second way is sacrifice space, and get time, I can't tell which is better in real dev environment.
if you want get the U[n] many times, store by a U is better
So if you want to store the value, maybe:
U = {1:1, 2:1}
def F(n):
for i in range(3, n+1):
U[i] = U[i - 2] + U[i - 1]
return U
fb = F(10)
print fb[3]
print fb[5]
print fb[10]

If you need to do memoization you might use functools.lru_cache (>= python 3.2). It does exactly what you want, except you don't need to modify the function and create additional data structures.
import functools
MAX_CACHE = 10**7
#functools.lru_cache(MAX_CACHE)
def F(n):
if n == 1 or n == 2:
return 1
else:
return F(n-2) + F(n - 1)
MAX_CACHE is the number of most recent calls to memorize.

Why is this algorithm worse?

In Wikipedia this is one of the given algorithms to generate prime numbers:
def eratosthenes_sieve(n):
# Create a candidate list within which non-primes will be
# marked as None; only candidates below sqrt(n) need be checked.
candidates = [i for i in range(n + 1)]
fin = int(n ** 0.5)
# Loop over the candidates, marking out each multiple.
for i in range(2, fin + 1):
if not candidates[i]:
continue
candidates[i + i::i] = [None] * (n // i - 1)
# Filter out non-primes and return the list.
return [i for i in candidates[2:] if i]
I changed the algorithm slightly.
def eratosthenes_sieve(n):
# Create a candidate list within which non-primes will be
# marked as None; only candidates below sqrt(n) need be checked.
candidates = [i for i in range(n + 1)]
fin = int(n ** 0.5)
# Loop over the candidates, marking out each multiple.
candidates[4::2] = [None] * (n // 2 - 1)
for i in range(3, fin + 1, 2):
if not candidates[i]:
continue
candidates[i + i::i] = [None] * (n // i - 1)
# Filter out non-primes and return the list.
return [i for i in candidates[2:] if i]
I first marked off all the multiples of 2, and then I considered odd numbers only. When I timed both algorithms (tried 40.000.000) the first one was always better (albeit very slightly). I don't understand why. Can somebody please explain?
P.S.: When I try 100.000.000, my computer freezes. Why is that? I have Core Duo E8500, 4GB RAM, Windows 7 Pro 64 Bit.
Update 1: This is Python 3.
Update 2: This is how I timed:
start = time.time()
a = eratosthenes_sieve(40000000)
end = time.time()
print(end - start)
UPDATE: Upon valuable comments (especially by nightcracker and Winston Ewert) I managed to code what I intended in the first place:
def eratosthenes_sieve(n):
# Create a candidate list within which non-primes will be
# marked as None; only c below sqrt(n) need be checked.
c = [i for i in range(3, n + 1, 2)]
fin = int(n ** 0.5) // 2
# Loop over the c, marking out each multiple.
for i in range(fin):
if not c[i]:
continue
c[c[i] + i::c[i]] = [None] * ((n // c[i]) - (n // (2 * c[i])) - 1)
# Filter out non-primes and return the list.
return [2] + [i for i in c if i]
This algorithm improves the original algorithm (mentioned at the top) by (usually) 50%. (Still, worse than the algorithm mentioned by nightcracker, naturally).
A question to Python Masters: Is there a more Pythonic way to express this last code, in a more "functional" way?
UPDATE 2: I still couldn't decode the algorithm mentioned by nightcracker. I guess I'm too stupid.

The question is, why would it even be faster? In both examples you are filtering multiples of two, the hard way. It doesn't matter whether you hardcode candidates[4::2] = [None] * (n // 2 - 1) or that it gets executed in the first loop of for i in range(2, fin + 1):.
If you are interested in an optimized sieve of Eratosthenes, here you go:
def primesbelow(N):
# https://stackoverflow.com/questions/2068372/fastest-way-to-list-all-primes-below-n-in-python/3035188#3035188
#""" Input N>=6, Returns a list of primes, 2 <= p < N """
correction = N % 6 > 1
N = (N, N-1, N+4, N+3, N+2, N+1)[N%6]
sieve = [True] * (N // 3)
sieve[0] = False
for i in range(int(N ** .5) // 3 + 1):
if sieve[i]:
k = (3 * i + 1) | 1
sieve[k*k // 3::2*k] = [False] * ((N//6 - (k*k)//6 - 1)//k + 1)
sieve[(k*k + 4*k - 2*k*(i%2)) // 3::2*k] = [False] * ((N // 6 - (k*k + 4*k - 2*k*(i%2))//6 - 1) // k + 1)
return [2, 3] + [(3 * i + 1) | 1 for i in range(1, N//3 - correction) if sieve[i]]
Explanation here: Porting optimized Sieve of Eratosthenes from Python to C++
The original source is here, but there was no explanation. In short this primesieve skips multiples of 2 and 3 and uses a few hacks to make use of fast Python assignment.

You do not save a lot of time avoiding the evens. Most of the computation time within the algorithm is spent doing this:
candidates[i + i::i] = [None] * (n // i - 1)
That line causes a lot of action on the part of the computer. Whenever the number in question is even, this is not run as the loop bails on the if statement. The time spent running the loop for even numbers is thus really really small. So eliminating those even rounds does not produce a significant change in the timing of the loop. That's why your method isn't considerably faster.
When python produces numbers for range it uses a formula: start + index * step. Multiplying by two as you do in your case is going to be slightly more expensive then one as in the original case.
There is also quite possibly a small overhead to having a longer function.
Neither are of those are really significant speed issues, but they override the very small amount of benefit your version brings.

Its probably slightly slower because you are performing extra set up to do something that was done in the first case anyway (marking off multiples of two). That setup time might be what you see if it is as slight as you say

Your extra step is unnecessary and will actually traverse the whole collection n once doing that 'get rid of evens' operation rather than just operating on n^1/2.