We Keep Coding

Gaussian Addition - Python Challenge

I am currently working on some beginner python challenges, I just finished a gaussian addition challenge. I was able to get the output that the challenge was looking for, but it seems like I over complicated things.
The challenge is as follows:
Write a program that passes a list of numbers to a function.
The function should use a while loop to keep popping the first and last numbers from the list and calculate the sum of those two numbers.
The function should print out the current numbers that are being added, and print their partial sum.
The function should keep track of how many partial sums there are.
The function should then print out how many partial sums there were.
The function should perform Gauss' multiplication, and report the final answer.
Prove that your function works, by passing in the range 1-100, and verifying that you get 5050.
gauss_addition(list(range(1,101)))
Your function should work for any set of consecutive numbers, as long as that set has an even length.
Bonus: Modify your function so that it works for any set of consecutive numbers, whether that set has an even or odd length.
My function is as follows:
def gauss(numbers):
for number in numbers:
while len(numbers) > 0:
num1 = numbers.pop(0)
print(num1)
num2 = numbers.pop(-1)
print(num2)
calc = num1 + num2
print(str(num1) + " + " + str(num2) + " = " + str(calc))
print("Final answer is: " + str(num1 * calc))
gauss(list(range(1,101)))
Can someone explain how I can simplify this function without the use of python modules? I understand how the function that I wrote is working, but I also want to know if there is an easier, "more condensed" way of achieving this.
I should specify that I only know the basics of python...

Using list to approach this problem seems to be too expensive, for the reason of repeatedly pop will be costly as cited in early notes.
So instead of it, you can consider using collections module deque which will allow you to do the operations (pop) in both end efficiently.
Note - it will work well if the list is even-sized, but that's part of requirements already.
So the solution will be like this:
#
from collections import deque
dq = deque(range(1, 101)) # create a list (dq list)
total = 0
count = 0
while dq:
x, y = dq.popleft(), dq.pop()
print(x, y, x+y, end='\t')
count += 1 # how many partial sum?
print(count)
total += (x+y)
print(f' final total: {total} ')
Outputs: (partial - it's too long)
1 100 101 1
2 99 101 2
3 98 101 3
4 97 101 4
5 96 101 5
...........
...........
48 53 101 48
49 52 101 49
50 51 101 50
final total: 5050

Project Euler #25 - Can the performance be improved further?

The 12th term, F12, is the first term to contain three digits.
What is the index of the first term in the Fibonacci sequence to contain 1000 digits?
a = 1
b = 1
i = 2
while(1):
c = a + b
i += 1
length = len(str(c))
if length == 1000:
print(i)
break
a = b
b = c
I got the answer(works fast enough). Just looking if there's a better way for this question

If you've answered the question, you'll find plenty of explanations on answers in the problem thread. The solution you posted is pretty much okay. You may get a slight speedup by simply checking that your c>=10^999 at every step instead of first converting it to a string.
The better method is to use the fact that when the Fibonacci numbers become large enough, the Fibonacci numbers converge to round(phi**n/(5**.5)) where phi=1.6180... is the golden ratio and round(x) rounds x to the nearest integer. Let's consider the general case of finding the first Fibonacci number with more than m digits. We are then looking for n such that round(phi**n/(5**.5)) >= 10**(m-1)
We can easily solve that by just taking the log of both sides and observe that
log(phi)*n - log(5)/2 >= m-1 and then solve for n.
If you're wondering "well how do I know that it has converged by the nth number?" Well, you can check for yourself, or you can look online.
Also, I think questions like these either belong on the Code Review SE or the Computer Science SE. Even Math Overflow might be a good place for Project Euler questions, since many are rooted in number theory.

Your solution is completely fine for #25 on project euler. However, if you really want to optimize for speed here you can try to calculate fibonacci using the identities I have written about in this blog post: https://sloperium.github.io/calculating-the-last-digits-of-large-fibonacci-numbers.html
from functools import lru_cache
#lru_cache(maxsize=None)
def fib4(n):
if n <= 1:
return n
if n % 2:
m = (n + 1) // 2
return fib4(m) ** 2 + fib4(m - 1) ** 2
else:
m = n // 2
return (2 * fib4(m - 1) + fib4(m)) * fib4(m)
def binarySearch( length):
first = 0
last = 10**5
found = False
while first <= last and not found:
midpoint = (first + last) // 2
length_string = len(str(fib4(midpoint)))
if length_string == length:
return midpoint -1
else:
if length < length_string:
last = midpoint - 1
else:
first = midpoint + 1
print(binarySearch(1000))
This code tests about 12 times faster than your solution. (it does require an initial guess about max size though)

Memory management for python scripts

So I'm trying to solve some problems from the Euler project in python. I'm currently working on Problem 92, square digit chains. Basically the idea is that if you take any integer, and square its component digits recursively (e.g. 42 = 42 + 22 = 20, then 22 + 02 = 4, etc.), you always end up either at 1 or 89.
I am trying to write a program that can compute how many numbers, in a range 1 to 10K, will end up in 89 and how many will end up in 1. I am not trying to store which integers end up where, only how many. The goal is to be able to do that for the largest K possible. (This is a challenge from Hackerrank for those curious).
In other to do for large number within my lifetime, I need to use caching. But then that's a balancing act between caching (which eventually takes up lots of RAM) and computing time.
My problem is that I eventually run out of memory. So I have tried to cap the length of the cache that I am using. However, I still run out of memory. I cannot seem to be able to find what is causing me to run out of memory.
I am running it on pycharm on ubuntu 14.04 LTS.
My question:
Is there a way to check what is taking up my RAM? Is there some tool (or script) that can allow me to basically monitor memory use by variables within my program? Or an wrong in assuming that if I run out of RAM, it is necessarily because some variable in my program is too large? I have to admit I am not all that clear on the fine details of memory use within a program....
EDIT: I run out of mem when K = 8, so for integers up to 108, which is not so large. Also, I did testing before 108 (so 107, which terminates but takes some time and uses more memory than smaller computation). And it doesn't seem that capping my cache size variables makes a differences.....

I would suggest testing various cache sizes to see if it is actually beneficial to have as large a cache as possible.
If you take any 10-digit number and compute the sum of squares of its digits, the sum will be at most 10*9*9 = 810. Thus, if you cache the result for numbers 1 to 810, then you should be able to process all numbers with between 4 and 10 digits without recursion.
In this way, I have processed the first 10^8 numbers in around 6 minutes with memory usage staying constant at roughly 10 MB.

This is a variation of Mathias Rav's excellent idea but keeps your idea of using a recursive function with memozation. The idea is to use a helper function to do the heavy lifting and have the main function just do the first step of the iteration. The very first step reduces the problem size to one for which caching is useful. The cache remains small. I was able to do all numbers up to 10**8 in about 10 minutes (the overhead due to the recursion makes this solution less efficient than Mathias' solution):
cache = {}
def helper(n):
if n == 1 or n == 89:
return n
elif n in cache:
return cache[n]
else:
ss = sum(int(d)**2 for d in str(n))
v = helper(ss)
cache[n] = v
return v
def f(n):
ss = sum(int(d)**2 for d in str(n))
return helper(ss)
def freq89(n):
total = 0
for i in range(1,n+1):
if f(i) == 89: total += 1
return total/n

This is an extended comment on the answers by Mathias Rav and John Coleman. I was going to make this a community wiki answer. John Coleman said not to do so, so I'm not.
I'll start with John Coleman's answer.
cache = {}
def helper(n):
if n == 1 or n == 89:
return n
elif n in cache:
return cache[n]
else:
ss = sum(int(d)**2 for d in str(n))
v = helper(ss)
cache[n] = v
return v
def f(n):
ss = sum(int(d)**2 for d in str(n))
return helper(ss)
A small thing that will speed things up a bit is to avoid that first if in helper(n) by initializing cache to {1:some_value, 89:some_other_value}. The obvious initialization is {1:1, 89:89}. A less obvious, but ultimately faster initialization is {1:False, 89:True}. This enables changing if f(i) == 89: total += 1 to if f(i): total += 1.
Another small thing that might help is to get rid of the recursion. That's not the case here. To get rid of the recursion, we'd have to do something along the lines of
def helper(n):
l = []
while n not in cache :
l.append(n)
n = sum(int(d)**2 for d in str(n))
v = cache[n]
for k in l :
cache[k] = v
return v
The problem is that almost all of the numbers encountered by f(n) will already be in the cache thanks to how helper is called from f(n). Getting rid of the recursion needlessly creates an empty list that needs to be garbage collected.
The big issue with John Coleman's answer is the calculation of the sum of the square of the digits via sum(int(d)**2 for d in str(n)). While very pythonic, this is extremely expensive. I'll start by changing the variable ss in helper and in f into a function:
def ss(n):
return sum(int(d)**2 for d in str(n))
This alone does nothing for performance. In fact, it hurts performance. Function calls are expensive in python. By making this a function, we can do some non-pythonic things by replacing the string operations with integer arithmetic:
def ss(n):
s = 0
while n != 0:
d = n % 10
n = n // 10
s += d**2
return s
The speedup here is quite significant; I get a 30% reduction in computation time. That's not great. There's another problem, the use of the exponentiation operator. In almost any language but Fortran and Matlab, using d*d is much faster than is d**2. That's certainly the case in python. That simple change almost halves the execution time from that already significant 30% reduction.
Putting this all together yields
cache = {1:False, 89:True}
def ss (n):
s = 0
while n != 0:
d = n % 10
n = n // 10
s += d*d
return s
def helper(n):
if n in cache:
return cache[n]
else:
v = helper(ss(n))
cache[n] = v
return v
def f(n):
return helper(ss(n))
def freq89(n):
total = 0
for i in range(1,n+1):
if f(i): total += 1
return total/n
print (freq89(int(1e7)))
I have yet to take advantage of Mathias Rav's answer. In this case, it will make sense to get rid of the recursion. It will also help to embed the loop over the initial range inside of the function that initializes the cache (function calls are expensive in python).
N = int(1e7)
cache = {1:False, 89:True}
def ss(n):
s = 0
while n != 0:
d = n % 10
n //= 10
s += d*d
return s
def initialize_cache(maxsum):
for n in range(1,maxsum+1):
l = []
while n not in cache:
l.append(n)
n = ss(n)
v = cache[n]
for k in l:
cache[k] = v
def freq89(n):
total = 0
for i in range(1,n):
if cache[ss(i)]:
total += 1
return total/n
maxsum = 81*len(str(N-1))
initialize_cache(maxsum)
print (freq89(N))
The above takes about 16.5 seconds (on my computer) to calculate the ratio for numbers between 1 (inclusive) and 10000000 (exclusive) on my computer. This is almost three times faster than the initial version (44.7 seconds). It takes a bit over three minutes for the above to calculate calculate the ratio for numbers between 1 (inclusive) and 1e8 (exclusive).
It turns out I'm not done. There's no need to calculate the sum of the squares of the digits of (for example) 12345679 digit by digit when the program just did that for 12345678. A shortcut that reduces the calculation time for nine out of ten use cases pays off. The function ss(n) becomes a bit more complex:
prevn = 0
prevd = 0
prevs = 0
def ss(n):
global prevn, prevd, prevs
d = n % 10
if (n == prevn+1) and (d == prevd+1):
s = prevs + 2*prevd + 1
prevs = s
prevn = n
prevd = d
return s
s = 0
prevn = n
prevd = d
while n != 0:
d = n % 10
n //= 10
s += d*d
prevs = s
return s
With this, calculating the ratio for numbers up to (but not including) 1e7 takes 6.6 seconds, 68 seconds for numbers up to but not including 1e8.

Finding sum: Is recursion necessary, and what would a recursive algorithm look like? [closed]

This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 9 years ago.
def numPens(n):
"""
n is a non-negative integer
Returns True if some non-negative integer combination of 5, 8 and 24 equals n
Otherwise returns False.
"""
if n < 5:
return False
N = n
while N >= 0:
if N % 24 == 0 or N % 8 == 0 or N % 5 == 0: # if N / 24 is equal to 0 then we can buy N pens
return True
if N < 5:
return False # if N < 5 we cannot buy any pens
if N > 24: # if N is greater than 24 , take away 24 and continue loop
N -= 24
elif N > 8: # if N is greater than 8, take away 8 and continue loop
N -= 8
else:
N -= 5 # else take away 5 and continue loop
I had to create this function for a test, I am just wondering if the problem can be sorted recursively or what would be most efficient solution, I am new to programming so any help would be great, thanks.

if N % 24 == 0 or N % 8 == 0 or N % 5 == 0
If you get rid of the above modulus (%) checks then your algorithm is what's known as a greedy algorithm. It subtracts the largest number it can each iteration. As you might have noticed, the greedy algorithm doesn't work. It gives the wrong answer for 15 = 5 + 5 + 5, for example.
15 (-8) --> 7 (-5) --> 2 --> False
By adding in the modulus checks you've improved the greedy algorithm because it now correctly handles 15. But it still has holes: for instance, 26 = 8 + 8 + 5 + 5.
26 (-24) --> 2 --> False
In order to correctly solve this problem you must abandon the greedy approach. It's not always sufficient to subtract the largest number possible. To answer your question, yes, a recursive solution is called for here.
def numPens(n):
"""
n is a non-negative integer
Returns True if some non-negative integer combination of 5, 8 and 24 equals n
Otherwise returns False.
"""
# Base case: Negative numbers are by definition false.
if n < 0:
return False
# Base case: 0 is true. It is formed by a combination of zero addends,
# and zero is a non-negative integer.
if n == 0:
return True
# General case: Try subtracting *each* of the possible numbers, not just
# the largest one. No matter what n-x will always be smaller than n so
# eventually we'll reach one of the base cases (either a negative number or 0).
for x in (24, 8, 5):
if numPens(n - x):
return True
return False
This is the most straightforward way to solve the problem and will work reasonably well for smallish numbers. For large numbers it will be slow due to the way it evaluates the same numbers multiple times. An optimization left to the reader is to use dynamic programming to eliminate duplicate calculations.

There are more efficient (O(1)) algorithms.
For instance, you can add
if n > 40:
return True
as one of your base cases!
You can make it even more efficient, by maintaining a lookup table for the rest of the values (n < 40).
The reason you can do this is this: http://en.wikipedia.org/wiki/Coin_problem#n_.3D_2

Obviously, it's a recursive problem, below is maybe the simplest code:
def numPens(n):
if n < 5:
return False
elif n==5 or n==8 or n==24:
return True
else:
return numPens(n-5) or numPens(n-8) or numPens(n-24)
if you need to be more efficient and robust, you can improve by yourself.

n=5a+8b+24c <=> n=5a+8(b+3c),
hence you could have a function :
def numPens(n):
if n < 5:
return False
if n % 8 == 0 or n % 5 == 0:
return True
else:
return numPens(n-8) or numPens(n-5)

def numPens(n):
global cantidad
cantidad = cantidad + 1
if n < 5:
return False
elif n%5==0 or n%8==0 or n%24==0:
return True
else:
return numPens(n-5) or numPens(n-8) or numPens(n-24)

An informal definition of recursion given at http://en.wikipedia.org/wiki/Recursive defines recursion as: "Recursion is the process a procedure goes through when one of the steps of the procedure involves invoking the procedure itself." On the other hand iteration is defined as: "Iteration means the act of repeating a process with the aim of approaching a desired goal, target or result. Each repetition of the process is also called an "iteration," and the results of one iteration are used as the starting point for the next iteration." http://en.wikipedia.org/wiki/Itteration
As you can see the two processes are very similar. Any loop that is not infinite will be iterative, and could be written in a recursive manner, and any recursive call could be written as an iterative loop. However in many cases one method or the other is much easier to implement.

Project Euler #25: Keep getting Overflow error (result to large) - is it to do with calculating fibonacci number?

I'm working on solving the Project Euler problem 25:
What is the first term in the Fibonacci sequence to contain 1000
digits?
My piece of code works for smaller digits, but when I try a 1000 digits, i get the error:
OverflowError: (34, 'Result too large')
I'm thinking it may be on how I compute the fibonacci numbers, but i've tried several different methods, yet i get the same error.
Here's my code:
'''
What is the first term in the Fibonacci sequence to contain 1000 digits
'''
def fibonacci(n):
phi = (1 + pow(5, 0.5))/2 #Golden Ratio
return int((pow(phi, n) - pow(-phi, -n))/pow(5, 0.5)) #Formula: http://bit.ly/qDumIg
n = 0
while len(str(fibonacci(n))) < 1000:
n += 1
print n
Do you know what may the cause of this problem and how i could alter my code avoid this problem?
Thanks in advance.

The problem here is that only integers in Python have unlimited length, floating point values are still calculated using normal IEEE types which has a maximum precision.
As such, since you're using an approximation, using floating point calculations, you will get that problem eventually.
Instead, try calculating the Fibonacci sequence the normal way, one number (of the sequence) at a time, until you get to 1000 digits.
ie. calculate 1, 1, 2, 3, 5, 8, 13, 21, 34, etc.
By "normal way" I mean this:
/ 1 , n < 3
Fib(n) = |
\ Fib(n-2) + Fib(n-1) , n >= 3
Note that the "obvious" approach given the above formulas is wrong for this particular problem, so I'll post the code for the wrong approach just to make sure you don't waste time on that:
def fib(n):
if n <= 3:
return 1
else:
return fib(n-2) + fib(n-1)
n = 1
while True:
f = fib(n)
if len(str(f)) >= 1000:
print("#%d: %d" % (n, f))
exit()
n += 1
On my machine, the above code starts going really slow at around the 30th fibonacci number, which is still only 6 digits long.
I modified the above recursive approach to output the number of calls to the fib function for each number, and here are some values:
#1: 1
#10: 67
#20: 8361
#30: 1028457
#40: 126491971
I can reveal that the first Fibonacci number with 1000 digits or more is the 4782th number in the sequence (unless I miscalculated), and so the number of calls to the fib function in a recursive approach will be this number:
1322674645678488041058897524122997677251644370815418243017081997189365809170617080397240798694660940801306561333081985620826547131665853835988797427277436460008943552826302292637818371178869541946923675172160637882073812751617637975578859252434733232523159781720738111111789465039097802080315208597093485915332193691618926042255999185137115272769380924184682248184802491822233335279409301171526953109189313629293841597087510083986945111011402314286581478579689377521790151499066261906574161869200410684653808796432685809284286820053164879192557959922333112075826828349513158137604336674826721837135875890203904247933489561158950800113876836884059588285713810502973052057892127879455668391150708346800909439629659013173202984026200937561704281672042219641720514989818775239313026728787980474579564685426847905299010548673623281580547481750413205269166454195584292461766536845931986460985315260676689935535552432994592033224633385680958613360375475217820675316245314150525244440638913595353267694721961
And that is just for the 4782th number. The actual value is the sum of all those values for all the fibonacci numbers from 1 up to 4782. There is no way this will ever complete.
In fact, if we would give the code 1 year of running time (simplified as 365 days), and assuming that the machine could make 10.000.000.000 calls every second, the algorithm would get as far as to the 83rd number, which is still only 18 digits long.

Actually, althought the advice given above to avoid floating-point numbers is generally good advice for Project Euler problems, in this case it is incorrect. Fibonacci numbers can be computed by the formula F_n = phi^n / sqrt(5), so that the first fibonacci number greater than a thousand digits can be computed as 10^999 < phi^n / sqrt(5). Taking the logarithm to base ten of both sides -- recall that sqrt(5) is the same as 5^(1/2) -- gives 999 < n log_10(phi) - 1/2 log_10(5), and solving for n gives (999 + 1/2 log_10(5)) / log_10(phi) < n. The left-hand side of that equation evaluates to 4781.85927, so the smallest n that gives a thousand digits is 4782.

You can use the sliding window trick to compute the terms of the Fibonacci sequence iteratively, rather than using the closed form (or doing it recursively as it's normally defined).
The Python version for finding fib(n) is as follows:
def fib(n):
a = 1
b = 1
for i in range(2, n):
b = a + b
a = b - a
return b
This works when F(1) is defined as 1, as it is in Project Euler 25.
I won't give the exact solution to the problem here, but the code above can be reworked so it keeps track of n until a sentry value (10**999) is reached.

An iterative solution such as this one has no trouble executing. I get the answer in less than a second.
def fibonacci():
current = 0
previous = 1
while True:
temp = current
current = current + previous
previous = temp
yield current
def main():
for index, element in enumerate(fibonacci()):
if len(str(element)) >= 1000:
answer = index + 1 #starts from 0
break
print(answer)

import math as m
import time
start = time.time()
fib0 = 0
fib1 = 1
n = 0
k = 0
count = 1
while k<1000 :
n = fib0 + fib1
k = int(m.log10(n))+1
fib0 = fib1
fib1 = n
count += 1
print n
print count
print time.time()-start
takes 0.005388 s on my pc. did nothing fancy just followed simple code.
Iteration will always be better. Recursion was taking to long for me as well.
Also used a math function for calculating the number of digits in a number instead of taking the number in a list and iterating through it. Saves a lot of time

Here is my very simple solution
list = [1,1,2]
for i in range(2,5000):
if len(str(list[i]+list[i-1])) == 1000:
print (i + 2)
break
else:
list.append(list[i]+list[i-1])
This is sort of a "rogue" way of doing it, but if you change the 1000 to any number except one, it gets it right.

You can use the datatype Decimal. This is a little bit slower but you will be able to have arbitrary precision.
So your code:
'''
What is the first term in the Fibonacci sequence to contain 1000 digits
'''
from Decimal import *
def fibonacci(n):
phi = (Decimal(1) + pow(Decimal(5), Decimal(0.5))) / 2 #Golden Ratio
return int((pow(phi, Decimal(n))) - pow(-phi, Decimal(-n)))/pow(Decimal(5), Decimal(0.5)))
n = 0
while len(str(fibonacci(n))) < 1000:
n += 1
print n