We Keep Coding

Using Floor/Ceil to round decimals to integer of 5

I have been using the code below to handily round to 1dp where I needed the .5s to round upwards, e.g. I wanted a value such as 36.25 to round to 36.3, not 36.2.
def my_round(n, ndigits=1):
try:
part = n * 10 ** ndigits
delta = part - int(part)
# always round "away from 0"
if delta >= 0.5 or -0.5 < delta <= 0:
part = math.ceil(part)
else:
part = math.floor(part)
val = part/(10 ** ndigits)
except ValueError:
val = np.nan
return val
I now need to round a different decimal column in a df to the nearest 5 whole numbers. Can anyone help suggest how I can tweak my original code above to do this? I need, for example, a value such as 702.5000 rounding to 705. Using the normal Round() it obviously goes to 700. In the example, all other normal rounding rules should apply e.g.
702.222 to 700,
707.466 to 705,
703.021 to 705

To round to a certain number of decimal places, use this:
def my_round(x: float, round_to: float = 5, dp: int = 0) -> float:
round_to *= 10**dp
x *= 10**dp
return ((x + round_to / 2) // round_to * round_to) / 10**dp
Testing it:
values = [702.500, 702.222, 707.466, 703.021]
targets = [705., 700., 705., 705.]
assert [my_round(x, 5, 0) for x in values] == targets
assert my_round(3.25, 0.1, 1) == 36.3

def my_round(n):
lower = (n//5)*5;
upper = lower+5;
if (n-lower)<(upper-n):
return int(lower)
return int(upper)
print(my_round(703.021))
The above program tries to find out the proper multiple of 5 before and after the given number.
Then it finds the differences between the lower and upper possible number and returns the number with least difference. This is as good as rounding.

The quickest way would be to divide by 5, round and then multiply by 5. It's purely mathematical fact that this will work.
5*round(n/5)
And there's no need for the if statement deciding on when to use floor or ceil since round already defaults to this logic.
EDIT: As pointed out this works for all values except those ending in 2.5 when requiring that this rounds up (in practice many examples won't need this).
As Ashwin's answer accomplishes this already, I'll give a fix to do it on one line without a function definition, though it's not pretty:
5 * (int(x/5) + 1 - round(x/5 % 1))
Or slightly cleaner with ceil:
5*(math.ceil(x/5) - round(x/5 % 1))
exploiting the the fact that round behaves as we would like from above.
Even better, define:
def my_round(x): return int(x + 1) - round(x % 1)
def round5(x): return 5*my_round(x/5)
to 'fix' the round function and create a nice round5 function in 2 lines. This is what I'd do.

You could divide the number you are trying to round by 5, then take its round() and multiply again by five:
num = 707.466
int = num / 5 # returns 141.4932
rounded = round(int) # returns 141
answer = rounded * 5 # 705
Or as a function:
def round_to_five(num):
num = num / 5
num = round(num)
return num * 5

Hope this helps
# my standard point to which rounding shoulding happen
>>> standard_nums = [700, 702, 705, 708]
# example value I am taking
>>> my_num = 702.5
# this is how I would round my number
>>> ret = abs(standard_nums[0] - my_num)
>>> val = standard_nums[0]
>>> for each in standard_nums[1:]:
... if abs(each - my_num) < ret:
... ret = abs(each - my_num)
... val = each
...
>>> ret # min diff I can find
0.5
>>> val # value you are lookin for
702
>>>
## this above has a complexity of O(N)
## where N is the size of your standard number

Assuming that you use Python 3, would the following line do the job ?
5*(x//5)+5*(x%5>=2.5)
It seems to work for x being a scalar, numpy.array or a pandas.series

How to find sum of cubes of the divisors for every number from 1 to input number x in python where x can be very large

Examples,
1.Input=4
Output=111
Explanation,
1 = 1³(divisors of 1)
2 = 1³ + 2³(divisors of 2)
3 = 1³ + 3³(divisors of 3)
4 = 1³ + 2³ + 4³(divisors of 4)
------------------------
sum = 111(output)
1.Input=5
Output=237
Explanation,
1 = 1³(divisors of 1)
2 = 1³ + 2³(divisors of 2)
3 = 1³ + 3³(divisors of 3)
4 = 1³ + 2³ + 4³(divisors of 4)
5 = 1³ + 5³(divisors of 5)
-----------------------------
sum = 237 (output)
x=int(raw_input().strip())
tot=0
for i in range(1,x+1):
for j in range(1,i+1):
if(i%j==0):
tot+=j**3
print tot
Using this code I can find the answer for small number less than one million.
But I want to find the answer for very large numbers. Is there any algorithm
for how to solve it easily for large numbers?

Offhand I don't see a slick way to make this truly efficient, but it's easy to make it a whole lot faster. If you view your examples as matrices, you're summing them a row at a time. This requires, for each i, finding all the divisors of i and summing their cubes. In all, this requires a number of operations proportional to x**2.
You can easily cut that to a number of operations proportional to x, by summing the matrix by columns instead. Given an integer j, how many integers in 1..x are divisible by j? That's easy: there are x//j multiples of j in the range, so divisor j contributes j**3 * (x // j) to the grand total.
def better(x):
return sum(j**3 * (x // j) for j in range(1, x+1))
That runs much faster, but still takes time proportional to x.
There are lower-level tricks you can play to speed that in turn by constant factors, but they still take O(x) time overall. For example, note that x // j == 1 for all j such that x // 2 < j <= x. So about half the terms in the sum can be skipped, replaced by closed-form expressions for a sum of consecutive cubes:
def sum3(x):
"""Return sum(i**3 for i in range(1, x+1))"""
return (x * (x+1) // 2)**2
def better2(x):
result = sum(j**3 * (x // j) for j in range(1, x//2 + 1))
result += sum3(x) - sum3(x//2)
return result
better2() is about twice as fast as better(), but to get faster than O(x) would require deeper insight.
Quicker
Thinking about this in spare moments, I still don't have a truly clever idea. But the last idea I gave can be carried to a logical conclusion: don't just group together divisors with only one multiple in range, but also those with two multiples in range, and three, and four, and ... That leads to better3() below, which does a number of operations roughly proportional to the square root of x:
def better3(x):
result = 0
for i in range(1, x+1):
q1 = x // i
# value i has q1 multiples in range
result += i**3 * q1
# which values have i multiples?
q2 = x // (i+1) + 1
assert x // q1 == i == x // q2
if i < q2:
result += i * (sum3(q1) - sum3(q2 - 1))
if i+1 >= q2: # this becomes true when i reaches roughly sqrt(x)
break
return result
Of course O(sqrt(x)) is an enormous improvement over the original O(x**2), but for very large arguments it's still impractical. For example better3(10**6) appears to complete instantly, but better3(10**12) takes a few seconds, and better3(10**16) is time for a coffee break ;-)
Note: I'm using Python 3. If you're using Python 2, use xrange() instead of range().
One more
better4() has the same O(sqrt(x)) time behavior as better3(), but does the summations in a different order that allows for simpler code and fewer calls to sum3(). For "large" arguments, it's about 50% faster than better3() on my box.
def better4(x):
result = 0
for i in range(1, x+1):
d = x // i
if d >= i:
# d is the largest divisor that appears `i` times, and
# all divisors less than `d` also appear at least that
# often. Account for one occurence of each.
result += sum3(d)
else:
i -= 1
lastd = x // i
# We already accounted for i occurrences of all divisors
# < lastd, and all occurrences of divisors >= lastd.
# Account for the rest.
result += sum(j**3 * (x // j - i)
for j in range(1, lastd))
break
return result
It may be possible to do better by extending the algorithm in "A Successive Approximation Algorithm for Computing the Divisor Summatory Function". That takes O(cube_root(x)) time for the possibly simpler problem of summing the number of divisors. But it's much more involved, and I don't care enough about this problem to pursue it myself ;-)
Subtlety
There's a subtlety in the math that's easy to miss, so I'll spell it out, but only as it pertains to better4().
After d = x // i, the comment claims that d is the largest divisor that appears i times. But is that true? The actual number of times d appears is x // d, which we did not compute. How do we know that x // d in fact equals i?
That's the purpose of the if d >= i: guarding that comment. After d = x // i we know that
x == d*i + r
for some integer r satisfying 0 <= r < i. That's essentially what floor division means. But since d >= i is also known (that's what the if test ensures), it must also be the case that 0 <= r < d. And that's how we know x // d is i.
This can break down when d >= i is not true, which is why a different method needs to be used then. For example, if x == 500 and i == 51, d (x // i) is 9, but it's certainly not the case that 9 is the largest divisor that appears 51 times. In fact, 9 appears 500 // 9 == 55 times. While for positive real numbers
d == x/i
if and only if
i == x/d
that's not always so for floor division. But, as above, the first does imply the second if we also know that d >= i.
Just for Fun
better5() rewrites better4() for about another 10% speed gain. The real pedagogical point is to show that it's easy to compute all the loop limits in advance. Part of the point of the odd code structure above is that it magically returns 0 for a 0 input without needing to test for that. better5() gives up on that:
def isqrt(n):
"Return floor(sqrt(n)) for int n > 0."
g = 1 << ((n.bit_length() + 1) >> 1)
d = n // g
while d < g:
g = (d + g) >> 1
d = n // g
return g
def better5(x):
assert x > 0
u = isqrt(x)
v = x // u
return (sum(map(sum3, (x // d for d in range(1, u+1)))) +
sum(x // i * i**3 for i in range(1, v)) -
u * sum3(v-1))

def sum_divisors(n):
sum = 0
i = 0
for i in range (1, n) :
if n % i == 0 and n != 0 :
sum = sum + i
# Return the sum of all divisors of n, not including n
return sum
print(sum_divisors(0))
# 0
print(sum_divisors(3)) # Should sum of 1
# 1
print(sum_divisors(36)) # Should sum of 1+2+3+4+6+9+12+18
# 55
print(sum_divisors(102)) # Should be sum of 2+3+6+17+34+51
# 114

How can I re-calculate the common exponent?

To explain this, this is basically a way to shrink floating point vector data into 8-bit or 16-bit signed or unsigned integers with a single common unsigned exponent (the most common of which being bs16 for precision with a common exponent of 11).
I'm not sure what this pseudo-float method is called; all I know is to get the resulting float, you need to do this:
float_result = int_value / ( 2.0 ** exponent )
What I'd like to do is match this data by basically guessing the exponent by attempting to re-calculate it from the given floats.
(if done properly, it should be able to be re-calculated in other formats as well)
So if all I'm given is a large group of 1140 floats to work with, how can I find the common exponent and convert these floats into this shrunken bu8, bs8, bu16, or bs16 (specified) format?
EDIT: samples
>>> for value in array('h','\x28\xC0\x04\xC0\xF5\x00\x31\x60\x0D\xA0\xEB\x80'):
print( value / ( 2. ** 11 ) )
-7.98046875
-7.998046875
0.11962890625
12.0239257812
-11.9936523438
-15.8852539062
EDIT2:
I wouldn't exactly call this "compression", as all it really is, is an extracted mantissa to be re-computed via the shared exponent.

Maybe something like this:
def validExponent(x,e,a,b):
"""checks if x*2.0**e is an integer in range [a,b]"""
y = x*2.0**e
return a <= y <= b and y == int(y)
def allValid(xs,e,a,b):
return all(validExponent(x,e,a,b) for x in xs)
def firstValid(xs,a,b,maxE = 100):
for e in xrange(1+maxE):
if allValid(xs,e,a,b):
return e
return "None found"
#test:
xs = [x / ( 2. ** 11 ) for x in [-12,14,-5,16,28]]
print xs
print firstValid(xs,-2**15,2**15-1)
Output:
[-0.005859375, 0.0068359375, -0.00244140625, 0.0078125, 0.013671875]
11
You could of course write a wrapper function which will take a string argument such as 'bs16' and automatically compute the bounds a,b
On Edit:
1) If you have the exact values of the floats the above should work. It anything has introduced any round-off error you might want to replace y == int(y) by abs(y-round(y)) < 0.00001 (or something similar).
2) The first valid exponent will be the exponent you want unless all of the integers in the original integer list are even. If you have 1140 values and they are in some sense random, the chance of this happening is vanishingly small.
On Further Edit: If the floats in question are not generated by this process but you want to find an optimal exponent which allows for (lossy) compression to ints of a given size you can do something like this (not thoroughly tested):
import math
def maxExp(x,a,b):
"""returns largest nonnegative integer exponent e with
a <= x*2**e <= b, where a, b are integers with a <= 0 and b > 0
Throws an error if no such e exists"""
if x == 0.0:
e = -1
elif x < 0.0:
e = -1 if a == 0 else math.floor(math.log(a/float(x),2))
else:
e = math.floor(math.log(b/float(x),2))
if e >= 0:
return int(e)
else:
raise ValueError()
def bestExponent(floats,a,b):
m = min(floats)
M = max(floats)
e1 = maxExp(m,a,b)
e2 = maxExp(M,a,b)
MSE = []
for e in range(1+min(e1,e2)):
MSE.append(sum((x - round(x*2.0**e)/2.0**e)**2 for x in floats)/float(len(floats)))
minMSE = min(MSE)
for e,error in enumerate(MSE):
if error == minMSE:
return e
To test it:
>>> import random
>>> xs = [random.uniform(-10,10) for i in xrange(1000)]
>>> bestExponent(xs,-2**15,2**15-1)
11
It seems like the common exponent 11 is chosen for a reason.

If you've got the original values, and the corresponding result, you can use log to find the exponent. Math has a log function you can use. You'd have to log Int_value/float_result to the base 2.
EG:
import Math
x = (int_value/float_result)
math.log(x,2)

Python: Streamlining Code for Brown Numbers

I was curious if any of you could come up with a more streamline version of code to calculate Brown numbers. as of the moment, this code can do ~650! before it moves to a crawl. Brown Numbers are calculated thought the equation n! + 1 = m**(2) Where M is an integer
brownNum = 8
import math
def squareNum(n):
x = n // 2
seen = set([x])
while x * x != n:
x = (x + (n // x)) // 2
if x in seen: return False
seen.add(x)
return True
while True:
for i in range(math.factorial(brownNum)+1,math.factorial(brownNum)+2):
if squareNum(i) is True:
print("pass")
print(brownNum)
print(math.factorial(brownNum)+1)
break
else:
print(brownNum)
print(math.factorial(brownNum)+1)
brownNum = brownNum + 1
continue
break
print(input(" "))

Sorry, I don't understand the logic behind your code.
I don't understand why you calculate math.factorial(brownNum) 4 times with the same value of brownNum each time through the while True loop. And in the for loop:
for i in range(math.factorial(brownNum)+1,math.factorial(brownNum)+2):
i will only take on the value of math.factorial(brownNum)+1
Anyway, here's my Python 3 code for a brute force search of Brown numbers. It quickly finds the only 3 known pairs, and then proceeds to test all the other numbers under 1000 in around 1.8 seconds on this 2GHz 32 bit machine. After that point you can see it slowing down (it hits 2000 around the 20 second mark) but it will chug along happily until the factorials get too large for your machine to hold.
I print progress information to stderr so that it can be separated from the Brown_number pair output. Also, stderr doesn't require flushing when you don't print a newline, unlike stdout (at least, it doesn't on Linux).
import sys
# Calculate the integer square root of `m` using Newton's method.
# Returns r: r**2 <= m < (r+1)**2
def int_sqrt(m):
if m <= 0:
return 0
n = m << 2
r = n >> (n.bit_length() // 2)
while True:
d = (n // r - r) >> 1
r += d
if -1 <= d <= 1:
break
return r >> 1
# Search for Browns numbers
fac = i = 1
while True:
if i % 100 == 0:
print('\r', i, file=sys.stderr, end='')
fac *= i
n = fac + 1
r = int_sqrt(n)
if r*r == n:
print('\nFound', i, r)
i += 1

You might want to:
pre calculate your square numbers, instead of testing for them on the fly
pre calculate your factorial for each loop iteration num_fac = math.factorial(brownNum) instead of multiple calls
implement your own, memoized, factorial
that should let you run to the hard limits of your machine

one optimization i would make would be to implement a 'wrapper' function around math.factorial that caches previous values of factorial so that as your brownNum increases, factorial doesn't have as much work to do. this is known as 'memoization' in computer science.
edit: found another SO answer with similar intention: Python: Is math.factorial memoized?

You should also initialize the square root more closely to the root.
e = int(math.log(n,4))
x = n//2**e
Because of 4**e <= n <= 4**(e+1) the square root will be between x/2 and x which should yield quadratic convergence of the Heron formula from the first iteration on.

Generating digits of square root of 2

I want to generate the digits of the square root of two to 3 million digits.
I am aware of Newton-Raphson but I don't have much clue how to implement it in C or C++ due to lack of biginteger support. Can somebody point me in the right direction?
Also, if anybody knows how to do it in python (I'm a beginner), I would also appreciate it.

You could try using the mapping:
a/b -> (a+2b)/(a+b) starting with a= 1, b= 1. This converges to sqrt(2) (in fact gives the continued fraction representations of it).
Now the key point: This can be represented as a matrix multiplication (similar to fibonacci)
If a_n and b_n are the nth numbers in the steps then
[1 2] [a_n b_n]T = [a_(n+1) b_(n+1)]T
[1 1]
which now gives us
[1 2]n [a_1 b_1]T = [a_(n+1) b_(n+1)]T
[1 1]
Thus if the 2x2 matrix is A, we need to compute An which can be done by repeated squaring and only uses integer arithmetic (so you don't have to worry about precision issues).
Also note that the a/b you get will always be in reduced form (as gcd(a,b) = gcd(a+2b, a+b)), so if you are thinking of using a fraction class to represent the intermediate results, don't!
Since the nth denominators is like (1+sqrt(2))^n, to get 3 million digits you would likely need to compute till the 3671656th term.
Note, even though you are looking for the ~3.6 millionth term, repeated squaring will allow you to compute the nth term in O(Log n) multiplications and additions.
Also, this can easily be made parallel, unlike the iterative ones like Newton-Raphson etc.

EDIT: I like this version better than the previous. It's a general solution that accepts both integers and decimal fractions; with n = 2 and precision = 100000, it takes about two minutes. Thanks to Paul McGuire for his suggestions & other suggestions welcome!
def sqrt_list(n, precision):
ndigits = [] # break n into list of digits
n_int = int(n)
n_fraction = n - n_int
while n_int: # generate list of digits of integral part
ndigits.append(n_int % 10)
n_int /= 10
if len(ndigits) % 2: ndigits.append(0) # ndigits will be processed in groups of 2
decimal_point_index = len(ndigits) / 2 # remember decimal point position
while n_fraction: # insert digits from fractional part
n_fraction *= 10
ndigits.insert(0, int(n_fraction))
n_fraction -= int(n_fraction)
if len(ndigits) % 2: ndigits.insert(0, 0) # ndigits will be processed in groups of 2
rootlist = []
root = carry = 0 # the algorithm
while root == 0 or (len(rootlist) < precision and (ndigits or carry != 0)):
carry = carry * 100
if ndigits: carry += ndigits.pop() * 10 + ndigits.pop()
x = 9
while (20 * root + x) * x > carry:
x -= 1
carry -= (20 * root + x) * x
root = root * 10 + x
rootlist.append(x)
return rootlist, decimal_point_index

As for arbitrary big numbers you could have a look at The GNU Multiple Precision Arithmetic Library (for C/C++).

For work? Use a library!
For fun? Good for you :)
Write a program to imitate what you would do with pencil and paper. Start with 1 digit, then 2 digits, then 3, ..., ...
Don't worry about Newton or anybody else. Just do it your way.

Here is a short version for calculating the square root of an integer a to digits of precision. It works by finding the integer square root of a after multiplying by 10 raised to the 2 x digits.
def sqroot(a, digits):
a = a * (10**(2*digits))
x_prev = 0
x_next = 1 * (10**digits)
while x_prev != x_next:
x_prev = x_next
x_next = (x_prev + (a // x_prev)) >> 1
return x_next
Just a few caveats.
You'll need to convert the result to a string and add the decimal point at the correct location (if you want the decimal point printed).
Converting a very large integer to a string isn't very fast.
Dividing very large integers isn't very fast (in Python) either.
Depending on the performance of your system, it may take an hour or longer to calculate the square root of 2 to 3 million decimal places.
I haven't proven the loop will always terminate. It may oscillate between two values differing in the last digit. Or it may not.

The nicest way is probably using the continued fraction expansion [1; 2, 2, ...] the square root of two.
def root_two_cf_expansion():
yield 1
while True:
yield 2
def z(a,b,c,d, contfrac):
for x in contfrac:
while a > 0 and b > 0 and c > 0 and d > 0:
t = a // c
t2 = b // d
if not t == t2:
break
yield t
a = (10 * (a - c*t))
b = (10 * (b - d*t))
# continue with same fraction, don't pull new x
a, b = x*a+b, a
c, d = x*c+d, c
for digit in rdigits(a, c):
yield digit
def rdigits(p, q):
while p > 0:
if p > q:
d = p // q
p = p - q * d
else:
d = (10 * p) // q
p = 10 * p - q * d
yield d
def decimal(contfrac):
return z(1,0,0,1,contfrac)
decimal((root_two_cf_expansion()) returns an iterator of all the decimal digits. t1 and t2 in the algorithm are minimum and maximum values of the next digit. When they are equal, we output that digit.
Note that this does not handle certain exceptional cases such as negative numbers in the continued fraction.
(This code is an adaptation of Haskell code for handling continued fractions that has been floating around.)

Well, the following is the code that I wrote. It generated a million digits after the decimal for the square root of 2 in about 60800 seconds for me, but my laptop was sleeping when it was running the program, it should be faster that. You can try to generate 3 million digits, but it might take a couple days to get it.
def sqrt(number,digits_after_decimal=20):
import time
start=time.time()
original_number=number
number=str(number)
list=[]
for a in range(len(number)):
if number[a]=='.':
decimal_point_locaiton=a
break
if a==len(number)-1:
number+='.'
decimal_point_locaiton=a+1
if decimal_point_locaiton/2!=round(decimal_point_locaiton/2):
number='0'+number
decimal_point_locaiton+=1
if len(number)/2!=round(len(number)/2):
number+='0'
number=number[:decimal_point_locaiton]+number[decimal_point_locaiton+1:]
decimal_point_ans=int((decimal_point_locaiton-2)/2)+1
for a in range(0,len(number),2):
if number[a]!='0':
list.append(eval(number[a:a+2]))
else:
try:
list.append(eval(number[a+1]))
except IndexError:
pass
p=0
c=list[0]
x=0
ans=''
for a in range(len(list)):
while c>=(20*p+x)*(x):
x+=1
y=(20*p+x-1)*(x-1)
p=p*10+x-1
ans+=str(x-1)
c-=y
try:
c=c*100+list[a+1]
except IndexError:
c=c*100
while c!=0:
x=0
while c>=(20*p+x)*(x):
x+=1
y=(20*p+x-1)*(x-1)
p=p*10+x-1
ans+=str(x-1)
c-=y
c=c*100
if len(ans)-decimal_point_ans>=digits_after_decimal:
break
ans=ans[:decimal_point_ans]+'.'+ans[decimal_point_ans:]
total=time.time()-start
return ans,total

Python already supports big integers out of the box, and if that's the only thing holding you back in C/C++ you can always write a quick container class yourself.
The only problem you've mentioned is a lack of big integers. If you don't want to use a library for that, then are you looking for help writing such a class?

Here's a more efficient integer square root function (in Python 3.x) that should terminate in all cases. It starts with a number much closer to the square root, so it takes fewer steps. Note that int.bit_length requires Python 3.1+. Error checking left out for brevity.
def isqrt(n):
x = (n >> n.bit_length() // 2) + 1
result = (x + n // x) // 2
while abs(result - x) > 1:
x = result
result = (x + n // x) // 2
while result * result > n:
result -= 1
return result