We Keep Coding

Fast calculation of sum for function defined over range of integers - (0,2^52)

I was looking at the code for a particular cryptocurrency casino game (EthCrash - if you're interested). The game generates crash points using a function (I call this crash(x)) where x is an integer that is randomly drawn from the space of integers (0,2^52).
I'd like to calculate the expected value of the crash points. The code below should explain everything, but a clean picture of the function is here: https://i.imgur.com/8dPBALa.png, and what I'm trying to calculate is here: https://i.imgur.com/nllykDQ.png (apologies - can't paste pictures yet).
I wrote the following code:
import math
two52 = 2**52
def crash(x):
crash_point = math.floor((100*two52-x)/(two52-x))
return(crash_point/100)
crashes_sum = 0
for i in range(two52+1):
crashes_sum += crash(i)
expected_crash = crashes_sum/two52
Unfortunately, the loop is taking too long to run - any ideas for how I can do this faster?

ok, if you cannot do it straightforward, time to get smart, right?
So idea to get ranges where whole sum could be computed fast. I will put some pseudocode which not even compiles, could have bugs etc. Use it as illustration.
First, lets rewrite the term in the sum as
floor( 100 + 99*x/(252 - x) )
First idea - get ranges where floor is not changing due to the fact that term
n =< 99*x/(252 - x) < n+1. Obviously, for this whole range we could add to sum range_length*(100 + n), no need to do it term by term
sum = 0
r_lo = 0
for k in range(0, 2*52): # LOOP OVER RANGES
r_hi = floor(2**52/(1 + 99/n))
sum += (100 + n -1)*(r_hi - r_lo)
if r_hi-r_lo == 1:
break
r_lo = r_hi + 1
Obviously, range size will shrink till it is equal to 1, and then this method will be useless, we break out. Obviously, by that time each term would be different from previous one by 1 or more.
Ok, second idea - again ranges, where sum is arithmetic series. First we have to find range where increment is equal to 1. Then range where increment is equal to 2, etc. Looks like you have to find roots of quadratic equation for this, but code would be about the same
r_lo = pos_for_increment(1)
t_lo = ... # term at r_lo
for n in range(2, 2*52): # LOOP OVER RANGES
r_hi = pos_for_increment(n) - 1
t_hi = ... # term at r_lo
sum += (t_lo + t_hi)*(r_hi - r_lo) / 2 # arith.series sum
if r_hi > 2**52:
break
r_lo = r_hi + 1
t_lo = t_hi + n
might think about something else, but those tricks are worth trying

Using the map function might help increase the speed since it makes the computation in parallel
import math
two52 = 2**52
def crash(x):
crash_point = math.floor((100*two52-x)/(two52-x))
return(crash_point/100)
crashes_sum = sum(map(crash,range(two52)))
expected_crash = crashes_sum/two52

I have been able to speed up your code by taking advantage of numpy vectorization:
import numpy as np
import time
two52 = 2**52
crash = lambda x: np.floor( ( 100 * two52 - x ) / ( two52 - x ) ) / 100
starttime = time.time()
icur = 0
ispan = 100000
crashes_sum = 0
while icur < two52-ispan:
i = np.arange(icur, icur+ispan, 1)
crashes_sum += np.sum(crash(i))
icur += ispan
crashes_sum += np.sum(crash(np.arange(icur, two52, 1)))
expected_crash = crashes_sum / two52
print(time.time() - starttime)
The trick is to compute the sum on a moving windows to take advantage of numpy's vectorization (written in C). I tried up to 2**30 and it takes 9 seconds on my laptop (and too long for your code to be able to benchmark).
Python is probably not the most suitable language for what you want to do, you may want to try C or Fortran for that (and take advantage of threading).

You will have to use a powerful GPU if you wan't the result within some hours.
A possible CPU implementation
import numpy as np
import numba as nb
import time
two52 = 2**52
loop_to=2**30
#nb.njit(fastmath=True,parallel=True)
def sum_over_crash(two52,loop_to): #loop_to is only for testing performance
crashes_sum = nb.float64(0)
for i in nb.prange(loop_to):#nb.prange(two52+1):
crashes_sum += np.floor((100*two52-i)/(two52-i))/100
return crashes_sum/two52
sum_over_crash(two52,2)#don't measure static compilation overhead
t1=time.time()
sum_over_crash(two52,2**30)
print(time.time()-t1)
This takes 0.57s for on my quadcore i7. eg. 28 days for the whole calculation.

As the calculation can not be minimized mathematically, the only option is to calculate it step by step.
This takes a long time (as stated in other answers). Your best bet on calculating it fast is to use a lower level language than python. Since python is an interpreted language, it is rather slow to calculate this kind of thing.
Additionally you can use multithreading (if availible in the chosen language) to make it even faster.
Cloud Computing is also an option that could be suitable for this, as you are only going to calculate the number once. Amazon and Google (and many more) provide this kind of service for a relatively small fee.
But before performing any of the calculations you need to adjust your formula, as with the way it stands right now, you're going to get a ZeroDivisionError at the very last iteration of your loop.

Need help finding Big Oh, c, and No

Need help with these two examples. As I kind of understand Big Oh, but not really the c and No concepts. First one seems pretty straight forward. I'm pretty sure the Big Oh would be O(n^3) but I'm not sure.
f(x) = 2n3 + 5n + 2
The next one is the one that really makes me feel like idk what I'm doing.
def analyze(alist):
1 exchanges = True
2 passnum = len(alist)-1
3 while passnum > 0 and exchanges:
4 exchanges = False
5 for i in range(passnum):
6 if alist[i]>alist[i+1]:
7 exchanges = True
8 alist[i],alist[i+1]=alist[i+1],alist[i]
9 passnum = passnum-1
HE wants me to label each line regarding Big Oh (what?), and then calculate the Big Oh, c, and No.
Any help/explanation would be a huge help, I'm feeling lost. THought I had it, but its clear that I do not. Thanks

Each line represents either an elementary operation or a function.
An elementary operation is often considered to take O(1) time (constant time).
if i had this simple code and i'm looking the big Oh relative to N, a number given in argument
a = 1 // O(1) assignement is constant
b = a + 2 // O(1) addition is constant
for (a=0 to N-1) // a will take value between [0..n-1], which is n different values
a = a + 1 // addition is O(1)
//resume: i have => O(1) + O(1) + N * O(1) => O(N)
Here you see i did label each line, and i can conclude , this code is O(N) cause it's the dominant factor. You need to try to do the same for your snippet of code.

Lua: Decompose a number by powers of 2

This question is a parallel to python - How do I decompose a number into powers of 2?. Indeed, it is the same question, but rather than using Python (or Javascript, or C++, as these also seem to exist), I'm wondering how it can be done using Lua. I have a very basic understanding of Python, so I took the code first listed in the site above and attempted to translate it to Lua, with no success. Here's the original, and following, my translation:
Python
def myfunc(x):
powers = []
i = 1
while i <= x:
if i & x:
powers.append(i)
i <<= 1
return powers
Lua
function powerfind(n)
local powers = {}
i = 1
while i <= n do
if bit.band(i, n) then -- bitwise and check
table.insert(powers, i)
end
i = bit.shl(i, 1) -- bitwise shift to the left
end
return powers
end
Unfortunately, my version locks and "runs out of memory". This was after using the number 12 as a test. It's more than likely that my primitive knowledge of Python is failing me, and I'm not able to translate the code from Python to Lua correctly, so hopefully someone can offer a fresh set of eyes and help me fix it.

Thanks to the comments from user2357112, I've got it fixed, so I'm posting the answer in case anyone else comes across this issue:
function powerfind(n)
local powers = {}
i = 1
while i <= n do
if bit.band(i, n) ~= 0 then -- bitwise and check
table.insert(powers, i)
end
i = bit.shl(i, 1) -- bitwise shift to the left
end
return powers
end

I saw that in the other one, it became a sort of speed contest. This one should also be easy to understand.
i is the current power. It isn't used for calculations.
n is the current place in the array.
r is the remainder after a division of x by two.
If the remainder is 1 then you know that i is a power of two which is used in the binary representation of x.
local function powerfind(x)
local powers={
nil,nil,nil,nil,
nil,nil,nil,nil,
nil,nil,nil,nil,
nil,nil,nil,nil,
}
local i,n=1,0
while x~=0 do
local r=x%2
if r==1 then
x,n=x-1,n+1
powers[n]=i
end
x,i=x/2,2*i
end
end
Running a million iterations, x from 1 to 1000000, takes me 0.29 seconds. I initialize the size of the powers table to 16.

Writing a double sum in Python

I am new to StackOverflow, and I am extremely new to Python.
My problem is this... I am needing to write a double-sum, as follows:
The motivation is that this is the angular correction to the gravitational potential used for the geoid.
I am having difficulty writing the sums. And please, before you say "Go to such-and-such a resource," or get impatient with me, this is the first time I have ever done coding/programming/whatever this is.
Is this a good place to use a "for" loop?
I have data for the two indices (n,m) and for the coefficients c_{nm} and s_{nm} in a .txt file. Each of those items is a column. When I say usecols, do I number them 0 through 3, or 1 through 4?
(the equation above)
\begin{equation}
V(r, \phi, \lambda) = \sum_{n=2}^{360}\left(\frac{a}{r}\right)^{n}\sum_{m=0}^{n}\left[c_{nm}*\cos{(m\lambda)} + s_{nm}*\sin{(m\lambda)}\right]*\sqrt{\frac{(n-m)!}{(n+m)!}(2n + 1)(2 - \delta_{m0})}P_{nm}(\sin{\lambda})
\end{equation}

(2) Yes, a "for" loop is fine. As #jpmc26 notes, a generator expression is a good alternative to a "for" loop. IMO, you'll want to use numpy if efficiency is important to you.
(3) As #askewchan notes, "usecols" refers to an argument of genfromtxt; as specified in that documentation, column indexes start at 0, so you'll want to use 0 to 3.
A naive implementation might be okay since the larger factorial is the denominator, but I wouldn't be surprised if you run into numerical issues. Here's something to get you started. Note that you'll need to define P() and a. I don't understand how "0 through 3" relates to c and s since their indexes range much further. I'm going to assume that each (and delta) has its own file of values.
import math
import numpy
c = numpy.getfromtxt("the_c_file.txt")
s = numpy.getfromtxt("the_s_file.txt")
delta = numpy.getfromtxt("the_delta_file.txt")
def V(r, phi, lam):
ret = 0
for n in xrange(2, 361):
for m in xrange(0, n + 1):
inner = c[n,m]*math.cos(m*lam) + s[n,m]*math.sin(m*lam)
inner *= math.sqrt(math.factorial(n-m)/math.factorial(n+m)*(2*n+1)*(2-delta[m,0]))
inner *= P(n, m, math.sin(lam))
ret += math.pow(a/r, n) * inner
return ret
Make sure to write unittests to check the math. Note that "lambda" is a reserved word.

Python: terminating while statement in liebniz pie approximation

I am a beginner in python and programming and have already hit a roadblock while doing excercises on HTLCS.
The problem is to use Liebniz approximation to calculate the value of pi (3.14...).
Here is my pitty attempt to solve the question:
def myPi():
n = 0
value = ((-1) ** n)/(2 * n + 1)
runningtotal = 0
while True:
runningtotal += value
n += 1
value = ((-1) ** n)/(2 * n + 1)
runningtotal *= 4
return runningtotal
Of course, the Python interpreter shell never finishes my function because of while True, and I understand solutions like while n != 5000 also work, but I want Python to find the terminating point itself and return the result.
I attempted to run while statement until the value of runningtotal and that of the updated runningtotal are same at certain floating number, but failed because for some reason the last floating points of two values missed by one at each loop. (runningtotal: 3.14157, updated runningtotal: 3.14158 -> runningtotal: 3.14158, updated runningtotal: 3.14157 -> repeat).
This is my first question on this forum, so let me know if I didn't make myself clear or violated the rules of Stackflow that I was unaware of.

Floating point numbers have inherent imprecision, and testing them for equality is a risky business. I would recommend using a small tolerance instead of testing for equality. Instead of oldTotal == newTotal, test for something like abs(oldTotal-newTotal)<0.0001 (or whatever tolerance you like).

I would use an arbitrary-precision library to do this sort of thing, but regular Python works too.
Here's how I'd do it:
precision = 10
epsilon = 10 ** (-precision)
while True:
...
if abs(oldvalue - newvalue) < epsilon:
break