We Keep Coding

Generalized Random Response for local differential privacy implementation

I have been tasked with implementing a local (non-interactive) differential privacy mechanism. I am working with a large database of census data. The only sensitive attribute is "Number of children" which is a numerical value ranging from 0 to 13.
I decided to go with the Generalized Random Response mechanism as it seems like the most intuitive method. This mechanism is described here and presented here.
After loading each value into an array (ignoring the other attributes for now), I perform the perturbation as follows.
d = 14 # values may range from 0 to 13
eps = 1 # epsilon level of privacy
p = (math.exp(eps)/(math.exp(eps)+d-1))
q = 1/(math.exp(eps)+d-1)
p_dataset = []
for row in dataset:
coin = random.random()
if coin <= p:
p_dataset.append(row)
else:
p_dataset.append(random.randint(0,13))
Unless I have misinterpreted the definition, I believe this will guarantee epsilon differential privacy on p_dataset.
However, I am having difficulty understanding how the aggregator must interpret this dataset. Following the presentation above, I attempted to implement a method for estimating the number of individuals who answered a particular value.
v = 0 # we are estimating the number of individuals in the dataset who answered 0
nv = 0 # number of users in the perturbed dataset who answered the value
for row in p_dataset:
if row == v:
nv += 1
Iv = nv * p + (n - nv) * q
estimation = (Iv - (n*q)) / (p-q)
I do not know if I have correctly implemented the method described as I do not completely understand what it is doing, and cannot find a clear definition.
Regardless, I used this method to estimate the total amount of individuals who answered each value in the dataset with a value for epsilon ranging from 1 to 14, and then compared this to the actual values. The results are below (please excuse the formatting).
As you can see, the utility of the dataset suffers greatly for low values of epsilon. Additionally, when executed multiple times, there was relatively little deviation in estimations, even for small values of epsilon.
For example, when estimating the number of participants who answered 0 and using an epsilon of 1, all estimations seemed to be centered around 1600, with the largest distance between estimations being 100. Considering the actual value of this query is 5969, I am led to believe that I may have implemented something incorrectly.
Is this the expected behaviour of the Generalized Random Response mechanism, or have I made a mistake in my implementation?

I think when getting a false answer， we cannot directly use p_dataset.append(random.randint(0,13)), because it contains true answer

max_v = 13
min_v = 0
for row in dataset: #row就是dataset里的真实值
coin = random.random()
if coin <= p:
p_dataset.append(row)
else:
ans = []
if row == min_v:
ans = np.arange(min_v + 1, max_v + 1).tolist()
elif row == max_v:
ans = np.arange(min_v, max_v).tolist()
else:
a = np.arange(min_v, row).tolist()
b = np.arange(row + 1, max_v + 1).tolist()
[ans.append(i) for i in a]
[ans.append(i) for i in b]
p_dataset.append(random.sample(ans, 1)) # 这其实有一点问题 应该是真实值以外的其他值 这样写还包括真实值

(python) solving transcendental equation

i need to solve following equation:
0 = -1 / x**0.5) - 2 * log((alpha * x**0.5) + beta)
alpha and beta are given, i just need to iterate x until a certain extent.
I'm not a great python programmer, but like to implement this one.
How might this be possible?
Best regards

The smartest to do would be to implement a solve function like Stanislav recommended. You can't just iterate over values of x until the equation reaches 0 due to Floating Point Arithmetic. You would have to .floor or .ceil your value to avoid an infinity loop. An example of this would be something like:
x = 0
while True:
x += 0.1
print(x)
if x == 10:
break
Here you'd think that x eventually reaches 10 when it adds 0.1 to 9.9, but this will continue forever. Now, I don't know if your values are integers or floats, but what I'm getting at is: Don't iterate. Use already built solve libraries.

Compute cosine without import math in python

I have a general idea of what to do, but my code is a mess and I'm having some trouble writing the algorithm in python for
cos(x)=1-(x^2)/2!+(x^4)/4!-(x^6)/6!+...
where x is in radians, computing cos(x) after 20 terms using while loops. So far what I've written is
x = float(input("Enter a value for x in degrees."))
x = (x*3.14159)/180
num_of_terms = 0
num = 1.0 ##numerator
y = 1.0
cosx = 1.0
while num_of_terms<1:
num_of_terms+=1
cosx = (num/y)
while num_of_terms>=1 and num_of_terms<=20:
num_of_terms+=1
num = num*(x*x)
y = y*num_of_terms*(num_of_terms-1)
if num_of_terms%2==0:
cosx = cosx+(-num/y)
else:
cosx = cosx+(num/y)
print(cosx)
I don't know how close I even am to being correct (I know it's wrong in at least some places so I can't properly check using math.cos) but the main question I have is how to switch from positive --> negative each term. The assignment states that I cannot use exponentiation operators, and before I was trying to do something like
x = float(input("Enter a value for x in degrees."))
x = (x*3.14)/180
num_of_terms = 0
y = 0
z = 1
cosx = ((-1)**(z-1))*((x**z)/(y))
so that the sign would switch for every other term. Now I have (as you can see above)
if num_of_terms%2==0:
cosx = cosx+(-num/y)
else:
cosx = cosx+(num/y)
which is incorrect, or at least the output I'm getting is incorrect.

You can handle the sign quite simply:
sign = -1
while num_of_terms <= 20:
sign = -sign
...
cosx += sign * num/y
You also have a structure problem in your loops: the first loop will terminate after one iteration ... except you've properly prevented it from getting back there. This is poor use of a loop.
Just initialize your variables before the loop, and then proceed as expected. Since you know how many times to iterate, use a for instead of a while.
cosx = (num/y)
for num_of_terms in range(1, 21):
...
You will find other computational problems in your code. Print out values each time through the loop to help track your execution and computations. At the start, just go through 3 or 4 times instead of 20.
For the factorial, keep a running product: it's like a running sum, except that you initialize it at 1, and multiply each time through the loop.
Okay; stick with the while. Now, manage your loop index and computational index. If you're doing term #1, what should the exponent be? What numbers do you multiply into y? Now identify the same values for term #2 and term #3.
More directly, stick in a print statement to track num_of_terms, y, and cosx. When you're doing term #3, what is y? It should be 4! or 6! (depending on how you number your terms), but it's not. Where did you go wrong?

Your problem is in the computation of the factorial. You're multiplying by num_of_terms, which only increments by one each time through the loop - you need something that changes by 2 each time through the loop. At least you're correctly multiplying both that number and the number-1.

Wrong answer in SPOJ `CUBERT` [closed]

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 7 years ago.
Improve this question
I am getting a Wrong Answer for my solution to this problem on SPOJ.
The problem asks to calculate the cube root of an integer(which can be upto 150 digits long), and output the answer truncated upto 10 decimal places.
It also asks to calculate the sum of all the digits in the answer modulo 10 as a 'checksum' value.
Here is the exact problem statement:
Your task is to calculate the cube root of a given positive integer.
We can not remember why exactly we need this, but it has something in
common with a princess, a young peasant, kissing and half of a kingdom
(a huge one, we can assure you).
Write a program to solve this crucial task.
Input
The input starts with a line containing a single integer t <= 20, the
number of test cases. t test cases follow.
The next lines consist of large positive integers of up to 150 decimal
digits. Each number is on its own separate line of the input file. The
input file may contain empty lines. Numbers can be preceded or
followed by whitespaces but no line exceeds 255 characters.
Output
For each number in the input file your program should output a line
consisting of two values separated by single space. The second value
is the cube root of the given number, truncated (not rounded!) after
the 10th decimal place. First value is a checksum of all printed
digits of the cube root, calculated as the sum of the printed digits
modulo 10.
Example
Input:
5
1
8
1000
2 33076161
Output:
1 1.0000000000
2 2.0000000000
1 10.0000000000
0 1.2599210498
6 321.0000000000
Here is my solution:
from math import pow
def foo(num):
num_cube_root = pow(num, 1.0 / 3)
# First round upto 11 decimal places
num_cube_root = "%.11f" % (num_cube_root)
# Then remove the last decimal digit
# to achieve a truncation of 10 decimal places
num_cube_root = str(num_cube_root)[0:-1]
num_cube_root_sum = 0
for digit in num_cube_root:
if digit != '.':
num_cube_root_sum += int(digit)
num_cube_root_sum %= 10
return (num_cube_root_sum, num_cube_root)
def main():
# Number of test cases
t = int(input())
while t:
t -= 1
num = input().strip()
# If line empty, ignore
if not num:
t += 1
continue
num = int(num)
ans = foo(num)
print(str(ans[0]) + " " + ans[1])
if __name__ == '__main__':
main()
It is working perfectly for the sample cases: Live demo.
Can anyone tell what is the problem with this solution?

Your solution has two problems, both related to the use of floating-point arithmetic. The first issue is that Python floats only carry roughly 16 significant decimal digits of precision, so as soon as your answer requires more than 16 significant digits or so (so more than 6 digits before the point, and 10 digits after), you've very little hope of getting the correct trailing digits. The second issue is more subtle, and affects even small values of n. That's that your approach of rounding to 11 decimal digits and then dropping the last digit suffers from potential errors due to double rounding. For an example, take n = 33. The cube root of n, to 20 decimal places or so, is:
3.20753432999582648755...
When that's rounded to 11 places after the point, you end up with
3.20753433000
and now dropping the last digit gives 3.2075343300, which isn't what you wanted. The problem is that that round to 11 decimal places can end up affecting digits to the left of the 11th place digit.
So what can you do to fix this? Well, you can avoid floating-point altogether and reduce this to a pure integer problem. We need the cube root of some integer n to 10 decimal places (rounding the last place down). That's equivalent to computing the cube root of 10**30 * n to the nearest integer, again rounding down, then dividing the result by 10**10. So the essential task here is to compute the floor of the cube root of any given integer n. I was unable to find any existing Stack Overflow answers about computing integer cube roots (still less in Python), so I thought it worth showing how to do so in detail.
Computing cube roots of integers turns out to be quite easy (with the help of a tiny bit of mathematics). There are various possible approaches, but one approach that's both efficient and easy to implement is to use a pure-integer version of the Newton-Raphson method. Over the real numbers, Newton's method for solving the equation x**3 = n takes an approximation x to the cube root of n, and iterates to return an improved approximation. The required iteration is:
x_next = (2*x + n/x**2)/3
In the real case, you'd repeat the iteration until you reached some desired tolerance. It turns out that over the integers, essentially the same iteration works, and with the right exit condition it will give us exactly the correct answer (no tolerance required). The iteration in the integer case is:
a_next = (2*a + n//a**2)//3
(Note the uses of the floor division operator // in place of the usual true division operator / above.) Mathematically, a_next is exactly the floor of (2*a + n/a**2)/3.
Here's some code based on this iteration:
def icbrt_v1(n, initial_guess=None):
"""
Given a positive integer n, find the floor of the cube root of n.
Args:
n : positive integer
initial_guess : positive integer, optional. If given, this is an
initial guess for the floor of the cube root. It must be greater
than or equal to floor(cube_root(n)).
Returns:
The floor of the cube root of n, as an integer.
"""
a = initial_guess if initial_guess is not None else n
while True:
d = n//a**2
if a <= d:
return a
a = (2*a + d)//3
And some example uses:
>>> icbrt_v1(100)
4
>>> icbrt_v1(1000000000)
1000
>>> large_int = 31415926535897932384626433
>>> icbrt_v1(large_int**3)
31415926535897932384626433
>>> icbrt_v1(large_int**3-1)
31415926535897932384626432
There are a couple of annoyances and inefficiencies in icbrt_v1 that we'll fix shortly. But first, a brief explanation of why the above code works. Note that we start with an initial guess that's assumed to be greater than or equal to the floor of the cube root. We'll show that this property is a loop invariant: every time we reach the top of the while loop, a is at least floor(cbrt(n)). Furthermore, each iteration produces a value of a strictly smaller than the old one, so our iteration is guaranteed to eventually converge to floor(cbrt(n)). To prove these facts, note that as we enter the while loop, there are two possibilities:
Case 1. a is strictly greater than the cube root of n. Then a > n//a**2, and the code proceeds to the next iteration. Write a_next = (2*a + n//a**2)//3, then we have:
a_next >= floor(cbrt(n)). This follows from the fact that (2*a + n/a**2)/3 is at least the cube root of n, which in turn follows from the AM-GM inequality applied to a, a and n/a**2: the geometric mean of these three quantities is exactly the cube root of n, so the arithmetic mean must be at least the cube root of n. So our loop invariant is preserved for the next iteration.
a_next < a: since we're assuming that a is larger than the cube root, n/a**2 < a, and it follows that (2a + n/a**2) / 3 is smaller than a, and hence that floor((2a + n/a**2) / 3) < a. This guarantees that we make progress towards the solution at each iteration.
Case 2. a is less than or equal to the cube root of n. Then a <= floor(cbrt(n)), but from the loop invariant established above we also know that a >= floor(cbrt(n)). So we're done: a is the value we're after. And the while loop exits at this point, since a <= n // a**2.
There are a couple of issues with the code above. First, starting with an initial guess of n is inefficient: the code will spend its first few iterations (roughly) dividing the current value of a by 3 each time until it gets into the neighborhood of the solution. A better choice for the initial guess (and one that's easily computable in Python) is to use the first power of two that exceeds the cube root of n.
initial_guess = 1 << -(-n.bit_length() // 3)
Even better, if n is small enough to avoid overflow, is to use floating-point arithmetic to provide the initial guess, with something like:
initial_guess = int(round(n ** (1/3.)))
But this brings us to our second issue: the correctness of our algorithm requires that the initial guess is no smaller than the actual integer cube root, and as n gets large we can't guarantee that for the float-based initial_guess above (though for small enough n, we can). Luckily, there's a very simple fix: for any positive integer a, if we perform a single iteration we always end up with a value that's at least floor(cbrt(a)) (using the same AM-GM argument that we used above). So all we have to do is perform at least one iteration before we start testing for convergence.
With that in mind, here's a more efficient version of the above code:
def icbrt(n):
"""
Given a positive integer n, find the floor of the cube root of n.
Args:
n : positive integer
Returns:
The floor of the cube root of n, as an integer.
"""
if n.bit_length() < 1024: # float(n) safe from overflow
a = int(round(n**(1/3.)))
a = (2*a + n//a**2)//3 # Ensure a >= floor(cbrt(n)).
else:
a = 1 << -(-n.bit_length()//3)
while True:
d = n//a**2
if a <= d:
return a
a = (2*a + d)//3
And with icbrt in hand, it's easy to put everything together to compute cube roots to ten decimal places. Here, for simplicity, I output the result as a string, but you could just as easily construct a Decimal instance.
def cbrt_to_ten_places(n):
"""
Compute the cube root of `n`, truncated to ten decimal places.
Returns the answer as a string.
"""
a = icbrt(n * 10**30)
q, r = divmod(a, 10**10)
return "{}.{:010d}".format(q, r)
Example outputs:
>>> cbrt_to_ten_places(2)
'1.2599210498'
>>> cbrt_to_ten_places(8)
'2.0000000000'
>>> cbrt_to_ten_places(31415926535897932384626433)
'315536756.9301821867'
>>> cbrt_to_ten_places(31415926535897932384626433**3)
'31415926535897932384626433.0000000000'

You may try to use the decimal module with a sufficiently large precision value.
EDIT: Thanks to #DSM, I realised that decimal module will not produce very exact cube roots. I suggest that you check whether all digits are 9s and round it to a integer if that is a case.
Also, I now perform the 1/3 division with Decimals as well, because passing the result of 1/3 to Decimal constructor leads to reduced precision.
import decimal
def cbrt(n):
nd = decimal.Decimal(n)
with decimal.localcontext() as ctx:
ctx.prec = 50
i = nd ** (decimal.Decimal(1) / decimal.Decimal(3))
return i
ret = str(cbrt(1233412412430519230351035712112421123121111))
print(ret)
left, right = ret.split('.')
print(left + '.' + ''.join(right[:10]))
Output:
107243119477324.80328931501744819161741924145124146
107243119477324.8032893150
Output of cbrt(10) is:
9.9999999999999999999999999999999999999999999999998

Finding if n! + 1 is a perfect square

I'm trying to write a program to look for a number, n, between 0 and 100 such that n! + 1 is a perfect square. I'm trying to do this because I know there are only three so it was meant as a test of my Python ability.
Refer to Brocard's problem.

math.sqrt always returns a float, even if that float happens to be, say, 4.0. As the docs say, "Except when explicitly noted otherwise, all return values are floats."
So, your test for type(math.sqrt(x)) == int will never be true.
You could try to work around that by checking whether the float represents an integer, like this:
sx = math.sqrt(x)
if round(sx) == sx:
There's even a built-in method that does this as well as possible:
if sx.is_integer():
But keep in mind that float values are not a perfect representation of real numbers, and there are always rounding issues. For example, for a too-large number, the sqrt might round to an integer, even though it really wasn't a perfect square. For example, if math.sqrt(10000000000**2 + 1).is_integer() is True, even though obviously the number is not a perfect square.
I could tell you whether this is safe within your range of values, but can you convince yourself? If not, you shouldn't just assume that it is.
So, is there a way we can check that isn't affected by float roading issues? Sure, we can use integer arithmetic to check:
sx = int(round(math.sqrt(x)))
if sx*sx == x:
But, as Stefan Pochmann points out, even if this check is safe, does that mean the whole algorithm is? No; sqrt itself could have already been rounded to the point where you've lost integer precision.
So, you need an exact sqrt. You could do this by using decimal.Decimal with a huge configured precision. This will take a bit of work, and a lot of memory, but it's doable. Like this:
decimal.getcontext().prec = ENOUGH_DIGITS
sx = decimal.Decimal(x).sqrt()
But how many digits is ENOUGH_DIGITS? Well, how many digits do you need to represent 100!+1 exactly?
So:
decimal.getcontext().prec = 156
while n <= 100:
x = math.factorial(n) + 1
sx = decimal.Decimal(x).sqrt()
if int(sx) ** 2 == x:
print(sx)
n = n + 1
If you think about it, there's a way to reduce the needed precision to 79 digits, but I'll leave that as an exercise for the reader.
The way you're presumably supposed to solve this is by using purely integer math. For example, you can find out whether an integer is a square in logarithmic time just by using Newton's method until your approximation error is small enough to just check the two bordering integers.

For very large numbers it's better to avoid using floating point square roots altogether because you will run into too many precision issues and you can't even guarantee that you will be within 1 integer value of the correct answer. Fortunately Python natively supports integers of arbitrary size, so you can write an integer square root checking function, like this:
def isSquare(x):
if x == 1:
return True
low = 0
high = x // 2
root = high
while root * root != x:
root = (low + high) // 2
if low + 1 >= high:
return False
if root * root > x:
high = root
else:
low = root
return True
Then you can run through the integers from 0 to 100 like this:
n = 0
while n <= 100:
x = math.factorial(n) + 1
if isSquare(x):
print n
n = n + 1

Here's another version working only with integers, computing the square root by adding decreasing powers of 2, for example intsqrt(24680) will be computed as 128+16+8+4+1.
def intsqrt(n):
pow2 = 1
while pow2 < n:
pow2 *= 2
sqrt = 0
while pow2:
if (sqrt + pow2) ** 2 <= n:
sqrt += pow2
pow2 //= 2
return sqrt
factorial = 1
for n in range(1, 101):
factorial *= n
if intsqrt(factorial + 1) ** 2 == factorial + 1:
print(n)

The number math.sqrt returns is never an int, even if it's an integer.How to check if a float value is a whole number