Which one is more efficient using math.pow or the ** operator? When should I use one over the other?
So far I know that x**y can return an int or a float if you use a decimal
the function pow will return a float
import math
print( math.pow(10, 2) )
print( 10. ** 2 )
Using the power operator ** will be faster as it won’t have the overhead of a function call. You can see this if you disassemble the Python code:
>>> dis.dis('7. ** i')
1 0 LOAD_CONST 0 (7.0)
3 LOAD_NAME 0 (i)
6 BINARY_POWER
7 RETURN_VALUE
>>> dis.dis('pow(7., i)')
1 0 LOAD_NAME 0 (pow)
3 LOAD_CONST 0 (7.0)
6 LOAD_NAME 1 (i)
9 CALL_FUNCTION 2 (2 positional, 0 keyword pair)
12 RETURN_VALUE
>>> dis.dis('math.pow(7, i)')
1 0 LOAD_NAME 0 (math)
3 LOAD_ATTR 1 (pow)
6 LOAD_CONST 0 (7)
9 LOAD_NAME 2 (i)
12 CALL_FUNCTION 2 (2 positional, 0 keyword pair)
15 RETURN_VALUE
Note that I’m using a variable i as the exponent here because constant expressions like 7. ** 5 are actually evaluated at compile time.
Now, in practice, this difference does not matter that much, as you can see when timing it:
>>> from timeit import timeit
>>> timeit('7. ** i', setup='i = 5')
0.2894785532627111
>>> timeit('pow(7., i)', setup='i = 5')
0.41218495570683444
>>> timeit('math.pow(7, i)', setup='import math; i = 5')
0.5655053168791255
So, while pow and math.pow are about twice as slow, they are still fast enough to not care much. Unless you can actually identify the exponentiation as a bottleneck, there won’t be a reason to choose one method over the other if clarity decreases. This especially applies since pow offers an integrated modulo operation for example.
Alfe asked a good question in the comments above:
timeit shows that math.pow is slower than ** in all cases. What is math.pow() good for anyway? Has anybody an idea where it can be of any advantage then?
The big difference of math.pow to both the builtin pow and the power operator ** is that it always uses float semantics. So if you, for some reason, want to make sure you get a float as a result back, then math.pow will ensure this property.
Let’s think of an example: We have two numbers, i and j, and have no idea if they are floats or integers. But we want to have a float result of i^j. So what options do we have?
We can convert at least one of the arguments to a float and then do i ** j.
We can do i ** j and convert the result to a float (float exponentation is automatically used when either i or j are floats, so the result is the same).
We can use math.pow.
So, let’s test this:
>>> timeit('float(i) ** j', setup='i, j = 7, 5')
0.7610865891750791
>>> timeit('i ** float(j)', setup='i, j = 7, 5')
0.7930400942188385
>>> timeit('float(i ** j)', setup='i, j = 7, 5')
0.8946636625872202
>>> timeit('math.pow(i, j)', setup='import math; i, j = 7, 5')
0.5699394063529439
As you can see, math.pow is actually faster! And if you think about it, the overhead from the function call is also gone now, because in all the other alternatives we have to call float().
In addition, it might be worth to note that the behavior of ** and pow can be overridden by implementing the special __pow__ (and __rpow__) method for custom types. So if you don’t want that (for whatever reason), using math.pow won’t do that.
The pow() function will allow you to add a third argument as a modulus.
For example: I was recently faced with a memory error when doing
2**23375247598357347582 % 23375247598357347583
Instead I did:
pow(2, 23375247598357347582, 23375247598357347583)
This returns in mere milliseconds instead of the massive amount of time and memory that the plain exponent takes. So, when dealing with large numbers and parallel modulus, pow() is more efficient, however when dealing with smaller numbers without modulus, ** is more efficient.
Just for the protocol: The ** operator is equivalent to the two-argument version of the built-in pow function, the pow function accepts an optional third argument (modulus) if the first two arguments are integers.
So, if you intend to calculate remainders from powers, use the built-in function. The math.pow will give you false results for arguments of reasonable size:
import math
base = 13
exp = 100
mod = 2
print math.pow(base, exp) % mod
print pow(base, exp, mod)
When I ran this, I got 0.0 in the first case which obviously cannot be true, because 13 is odd (and therefore all of it's integral powers). The math.pow version uses the limited accuracy of the IEEE-754 Double precision (52 bits mantissa, slightly less than 16 decimal places) which causes an error here.
For sake of fairness, we must say, math.pow can also be faster:
>>> import timeit
>>> min(timeit.repeat("pow(1.1, 9.9)", number=2000000, repeat=5))
0.3063715160001266
>>> min(timeit.repeat("math.pow(1.1, 9.9)", setup="import math", number=2000000, repeat=5))
0.2647279420000359
The math.pow function had (and still has) its strength in engineering applications, but for number theoretical applications, you should use the built-in pow function.
Some online examples
http://ideone.com/qaDWRd (wrong remainder with math.pow)
http://ideone.com/g7J9Un (lower performance with pow on int values)
http://ideone.com/KnEtXj (slightly lower performance with pow on float values)
Update (inevitable correction):
I removed the timing comparison of math.pow(2,100) and pow(2,100) since math.pow gives a wrong result whereas, for example, the comparison between pow(2,50) and math.pow(2,50) would have been fair (although not a realistic use of the math-module function). I added a better one and also the details that cause the limitation of math.pow.
** is indeed faster then math.pow(), but if you want a simple quadratic function like in your example it is even faster to use a product.
10.*10.
will be faster then
10.**2
The difference is not big and not noticable with one operation (using timeit), but with a large number of operations it can be significant.
Well, they are for different tasks, really.
Use pow (equivalent to x ** y with two arguments) when you want integer arithmetic.
And use math.pow if either argument is float, and you want float output.
For a discussion on the differences between pow and math.pow, see this question.
operator ** (same as pow()) can be used to calculate very large integer number.
>>> 2 ** 12345
164171010688258216356020741663906501410127235530735881272116103087925094171390144280159034536439457734870419127140401667195510331085657185332721089236401193044493457116299768844344303479235489462...
>>> math.pow(2, 12345)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: math range error
For small powers, like 2, I prefer to just multiply the base:
Use x*x instead of x**2 or pow(x, 2).
I haven't timed it, but I'd bet the multiplier is as fast as either the exponential operator or the pow function.
Related
I am on Windows 10 (64-Bit machine with 32-Bit Python 3.7).
In IDLE, if I type:
>>> -0.001**-0.001
-1.0069316688518042
But if I do:
>>> x = -0.001
>>> x**x
(1.006926699847276 -0.0031633639300006526j)
Interestingly, the magnitude of this complex number is the same as the actual answer.
As a proof, I've attached screenshot of the same.
What could be causing this?
In the first case, you are not getting a complex number because ** has higher precedence than - (both in Python and in math), so you are actually doing -(0.001 ** -0.001). Try (-0.001) ** -0.001.
The complex number is the "correct" answer by the mathematical definition of the power operation.
in Python, operator ** means to the power of, a negative number taking a negative power shall create an imaginary number, as just like sqrt(-1)=i.
if you meant for multiplication, you should use x*x instead of x**x
I trying to use RSA to encrypt my data in Python.
I created two small (private and public) keys
e : 14918179 N : 15372757
D : 7495819 N : 15372757
I tried to encrypt a small value (10) with those keys, and it worked. But the problem is that it takes a long time to do.
For example, I compared it to openssl by using a big key and long string and it worked under a second.
And I know there is a third library for using RSA (not a big fan of them).
I am trying to use this method to encrypt my data that is going to be sent to the server and it should do it under a second
How can I do it?
I think in general your questions on performance are answered here. wikipedia : modular exponentiation
The article describes
Direct exponentiation
Memory efficient exponentiation
Binary exponentiation
Direct Exponentiation
raise to the power e and take the modulo.
This is straight forward, but the size of the number pre modulo is extermely large.
Memory efficient exponentiation
Replacing the power operation with a multiply e times, allows the accumulated result to always be within the modulo range. This limits the size of the bignum and speeds up the operation.
Binary exponentiation
If you convert the power to a binary number
if e = 13 => 1101
pow(n, 13) = pow( n, 8) * pow(n,4) * pow(n, 1)
So for an m bit exponent, then only about m operations need to be done.
Combining the memory efficient and binary exponentiation solves most of the performance.
Python offers an implementation of these improvements using the 3 argument power function e.g.
>>> import timeit
>>> t = timeit.Timer( 'print(pow( 10,14918179, 15372757))' )
>>> t.timeit(1)
10140931
0.06365180000000237
>>> u = timeit.Timer( 'print(pow( 10,14918179) % 15372757)' )
>>> u.timeit(1)
10140931
15.021656000000007
The 3 parameter of pow takes .06s whilst the 2 parameter version of pow takes 15 seconds.
They seem to be equal for small numbers but different for larger.
For example:
a = int(1267650600228229401496703205376/10)
b = 1267650600228229401496703205376 // 10
print(a - b) # prints 7036874417767
a = int(1493845793475/10)
b = 1493845793475 // 10
print(a - b) # prints 0
How come?
Complementing the existing answers it seems worthwile mentioning that you needn't go that far out to observe a difference:
>>> -1//2
-1
>>> int(-1/2)
0
In Python 3 / performs float division, which has 53 bits of precision; // does floor division, which has no precision limit when both operands are integers (apart from limitations imposed by available RAM).
You can get the Python 3 behaviour in Python 2 by using the true_division __future__ import.
As #khelwood explained, in Python 3, a/b performs float division. Try typing 1/2 into an interpreter -- you'll get 0.5, not 0.
So in your example, 1267650600228229401496703205376 / 10 in reality is equal to 126765060022822940149670320537.6 = 1.267650600228229401496703205376e+29 (floating point division), but due to the inaccuracy of floats, Python evaluates it as 1.2676506002282295e+29, so you've lost precision, which accounts for the difference.
See PEP 238.
The result of the integer division a//b
a=4//3=1
The result of the float division is a/b
a=4/3=1.33333
In other words, what exists behind the two asterisks? Is it simply multiplying the number x times or something else?
As a follow-up question, is it better to write 2**3 or 2*2*2. I'm asking because I've heard that in C++ it's better to not use pow() for simple calculations, since it calls a function.
If you're interested in the internals, I'd disassemble the instruction to get the CPython bytecode it maps to. Using Python3:
»»» def test():
return 2**3
...:
»»» dis.dis(test)
2 0 LOAD_CONST 3 (8)
3 RETURN_VALUE
OK, so that seems to have done the calculation right on entry, and stored the result. You get exactly the same CPython bytecode for 2*2*2 (feel free to try it). So, for the expressions that evaluate to a constant, you get the same result and it doesn't matter.
What if you want the power of a variable?
Now you get two different bits of bytecode:
»»» def test(n):
return n ** 3
»»» dis.dis(test)
2 0 LOAD_FAST 0 (n)
3 LOAD_CONST 1 (3)
6 BINARY_POWER
7 RETURN_VALUE
vs.
»»» def test(n):
return n * 2 * 2
....:
»»» dis.dis(test)
2 0 LOAD_FAST 0 (n)
3 LOAD_CONST 1 (2)
6 BINARY_MULTIPLY
7 LOAD_CONST 1 (2)
10 BINARY_MULTIPLY
11 RETURN_VALUE
Now the question is of course, is the BINARY_MULTIPLY quicker than the BINARY_POWER operation?
The best way to try that is to use timeit. I'll use the IPython %timeit magic. Here's the output for multiplication:
%timeit test(100)
The slowest run took 15.52 times longer than the fastest. This could mean that an intermediate result is being cached
10000000 loops, best of 3: 163 ns per loop
and for power
The slowest run took 5.44 times longer than the fastest. This could mean that an intermediate result is being cached
1000000 loops, best of 3: 473 ns per loop
You may wish to repeat this for representative inputs, but empirically it looks like the multiplication is quicker (but note the mentioned caveat about the variance in the output).
If you want further internals, I'd suggest digging into the CPython code.
While the second one is little bit faster for numbers, the advantage is very low compared to the first: readability. If you are going for time, and you are pressured to make such optimizations, then python probably is not the language you should use.
Note: for values other than numbers:
a ** b translates to
a.__pow__(b)
whereas a * a * a is a call to
a.__mul__(a.__mul__(a))
Test Code:
import time
s = time.time()
for x in xrange(1,1000000):
x**5
print "done in ", time.time() - s
s = time.time()
for x in xrange(1,1000000):
x*x*x*x*x
print "done in ", time.time() - s
For my machine it yields:
done in 0.975429058075
done in 0.260419845581
[Finished in 1.2s]
If you ask frankly, multiplication is a bit faster.
>>timeit.timeit('[i*i*i*i for i in range(100)]', number=10000)
0.262529843304
>>timeit.timeit('[i**4 for i in range(100)]', number=10000)
0.31143438383
But, speed isn't the only thing to consider when you will choose one from two options. From example, what is easier while computing 2 to the power 20? Simply writing 2**20 or using for loop that will iterate 20 times and do some multiplication task?
The ** operator will, internally, use an iterative function (same semantics as built-in pow() (Python docs), which likely means it just calls that function anyway).
Therefore, if you know the power and can hardcode it, using 2*2*2 would likely be a little faster than 2**3. This has a little to do with the function, but I believe the main performance issue is that it will use a loop.
Note that it's still quite silly to replace more readable code for less readable code when it's something as simple as 2**3, the performance gain is minimal at best.
From the docs:
The power operator binds more tightly than unary operators on its left; it binds less tightly than unary operators on its right. The syntax is:
power ::= primary ["**" u_expr]
Thus, in an unparenthesized sequence of power and unary operators, the operators are evaluated from right to left (this does not constrain the evaluation order for the operands): -1**2 results in -1.
The power operator has the same semantics as the built-in pow() function, when called with two arguments: it yields its left argument raised to the power of its right argument.
This means that, in Python: 2**2**3 is evaluated as 2**(2**3) = 2**8 = 256.
In mathematics, stacked exponents are applied from the top down. If it were not done this way you would just get multiplication of exponents:
(((2**3)**4)**5) = 2**(3*4*5)
It might be a little faster just to do the multiplication, but much less readable.
I want to do some simulation of C language calculation in Python.
For example, unsigned short, single precision float ...
ushort(0xffff) + 1 -> 0
0.1f * 0.1f -> ...
Are there some library to do this in Python?
I can use ctypes to create unsigned short, single float, but they
cann't do math operation:
a = c_uint16(0xffff)
b = c_uint16(0x01)
a+b -> TypeError
Or, I can use numpy:
>>> np.uint16(0xffff) + np.uint16(0x01)
Warning: overflow encountered in ushort_scalars
0
but it's very slow comparing to Python's normal calculation:
>>> timeit.timeit("a+b", "import numpy as np;a=np.uint16(0xfffe);b=np.uint16(0x01)")
0.35577465681618037
>>> timeit.timeit("0xfffe+0x01")
0.022638104432360251
>>> timeit.timeit("np.uint16(0xfffe) + np.uint16(0x01)", "import numpy as np")
5.904765399236851
Edit:
>>> timeit.timeit("a+b", "a=0xfffe;b=0x01")
0.040062221014295574
When compiling 0xfffe+0x01, this will be folded into the constant 65535. You aren't timing how long the addition takes -- you are just measuring the time of loading the constant:
>>> dis.dis(compile("0xfffe+0x01", "", "eval"))
1 0 LOAD_CONST 2 (65535)
3 RETURN_VALUE
The addition of NumPy scalars is slower than adding built-in integers nevertheless, but it won't get better than that in pure Python. Consider using Cython -- it will allow you to declare types and execute the computations in C speed. Alternatively, try to vectorise your code in NumPy (that is, if speed really matters).
You can make a function for each operation using modulo % with 2**sizeof (in your case, 2**16 or 65536)
def add(a, b, mod=2**16):
return (a+b) % mod
def sub(a, b, mod=2**16):
return (a-b) % mod
and any other function you need.
>>> add(0xffff, 1)
0
>>> sub(10, 20)
65526
Note this will work only for unsigned types. For signed ones, you can use half the value used to mod (i.e. 2**15) and will have to validate the result before applying modulo