We Keep Coding

numba: overload of function

I'm trying out numba, the python package that is said to make my nparray super fast. I want to run my function in nonpython mode. What it essentially does is that it takes in an 20x20 array, assigns random numbers to each of its elements, calculate its inverse matrix, then return it.
But here's the problem, when I initialize the array result with np.zeros(), my script crashes and gives me an error message 'overload of function zeros'.
Could someone kindly tell me what is going on? Much appreciated.
from numba import njit
import time
import numpy as np
import random
arr = np.zeros((20,20),dtype = float)
#njit
def aFunctionWithNumba (incomingArray):
result = np.zeros(np.shape(incomingArray), dtype = float)
for i in range(len(incomingArray[0])):
for j in range(len(incomingArray[1])):
incomingArray[i,j] = random.randrange(105150,1541586)
result = np.linalg.inv(incomingArray)
return result
t0 = time.time()
fastArray = aFunctionWithNumba(arr)
t1 = time.time()
s1 = t1 - t0
Here's the full error message:
Exception has occurred: TypingError Failed in nopython mode pipeline (step: nopython frontend) No implementation of function Function(<built-in function zeros>) found for signature:
>>> zeros(UniTuple(int64 x 2), dtype=Function(<class 'float'>)) There are 2 candidate implementations:
- Of which 2 did not match due to: Overload of function 'zeros': File: numba\core\typing\npydecl.py: Line 511.
With argument(s): '(UniTuple(int64 x 2), dtype=Function(<class 'float'>))': No match.
During: resolving callee type: Function(<built-in function zeros>) During: typing of call at c:\Users\Eric\Desktop\testNumba.py (9)
File "testNumba.py", line 9: def aFunctionWithNumba (incomingArray):
result = np.zeros(np.shape(incomingArray), dtype = float)
^ File "C:\Users\Eric\Desktop\testNumba.py", line 25, in <module>
fastArray = aFunctionWithNumba(arr)

The error
You should use Numpy or Numba types inside JITted functions.
Changing the following line your code works:
result = np.zeros(np.shape(incomingArray), dtype=np.float64)
But your code will be more generic using:
result = np.zeros(incomingArray.shape, dtype=incomingArray.dtype)
Or, even better:
result = np.zeros_like(incomingArray)
The timing
The first time you call a JITted function it will take some time to compile it, much longer than the time it will take to execute it. So you should call the function with the same parameter types once before you make any timings.
Additional optimization
If you are interested in comparing the execution time of nested loops with or without Numba, your code is fine. Otherwise you can replace the loops with something like:
incomingArray[:] = np.random.random(incomingArray.shape) * (1541586 - 105150) + 105150

Convert a sympy poly with imaginary powers to an mpmath mpc

I have a sympy poly that looks like:
Poly(0.764635937801645*I**4 + 7.14650839258644*I**3 - 0.667712176660315*I**2 - 2.81663805543677*I - 0.623299856233272, I, domain='RR')
I'm converting to mpc using the following code:
a = val.subs('I',1.0j)
b = sy.re(a)
c = sy.im(a)
d = mpmath.mpc(b,c)
Two questions.
Assuming my mpc and sympy type have equal precision (of eg 100 dps) is there a precision loss using this conversion from a to d?
Is there a better way to convert?
Aside: sympy seems to treat I just like a symbol here. How do I get sympy to simplify this polynomial?
Edit: Ive also noticed that the following works in place of a above:
a = val.args[0]

Strings and expressions
Root cause of the issue is seen in val.subs('I', 1.0j) -- you appear to pass strings as arguments to SymPy functions. There are some valid uses for this (such as creation of high-precision floats), but when symbols are concerned, using strings is a recipe for confusion. The string 'I' gets implicitly converted to SymPy expression Symbol('I'), which is different from SymPy expression I. So the answer to
How do I get sympy to simplify this polynomial?
is to revisit the process of creation of that polynomial, and fix that. If you really need to create it from a string, then use locals parameter:
>>> S('3.3*I**2 + 2*I', locals={'I': I})
-3.3 + 2*I
Polynomials and expressions
If the Poly structure is not needed, use the method as_expr() of Poly to get an expression from it.
Conversion to mpmath and precision loss
is there a precision loss using this conversion from a to d?
Yes, splitting into real and imaginary and then recombining can lead to precision loss. Pass a SymPy object directly to mpc if you know it's a complex number. Or to mpmathify if you want mpmath to decide what type it should have. An example:
>>> val = S('1.111111111111111111111111111111111111111111111111')*I**3 - 2
>>> val
-2 - 1.111111111111111111111111111111111111111111111111*I
>>> import mpmath
>>> mpmath.mp.dps = 40
>>> mpmath.mpc(val)
mpc(real='-2.0', imag='-1.111111111111111111111111111111111111111111')
>>> mpmath.mpmathify(val)
mpc(real='-2.0', imag='-1.111111111111111111111111111111111111111111')
>>> mpmath.mpc(re(val), im(val))
mpc(real='-2.0', imag='-1.111111111111111111111111111111111111111114')
Observations:
When I is actual imaginary unit, I**3 evaluates fo -I, you don't have to do anything for it to happen.
A string representation of high-precision decimal is used to create such a float in SymPy. Here S stands for sympify. One can also be more direct and use Float('1.1111111111111111111111111')
Direct conversion of a SymPy complex number to an mpmath complex number is preferable to splitting in real/complex and recombining.
Conclusion
Most of the above is just talking around an XY problem. Your expression with I was not what you think it was, so you tried to do strange things that were not needed, and my answer is mostly a waste of time.

I'm adding my own answer here, as FTP's answer, although relevant and very helpful, did not (directly) resolve my issue (which wasn't that clear from the question tbh). When I ran the code in his example I got the following:
>>> from sympy import *
>>> import mpmath
>>> val = S('1.111111111111111111111111111111111111111111111111')*I**3 - 2
>>> val
-2 - 1.111111111111111111111111111111111111111111111111*I
>>> mpmath.mp.dps = 40
>>> mpmath.mpc(val)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\mpmath\ctx_mp_python.py", line 373, in __new__
real = cls.context.mpf(real)
File "C:\Python27\lib\site-packages\mpmath\ctx_mp_python.py", line 77, in __new__
v._mpf_ = mpf_pos(cls.mpf_convert_arg(val, prec, rounding), prec, rounding)
File "C:\Python27\lib\site-packages\mpmath\ctx_mp_python.py", line 96, in mpf_convert_arg
raise TypeError("cannot create mpf from " + repr(x))
TypeError: cannot create mpf from -2 - 1.111111111111111111111111111111111111111111111111*I
>>> mpmath.mpmathify(val)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\mpmath\ctx_mp_python.py", line 662, in convert
return ctx._convert_fallback(x, strings)
File "C:\Python27\lib\site-packages\mpmath\ctx_mp.py", line 614, in _convert_fallback
raise TypeError("cannot create mpf from " + repr(x))
TypeError: cannot create mpf from -2 - 1.111111111111111111111111111111111111111111111111*I
>>> mpmath.mpc(re(val), im(val))
mpc(real='-2.0', imag='-1.111111111111111111111111111111111111111114')
>>> mpmath.mpmathify(val)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\mpmath\ctx_mp_python.py", line 662, in convert
return ctx._convert_fallback(x, strings)
File "C:\Python27\lib\site-packages\mpmath\ctx_mp.py", line 614, in _convert_fallback
raise TypeError("cannot create mpf from " + repr(x))
TypeError: cannot create mpf from -2 - 1.111111111111111111111111111111111111111111111111*I
Updating my sympy (1.0->1.1.1) and mpmath (0.19->1.0.0) fixed the exceptions. I did not test which of these upgrades actually resolved the issue.

Factorial in numpy and scipy

How can I import factorial function from numpy and scipy separately in order to see which one is faster?
I already imported factorial from python itself by import math. But, it does not work for numpy and scipy.

You can import them like this:
In [7]: import scipy, numpy, math
In [8]: scipy.math.factorial, numpy.math.factorial, math.factorial
Out[8]:
(<function math.factorial>,
<function math.factorial>,
<function math.factorial>)
scipy.math.factorial and numpy.math.factorial seem to simply be aliases/references for/to math.factorial, that is scipy.math.factorial is math.factorial and numpy.math.factorial is math.factorial should both give True.

The answer for Ashwini is great, in pointing out that scipy.math.factorial, numpy.math.factorial, math.factorial are the same functions. However, I'd recommend use the one that Janne mentioned, that scipy.special.factorial is different. The one from scipy can take np.ndarray as an input, while the others can't.
In [12]: import scipy.special
In [13]: temp = np.arange(10) # temp is an np.ndarray
In [14]: math.factorial(temp) # This won't work
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-14-039ec0734458> in <module>()
----> 1 math.factorial(temp)
TypeError: only length-1 arrays can be converted to Python scalars
In [15]: scipy.special.factorial(temp) # This works!
Out[15]:
array([ 1.00000000e+00, 1.00000000e+00, 2.00000000e+00,
6.00000000e+00, 2.40000000e+01, 1.20000000e+02,
7.20000000e+02, 5.04000000e+03, 4.03200000e+04,
3.62880000e+05])
So, if you are doing factorial to a np.ndarray, the one from scipy will be easier to code and faster than doing the for-loops.

SciPy has the function scipy.special.factorial (formerly scipy.misc.factorial)
>>> import math
>>> import scipy.special
>>> math.factorial(6)
720
>>> scipy.special.factorial(6)
array(720.0)

from numpy import prod
def factorial(n):
print prod(range(1,n+1))
or with mul from operator:
from operator import mul
def factorial(n):
print reduce(mul,range(1,n+1))
or completely without help:
def factorial(n):
print reduce((lambda x,y: x*y),range(1,n+1))

after running different aforementioned functions for factorial, by different people, turns out that math.factorial is the fastest to calculate the factorial.
find running times for different functions in the attached image

You can save some homemade factorial functions on a separate module, utils.py, and then import them and compare the performance with the predefinite one, in scipy, numpy and math using timeit.
In this case I used as external method the last proposed by Stefan Gruenwald:
import numpy as np
def factorial(n):
return reduce((lambda x,y: x*y),range(1,n+1))
Main code (I used a framework proposed by JoshAdel in another post, look for how-can-i-get-an-array-of-alternating-values-in-python):
from timeit import Timer
from utils import factorial
import scipy
n = 100
# test the time for the factorial function obtained in different ways:
if __name__ == '__main__':
setupstr="""
import scipy, numpy, math
from utils import factorial
n = 100
"""
method1="""
factorial(n)
"""
method2="""
scipy.math.factorial(n) # same algo as numpy.math.factorial, math.factorial
"""
nl = 1000
t1 = Timer(method1, setupstr).timeit(nl)
t2 = Timer(method2, setupstr).timeit(nl)
print 'method1', t1
print 'method2', t2
print factorial(n)
print scipy.math.factorial(n)
Which provides:
method1 0.0195569992065
method2 0.00638914108276
93326215443944152681699238856266700490715968264381621468592963895217599993229915608941463976156518286253697920827223758251185210916864000000000000000000000000
93326215443944152681699238856266700490715968264381621468592963895217599993229915608941463976156518286253697920827223758251185210916864000000000000000000000000
Process finished with exit code 0

numpy measure time - syntax error

I want to measure the time through numpy but I am not sure if I have the right arguments.
import numpy as np
import timeit
def svdsolve(a,b):
u,s,v = np.linalg.svd(a)
c = np.dot(u.T,b)
w = np.linalg.solve(np.diag(s),c)
x = np.dot(v.T,w)
return x
A_=np.fromfile('dataA.bin',count=-1,dtype=np.float32)
B_=np.fromfile('dataB.bin',count=-1,dtype=np.float32)
s='svdsolve({0},{1})'.format(A,B)
mytime= timeit.Timer(stmt=s,setup='import numpy').timeit(100)
print mytime
Right now it gives me :
File "/usr/lib64/python2.7/timeit.py", line 136, in init
code = compile(src, dummy_src_name, "exec") File "", line 6
svdsolve([[ 1.86248358e+09 1.54404045e+09]
^ SyntaxError: invalid syntax
Also , I didn' understand (neither have I found a reference) on what arguments to pass at timer.
And I am not sure how to use the repetition timeit(100)

Neither the str nor repr of a NumPy array is guaranteed to produce an output that can be used to reconstruct the original array. (format uses str, but repr wouldn't help.) Instead, import the arrays into the timed code's namespace. Assuming this code is being run as a script, that would be
mytime = timeit.Timer(stmt='svdsolve(A, B)',
setup='from __main__ import A, B, svdsolve'
).timeit(100)

my module won't load

i am sorry,i am just a beginner in python language,i am quite stuck in this problem quite long.actually,i want to make a descending and ascending of list that the user input by creating a module of the descending and the ascending.but i couldn't get it work.
the main python file is pythonaslab.py and the module for the ascending and the descending is selectionmodule.py..the code:
this is the selectionmodule:
import pythonaslab
def ascendingselection():
for q in range(len(b)):
w=q+1
for w in range(len(b)):
if b[q]>b[w]:
f=b[q]
b[q]=b[w]
b[w]=f
print b
def descendingselection():
for q in range(len(b)):
w=q+1
for w in range(len(b)):
if b[q]<b[w]:
f=b[q]
b[q]=b[w]
b[w]=f
print b
And this is the main file,the pythonaslab:
import selectionmodule
a = int(input())
b = [int(input()) for _ in range(a)]
print b
print "1.ascending 2.descending"
c=input()
if c==1:
selectionmodule.ascendingselection()
if c==2:
selectionmodule.descendingselection()
can you point me where's the cause of all this error i got?
Traceback (most recent call last):
File "E:\Coding\pythonaslab.py", line 1, in <module>
import selectionmodule
File "E:\Coding\selectionmodule.py", line 1, in <module>
import pythonaslab
File "E:\Coding\pythonaslab.py", line 16, in <module>
selectionmodule.descendingselection()
AttributeError: 'module' object has no attribute 'descendingselection'

You created a circular import; your pythonaslab module imports selectionmodule which imports the pythonaslab module. You end up with incomplete modules that way, don't do that.
Remove the import pythonaslab line from selectionmodule; you are not using pythonaslab in that module.
Also, another module cannot read your globals; you need to pass those in as arguments:
# this function takes one argument, and locally it is known as b
def ascendingselection(b):
# rest of function ..
then call that with:
selectionmodule.ascendingselection(b)
Note that you are not limited to one-letter variable names. Using longer, descriptive names makes your code more readable.

if you don't want to use module name such as:
selectionmodule.ascendingselection(b)
you should import :
from selectionmodule import *
then you can call:
ascendingselection(b) # without module name
Or you can import your module and assigne a alias name:
import selectionmodule as o
o.ascendingselection(b) # with alias name
for more information read: import confusaion