I am trying to implement user's #rkp solution to their own question of how to speed up sparse matrix multiplications with cython by using the pycuda library (please note this is their second solution in their post).
After installing pycuda, pymetis etc and running their exact same code (in IDLE Python 3.5.2) I am getting:
TypeError: 'numpy.float64' object cannot be interpreted as an integer
It turns out the (reproducible) part that produces this error is:
import numpy as np
import pycuda.autoinit
import pycuda.driver as drv
import pycuda.gpuarray as gpuarray
from pycuda.sparse.packeted import PacketedSpMV
from pycuda.tools import DeviceMemoryPool
from scipy.sparse import csr_matrix
COUNT = 100
N = 5000
P = 0.1
DTYPE = np.int32
#construct objects
np.random.seed(0)
a_dense = np.random.rand(N, N).astype(DTYPE)
a_dense[np.random.rand(N, N) >= P] = 0
a_sparse = csr_matrix(a_dense)
#PacketedSpMV produces the error
spmv = PacketedSpMV(a_sparse, is_symmetric=False, dtype=DTYPE)
And the full error:
Traceback (most recent call last):
File "C:/Users/svobodov/Desktop/data/tests/cython/t.py", line 23, in <module>
spmv = PacketedSpMV(a_sparse, is_symmetric=False, dtype=DTYPE)
File "C:\Python35\lib\site-packages\pycuda\sparse\packeted.py", line 185, in __init__
local_row_costs)
File "pkt_build_cython.pyx", line 22, in pycuda.sparse.pkt_build_cython.build_pkt_data_structure
TypeError: 'numpy.float64' object cannot be interpreted as an integer
I initially thought this to be the cython-related double-precision error but this is obviously something different as it is expecting specifically an integer rather than float32..
I tried tweaking the pkt_build_cython.pyx but without any success or confidence that I did it properly.
Any ideas on how to resolve this please?
As identified in comments, this was a result of a missing integer cast within an internal routine in the PyCUDA codebase.
The bug was actually fixed in 2018, so if you use any PyCUDA 2019 release, you should have the corrected code and this issue should not occur.
Related
I occasionally use the where clause in numpy's ufuncs. For example, the following:
import numpy as np
a = np.linspace(-1, 1, 10)
np.sqrt(a, where=a>0) * (a>0)
In Numpy 1.12 and earlier, this used to give me square root values where possible and zero otherwise.
Recently, though, I upgraded to numpy 1.13. The code above now gives me the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: Automatic allocation was requested for an iterator operand, and it was flagged as readable, but buffering without delayed allocation was enabled
I thought that this was exactly how the where clause was supposed to be used, but perhaps I was wrong. So I have two questions: first, what's wrong with this code; and second, what is the recommended way of achieving my goal?
For future reference: this turned out to be a bug in numpy. It has been fixed for the next numpy release, presumably version 1.13.1.
A workaround fix for 1.13.0 is to explicitly provide an out parameter to the ufunc. In the example above, np.sqrt(a, where=a>0, out=np.zeros(a.shape)) works.
I'm trying to draw plots in Python with Scipy module. According to http://docs.scipy.org/doc/scipy/reference/special.html I wrote code with scipy.special.spherical_jn(n,x,0):
import matplotlib.pyplot as plt
import numpy as np
import scipy.special as sp
from matplotlib import rcParams
rcParams.update({'figure.autolayout': True})
def odrazTE(a,o,d):
temp1 = sp.spherical_jn[1,a,0]
temp2 = 1
return abs(temp1/temp2)**2
t = np.arange(0.001, 2, 0.001)
plt.plot(t,odrazTE(t,t,1),label='TE1')
plt.show()
While I'm compiling the program, all I get is this error:
Traceback (most recent call last):
File "standing-sphere.py", line 33, in <module>
plt.plot(t,odrazTE(t,t,1),label='TE1')
File "standing-sphere.py", line 15, in odrazTE
temp1 = sp.spherical_jn[1,a,0]
AttributeError: 'module' object has no attribute 'spherical_jn'
There is way how to do it with regular Bessel function and relationship between Bessel and spherical Bessel function, but I don't like this solution because of derivative of sph.bess. function that I need too.
Is there any chance I have set something wrongly and it can be fixed to scipy.special.spherical_jn work?
scipy.special.spherical_jn was added in scipy version 0.18.0, which was released on July 25, 2016. My guess is you are using an older version of scipy. To check, run
import scipy
print(scipy.__version__)
I use python 2.7.8 with the Anaconda distribution and I have problems with scipy.
Let A be a sparse matrix; I want to calculate its eigenvalues but if I write:
import scipy
scipy.sparse.linalg.eigs(A)
I get the error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'module' object is not callable
What is the problem? (The version of scipy is 0.15.1)
Does this work for you?
from scipy import sparse
import scipy.sparse.linalg as sp_linalg
B = np.random.rand(10,10)
A_dense = np.dot(B.T, B)
A_sparse = sparse.lil_matrix(A_dense)
sp_linalg.eigs(A_sparse, 3)
It seems that you have to explicitly import the submodules. scipy does not load those per default.
I am using rpy2-2.2.2 with the new free Enthought python distribution that includes numpy 1.6.0 and python 2.7.2. I easy_installed rpy2 which resulted in v. 2.2.2 being installed and all tests were successful.
The problem I'm having is with code I wrote that worked fine with rpy2 2.1.8 and python 2.6. The issue is in converting from numpy to R for arrays.
Here is a snippet of the relevant code:
import rpy2
import rpy2.rinterface as rinterface
import rpy2.robjects as rob
import rpy2.rlike.container as rlc
import numpy as np
import rpy2.robjects.numpy2ri
r = rob.r
...
HGr = rob.conversion.py2ri(HG_reg)
RHSr = rob.conversion.py2ri(RHS)
#
CalData = rlc.TaggedList([HGr,RHSr],tags=('hg','rhs'))
CalData = rob.DataFrame(CalData)
r('''library(pls)''')
#rob.globalEnv["HGr"] = HGr
#rob.globalEnv["RHSr"] = RHSr
rob.globalenv["CalData"] = CalData
# perform the PLS regression
if wetlflag:
HGresults = r.plsr(r("hg ~ rhs.1 + rhs.2 + rhs.3 + rhs.4"),data=CalData,validation="LOO")
I will gladly admit it's not the most elegant way to do things, but it worked before and now when I need to provide results all is broken (!). The error I get is the following:
Traceback (most recent call last):
File "Mercury_PLS_WL_DF.py", line 224, in <module>
HGr = rob.conversion.py2ri(HG_reg)
File "/Library/Frameworks/Python.framework/Versions/7.1/lib/python2.7/site-packages/rpy2-2.2.2dev_20110726-py2.7-macosx-10.5-i386.egg/rpy2/robjects/__init__.py", line 134, in default_py2ri
raise(ValueError("Nothing can be done for the type %s at the moment." %(type(o))))
ValueError: Nothing can be done for the type <type 'numpy.ndarray'> at the moment.
I found the discussion here and got the impression that numpy arrays are now automatically converted to R arrays, but commenting out the rob.conversion.py2ri(HG_reg) statements and using the numpy arrays directly also seems to fail. Am I missing something obvious? Why would this break between 2.1.8 and 2.2.2?
From http://rpy.sourceforge.net/rpy2/doc-2.2/html/numpy.html#from-numpy-to-rpy2:
Warning
In earlier versions of rpy2, the import was all that was needed to have the conversion. A side-effect when importing a module can lead to problems, and there is now an extra step to make the conversion active: call the function rpy2.robjects.activate().
So put rpy2.robjects.activate() after the import and you should be fine.
The following code is supposed to create a heatmap in rpy2
import numpy as np
from rpy2.robjects import r
data = np.random.random((10,10))
r.heatmap(data)
However, it results in the following error
Traceback (most recent call last):
File "z.py", line 8, in <module>
labRow=rowNames, labCol=colNames)
File "C:\Python25\lib\site-packages\rpy2\robjects\__init__.py", line 418, in __call__
new_args = [conversion.py2ri(a) for a in args]
File "C:\Python25\lib\site-packages\rpy2\robjects\__init__.py", line 93, in default_py2ri
raise(ValueError("Nothing can be done for the type %s at the moment." %(type(o))))
ValueError: Nothing can be done for the type <type 'numpy.ndarray'> at the moment.
From the documentation I learn that r.heatmap expects "a numeric matrix". How do I convert np.array to the required data type?
You need to add
import rpy2.robjects.numpy2ri
rpy2.robjects.numpy2ri.activate()
See more in rpy2 documentation numpy section (here for the older 2.x version)
Prior to 2.2.x the import alone was sufficient.
That import alone is sufficient to
switch an automatic conversion of
numpy objects into rpy2 objects.
Why make this an optional import,
while it could have been included in
the function py2ri() (as done in the
original patch submitted for that
function) ?
Although both are valid and reasonable
options, the design decision was taken
in order to decouple rpy2 from numpy
the most, and do not assume that
having numpy installed automatically
meant that a programmer wanted to use
it.
For rpy2 2.2.4 I had to add:
import rpy2.robjects.numpy2ri
rpy2.robjects.numpy2ri.activate()
For me (2.2.1) the following also worked (as documented on http://rpy.sourceforge.net/rpy2/doc-2.2/html/numpy.html):
import rpy2.robjects as ro
from rpy2.robjects.numpy2ri import numpy2ri
ro.conversion.py2ri = numpy2ri