We Keep Coding

Conda Numba Cuda: libNVVM cannot be found

My development environment is: Ubuntu 18.04.5 LTS, Python3.6 and I have installed via conda (numba and cudatoolkit). Nvidia GPU GeForce GTX 1050 Ti, which is supported by cuda.
The installation of conda and numba seem to work as intended as I can import numba within python3.6 scripts.
The problem seems identical situation to the question asked here: Cuda: library nvvm not found
but none of the proposed solutions seem to work in my case, and I'm not sure how to highlight my situation properly (I can't do it through an answer in the other thread...). If raising a duplicate of the question is inappropriate, then guide me to proper conduct.
When I try to run the code below I get the following error: numba.cuda.cudadrv.error.NvvmSupportError: libNVVM cannot be found. Do conda install cudatoolkit: library nvvm not found
from numba import cuda, float32
#Controls threads per block and shared memory usage.
#The computation will be done on blocks of TPBxTPB elements.
TPB = 16
#cuda.jit
def fast_matmul(A, B, C):
# Define an array in the shared memory
# The size and type of the arrays must be known at compile time
sA = cuda.shared.array(shape=(TPB, TPB), dtype=float32)
sB = cuda.shared.array(shape=(TPB, TPB), dtype=float32)
x, y = cuda.grid(2)
tx = cuda.threadIdx.x
ty = cuda.threadIdx.y
bpg = cuda.gridDim.x # blocks per grid
if x >= C.shape[0] and y >= C.shape[1]:
# Quit if (x, y) is outside of valid C boundary
return
# Each thread computes one element in the result matrix.
# The dot product is chunked into dot products of TPB-long vectors.
tmp = 0.
for i in range(bpg):
# Preload data into shared memory
sA[tx, ty] = A[x, ty + i * TPB]
sB[tx, ty] = B[tx + i * TPB, y]
# Wait until all threads finish preloading
cuda.syncthreads()
# Computes partial product on the shared memory
for j in range(TPB):
tmp += sA[tx, j] * sB[j, ty]
# Wait until all threads finish computing
cuda.syncthreads()
C[x, y] = tmp
import numpy as np
matrix_A = np.array([[0.1,0.2],[0.1,0.2]])
Doing as suggested and running conda install cudatoolkit does not work. I have tried many variations on this install that I've found online to no avail.
In the other post a solution that seems to have worked for many is to add lines about environment variables in the .bashrc file in the home directory. The suggestions however refer to files that exist in the /usr directory, where I have no cuda data since I've installed through conda. I have tried many variations on these exports without success. This is perhaps where the solution lies, but if so then the solution would benefit from being generalized.
Does anyone have any up-to-date or generalized solutions to this problem?
EDIT: adding information from terminal outputs (thanks for the hint of editing the question to do so)
> conda list numba
# packages in environment at /home/tobka/anaconda3:
#
# Name Version Build Channel
numba 0.51.2 py38h0573a6f_1
> conda list cudatoolkit
# packages in environment at /home/tobka/anaconda3:
#
# Name Version Build Channel
cudatoolkit 11.0.221 h6bb024c_0
Also adding output from numba -s: https://pastebin.com/raw/6u1MUkxg
Idea of possible cause (not yet confirmed): I noticed in the numba -s output that it specifies Python Version: 3.8.3, where I've been explicitly using python3.6 in the terminal since simply using python has usually meant using python2.7. I checked however, and my system now uses Python 3.8.3 with the python command, and Python 3.6.9 with the python3.6. And when running the code using python I get a syntax error instead, which is a good sign: raise ValueError(missing_launch_config_msg).
I will try to fix the syntax errors and confirm that the code works, after which I will report here of the situation.

Confirmation of solution: using python instead of python3.6 in the terminal solved the problem. The root cause was the user.

Why jupyter notebook only prints the cython result once?

I am new to cython(only use it for doing a little hw now).
I use the following code to see a general idea of it in jupyter notebook.
%load_ext Cython
%%cython
def cfunc(int n):
cdef int a = 0
for i in range(n):
a += i
return a
print(cfunc(10))
However, it only prints out the result 45 once. When I run the print function, the cell doesn't show 45 anyone.
Is there any problems with the code? How can I make the cell prints out 45 just the same as a normal python code? Thanks.

When running %%cython-magic a lot happens under the hood. One can see parts of it when calling the magic in verbose mode, i.e. %%cython --verbose:
A file called _cython_magic_b599dcf313706e8c6031a4a7058da2a2.pyx is generated. b599dcf313706e8c6031a4a7058da2a2 is the sha1-hash of the %%cython-cell, which is needed for example to be able to reload a %%cython-cell (see this SO-post).
This file is cythonized and build to a c-extension called _cython_magic_b599dcf313706e8c6031a4a7058da2a2.
This extension gets imported - this is the moment your code prints 45, and everything from this module is added to the global namespace.
When you execute the cell again, nothing of the above happens: given the sha-hash the machinery can see, that this cell was already executed and loaded - so nothing to be done. Only when the content of the cell is changed and thus its hash the cash will not be used but the 3 steps above executed.
To enforce that the steps above are performed one has to pass --force (or -f) options to the %%cython-magic-cell, i.e.:
%%cython --force
...
# 45 is printed
However, because building extension anew is quite time consuming one would probably prefer the following
%%cython
def cfunc(int n):
cdef int a = 0
for i in range(n):
a += i
return a
# put the code of __main__ into a function
def cython_main():
print(cfunc(10))
# execute the old main
cython_main()
and now calling cython_main() in a new cell, so it gets reevaluated the same way the normal python code would.

Simple pybind11 module fails with No module named

I've created a python binding for one of my projects a while back and just now wanted to pick it up again.
The binding was no longer working as python was no longer able to import it - this all was working fine back then.
I've then decided to break it down to the simplest possible example:
binding.cpp
#include <pybind11/pybind11.h>
int add(int i, int j) {
return i + j;
}
PYBIND11_MODULE(TestBinding, m) {
m.doc() = "pybind11 example plugin"; // optional module docstring
m.def("add", &add, "A function which adds two numbers");
}
CMakeLists.txt:
cmake_minimum_required( VERSION 3.2 )
project(TestBinding)
add_subdirectory(pybind11) # or find_package(pybind11)
pybind11_add_module(TestBinding binding.cpp)
# Configure project to inject source path as include directory on dependent projects
target_include_directories( TestBinding
INTERFACE
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/pybind11/include/> )
set_target_properties( TestBinding
PROPERTIES
CXX_STANDARD 17
CXX_STANDARD_REQUIRED ON
PREFIX ""
SUFFIX ".so"
)
Then I have a very simple test.py file which goes like this:
sys.path.insert(0, "/path/to/so/lib/")
from TestBinding import *
...which once executed always gives me the following error:
from TestBinding import *
ModuleNotFoundError: No module named 'TestBinding'
I have literally no idea anymore what in the world could have changed from when it worked just fine and now.
Here are some more informations about my working environment:
Windows 10
Visual Studio 15 2017 Win64
Python 3.7 (also tried 3.5 and 3.6)
Am I missing anything really obvious?

I've been able to resolve this by removing the SUFFIX ".so" rule from my CMakeLists.txt.
This was needed back when I've initially created my bindings, but it no longer is apparently.

I have the same problem as you. After checking, it is found that the problem is caused by the inconsistency between the python version of pybind11 and the python version of the local environment. My problem was solved when I adjusted to the same python version.

How to use cython compiler in python

in this link: http://earthpy.org/speed.html I found the following
%%cython
import numpy as np
def useless_cython(year):
# define types of variables
cdef int i, j, n
cdef double a_cum
from netCDF4 import Dataset
f = Dataset('air.sig995.'+year+'.nc')
a = f.variables['air'][:]
a_cum = 0.
for i in range(a.shape[0]):
for j in range(a.shape[1]):
for n in range(a.shape[2]):
#here we have to convert numpy value to simple float
a_cum = a_cum+float(a[i,j,n])
# since a_cum is not numpy variable anymore,
# we introduce new variable d in order to save
# data to the file easily
d = np.array(a_cum)
d.tofile(year+'.bin')
print(year)
return d
It seems to be as easy as to just write %%cython over the function. However this just doesnt work for me -> "Statement seems to have no effect" says my IDE.
After a bit of research I found that the %% syntax comes from iphyton which I did also install (as well as cython). Still doesnt work. Iam using python3.6
Any ideas?

Once you are in the IPython interpreter you have to load the extension prior to using it. It can be done with the statement %load_ext, so in your case :
%load_ext cython
These two tools are pretty well documented, if you have not seen it yet, take a look at the relevant part of the document on cython side and on IPython side.

Understanding Cython "typedness" report

I'm using Cython to make my
Python code more efficient. I have read about the Cython's function cython -a filename.pyx to see the "typedness" of my cython code. Here is the short reference from Cython web page. My environment is Windows 7, Eclipse PyDev, Python 2.7.5 32-bit, Cython 0.20.1 32-bit, MinGW 32-bit.
Here is the report for my code:
So does the color yellow mean efficient or non-efficient code? The more yellow it is the more....what?
Another question, I can click on the numbered rows and the following report opens (e.g. row 23):
What does this mean? P.S. If you can't see the image well enough --> right click --> view image (on Windows 7) ;)
Thnx for any assistance =)
UPDATE:
In case somebody wants to try my toy code here they are:
hello.pyx
import time
cdef char say_hello_to(char name):
print("Hello %s!" % name)
cdef double f(double x) except? -2:
return x**2-x
cdef double integrate_f(double a, double b, int N) except? -2:
cdef int i
cdef double s, dx
s = 0
dx = (b-a)/N
for i in range(N):
s += f(a+i*dx)
return s * dx
cpdef p():
s = 0
for i in range(0, 1000000):
c = time.time()
integrate_f(0,100,5)
s += time.time()- c
print s
test_cython.py
import hello as hel
hel.p()
setup.py
from distutils.core import setup
from Cython.Build import cythonize
setup(
name = 'Hello world app',
ext_modules = cythonize("hello.pyx"),
)
From command line prompt I used the command (to generate C, pyd etc files):
python setup.py install build --compiler=mingw32
To generate the report I used:
cython -a hello.pyx

This is more or less what you wrote. To be more precise:
Yellowish line signal Cython command which are not directly translated to pure C code but which work by calling CPython API to do the job. Those line includes:
Python object allocation
calls to Python function and builtins
operating on Python high level data stuctures (eg: list, tuples, dictionary)
use of overloaded operation on Python types (eg: +, * in Python integers vs C int)
In any case, this is a good indication that thing might be improved.

I think I perhaps got it myself already. Someone can correct me if I'm mistaken:
The more "yellowish" the line is, then less efficient it is
The most efficient lines are the white-colored lines, because these are translated into pure C-code.