We Keep Coding

Numba doesn't work for numpy.random function

I am trying to implement Numba on a simple Monte Carlo integrator.
import numpy as np
import numba
import time
#numba.njit
def integrator(integrand,
lower_bounds,
upper_bounds,
n):
if len(lower_bounds) != len(upper_bounds):
raise ValueError("Lower and upper bounds are of different dimensions.")
x = np.random.uniform(lower_bounds,
upper_bounds,
(n, len(lower_bounds)))
summation = 0
for i in x:
summation += integrand(i)
domain = np.prod(np.subtract(upper_bounds, lower_bounds))
integral = domain/n*summation
return integral
start = time.time()
integrator(numba.njit(lambda x: x**2),
np.array([-5]),
np.array([5]),
10000000)
end = time.time()
print(end-start)
However, I am getting the following error:
>>> integrator(numba.njit(lambda x: x**2),
... np.array([-5]),
... np.array([5]),
... 10000000)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\numba\core\dispatcher.py", line 468, in _compile_for_args
error_rewrite(e, 'typing')
File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\numba\core\dispatcher.py", line 409, in error_rewrite
raise e.with_traceback(None)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
←[1m←[1m←[1mNo implementation of function Function(<built-in method uniform of numpy.random.mtrand.RandomState object at 0x0000019A9E477040>) found for signature:
>>> uniform(array(int32, 1d, C), array(int32, 1d, C), UniTuple(int64 x 2))
There are 4 candidate implementations:
←[1m - Of which 4 did not match due to:
Overload in function '_OverloadWrapper._build.<locals>.ol_generated': File: numba\core\overload_glue.py: Line 129.
With argument(s): '(array(int32, 1d, C), array(int32, 1d, C), UniTuple(int64 x 2))':←[0m
←[1m Rejected as the implementation raised a specific error:
TypingError: ←[1mNo match←[0m←[0m
raised from C:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\numba\core\overload_glue.py:162
←[0m
←[0m←[1mDuring: resolving callee type: Function(<built-in method uniform of numpy.random.mtrand.RandomState object at 0x0000019A9E477040>)←[0m
←[0m←[1mDuring: typing of call at <stdin> (21)
←[0m
←[1m
File "<stdin>", line 21:←[0m
←[1m<source missing, REPL/exec in use?>←[0m
It seems that there is some issue with passing dimensions as a tuple argument to np.random.uniform(). However, I know that np.random.uniform() is supported by Numba and using tuples there is necessary. What is the problem here?

Why this error is coming while using numpy and jit

I am trying to run this code but it is throwing error that:
Failed in nopython mode pipeline
If I am running this with #jit it working but not with jit.
import numpy as np
from numba import jit
#jit(nopython=True)
def np_clean(data):
data[:, 0] = np.char.replace(data[:, 0].astype('U'), '-I.NFO', '-I')
data = data[np.char.endswith(data[:, 0].astype('U'), '-I')]
data = data[~np.char.find(data[:, 0].astype('U'), 'NIFTY')]
data[:, 1] = np.char.replace(data[:, 1].astype('U'), '-', '/')
data[:, 2] = [i[:-2]+'00'for i in data[:, 2]]
return data
np_clean(df.values)
Error:
---------------------------------------------------------------------------
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
non-precise type array(pyobject, 2d, F)
During: typing of argument at C:\Users\caaka\AppData\Local\Temp/ipykernel_12896/1978023246.py (4)
File "..\..\..\..\AppData\Local\Temp\ipykernel_12896\1978023246.py", line 4:
<source missing, REPL/exec in use?>

TypeError: unsupported operand type(s) for *: 'float' and 'NoneType' when building pyomo model

I have a problem with a pyomo model, which has been partly answered here already: TypeError PYOMO: Defining constraints based on pandas dataframe
However, first of all I don't really understand the problem and if I apply the proposed fix I get another error.
So here I will provide you with some of my code. I have a photovoltaic model with some constraints and a solar thermal model with some constraints that theoretically could fight for the same available area:
#total PV power generation from ground/tilt roof/wall at teach timestep
m.pv_groundGen = Var(year_i, ts_i, within = NonNegativeReals, initialize = 0)
m.pv_roofGen = Var(year_i, ts_i, within = NonNegativeReals, initialize = 0)
#PV covered roof area at different angles
m.pv_roofPanelArea = Var(r_az, r_tilt, within = NonNegativeReals, initialize = 0)
m.pv_groundPanelArea = Var(within = NonNegativeReals, initialize = 0)
if "pv_ground" in ders_df.index.values:
# Relation between installed capacity and panel area
def pv_groundPanelAreaRule(m):
return m.c_c["pv_ground"] == m.pv_groundPanelArea * pv_eff
m.const_pvGroundArea = Constraint(rule = pv_groundPanelAreaRule)
#Some constraints describing pv generation cut for visibility
#Limits the area available for ground and flat roof pv
if "solar_thermal_ground" not in ders_df.index.values:
def panel_groundAreaLimit(m):
return m.pv_groundPanelArea <= pv_groundareaAvail * pv_gcr_ground
m.const_panelGroundAreaLimit = Constraint(rule = panel_groundAreaLimit)
if "pv_roof" in ders_df.index.values:
# Relation between installed capacity and panel area
def pv_roofPanelAreaRule(m):
return m.c_c["pv_roof"] == sum(m.pv_roofPanelArea[az, tilt] for az in r_az \
for tilt in r_tilt) * pv_eff
m.const_pvRoofArea = Constraint(rule = pv_roofPanelAreaRule)
#Some constraints describing pv generation cut for visibility
#Limits the area available for ground and flat roof pv
if "solar_thermal_roof" not in ders_df.index.values:
def panel_roofAreaLimit(m, az, tilt):
return m.pv_roofPanelArea[az, tilt] <= pv_roofwallareaAvail.at[az, tilt] * pv_gcr_roof
m.const_panelRoofAreaLimit = Constraint(r_az, r_tilt, rule = panel_roofAreaLimit)
m.st_groundPanelArea = Var(within = NonNegativeReals, initialize = 0)
m.st_roofPanelArea = Var(r_az, r_tilt, within = NonNegativeReals, initialize = 0)
if "solar_thermal_ground" in ders_df.index.values:
def stUsefulHeatG_rule(m,y,ts):
return sum(m.heat["solar_thermal_ground", hCons, y, ts] for hCons in hIn) == \
(st_eff_0*irrad_tilt_flat[ts] *1000 - st_a1*(st_Tf-temp[ts]) - \
st_a2*((st_Tf-temp[ts])**2)) * m.st_groundPanelArea
m.const_stUsefulHeatG = Constraint(year_i,ts_i,rule = stUsefulHeatG_rule)
def stpanelAreaG_rule(m):
return m.c_c["solar_thermal_ground"] == m.st_groundPanelArea * 0.717 # official conversion factor
m.const_stpanelAreaG = Constraint(rule = stpanelAreaG_rule)
if "pv_ground" in ders_df.index.values:
def panel_groundAreaLimit(m):
return m.pv_groundPanelArea/pv_gcr_ground + m.st_groundPanelArea*1.7 <= pv_groundareaAvail
m.const_panelGroundAreaLimit = Constraint(rule = panel_groundAreaLimit)
else:
def panel_groundAreaLimit(m):
return m.st_groundPanelArea*1.7 <= pv_groundareaAvail
m.const_panelGroundAreaLimit = Constraint(rule = panel_groundAreaLimit)
if "solar_thermal_roof" in ders_df.index.values:
def stUsefulHeatR_rule(m,y,ts):
return sum(m.heat["solar_thermal_roof", hCons, y, ts] for hCons in hIn) == \
sum((st_eff_0*irrad_tilt_tilt_df.at[az,tilt][ts] *1000-st_a1*(st_Tf-temp[ts]) - \
st_a2*((st_Tf-temp[ts])**2)) * m.st_roofPanelArea[az, tilt] for az in r_az for tilt in r_tilt)
m.const_stUsefulHeatR = Constraint(year_i,ts_i,rule = stUsefulHeatR_rule)
def stpanelAreaR_rule(m):
return m.c_c["solar_thermal_roof"] == sum(m.st_roofPanelArea[az, tilt] \
for az in r_az for tilt in r_tilt) * 0.717 # official conversion factor
m.const_stpanelAreaR = Constraint(rule = stpanelAreaR_rule)
if "pv_roof" in ders_df.index.values:
def panel_roofAreaLimit(m, az, tilt):
return m.pv_roofPanelArea[az, tilt]/pv_gcr_roof + m.st_roofPanelArea[az, tilt] * \
1.7 <= pv_roofwallareaAvail.at[az, tilt]
m.const_panelRoofAreaLimit = Constraint(r_az, r_tilt, rule = panel_roofAreaLimit)
else:
def panel_roofAreaLimit(m, az, tilt):
return m.st_roofPanelArea[az, tilt] * 1.7 <= pv_roofwallareaAvail.at[az, tilt]
m.const_panelRoofAreaLimit = Constraint(r_az, r_tilt, rule = panel_roofAreaLimit)
Now the problem is, if I build the model, which includes the part I posted here, I get this error:
TypeError: unsupported operand type(s) for *: 'float' and 'NoneType' (showing me this line st_a2*((st_Tf-temp[ts])**2)) * m.st_groundPanelArea
During handling of the above exception, another exception occurred:
TypeError: stUsefulHeatG_rule() missing 2 required positional arguments: 'y' and 'ts'
Now, if I take the advice from the link above and create the variable m.st_groundPanelArea which seems to be the trouble maker with a Set [1,] I get a new error:
AttributeError: 'numpy.ndarray' object has no attribute 'is_expression_type'
Reading the other thread and the error codes I reckon this all has something to do with an interaction problem of numpy and pyomo but I really do not understand why I do not have the same issue with m.pv_groundGen for example
Thanks a lot and all the best :)

Your post does not have all of the variables defined in order to really troubleshoot this well. If the below doesn't answer your question, you should edit your post by removing unnecessary parts of the model and include variable definitions and a snippet of sample data to reproduce the error.
That said, I think you are getting bit by another manifestation of the same issue regarding the interaction of non-indexed pyomo variables with numpy data types. I think that both pyomo and numpy are trying to override some of the basic algebraic functions to produce consistent data types. pyomo is trying to create pyomo.expressions, while numpy is trying to maintain ndarrays. (There's probably a fuller description of this issue in the pyomo bug report on this issue, but that, I think, is biting you.)
My suggestion, especially when you have a singleton pyomo variable, is just to not use numpy arrays or variables in your model. Just force convert them to basic python types and you'll be fine. (As shown in linked post.)
If you want to persist with the way it is now, you should pull the equation out of your constraint and tinker with it after defining all the variables and ensure that it evaluates to a pyomo expression. When the variable is a singleton, order matters when interfacing with numpy arrays. Yes, this is ugly. So strongly recommend either indexing all variables or converting everything to basic types, which is easy.
In [57]: import numpy as np
In [58]: from pyomo.environ import *
In [59]: m = ConcreteModel()
In [60]: s = Set(initialize=[1,2,3])
In [61]: m.x = Var(s)
In [62]: m.y = Var() # a non-indexed variable!
In [63]: temp = np.array([1.5, 2.5])
In [64]: type(m.x[1] * temp[1])
Out[64]: pyomo.core.expr.numeric_expr.MonomialTermExpression
In [65]: type(temp[1] * m.x[1])
Out[65]: pyomo.core.expr.numeric_expr.MonomialTermExpression
In [66]: # All is good.... these above evaluated to pyomo expressions
In [67]: type(m.y * temp[1])
Out[67]: pyomo.core.expr.numeric_expr.MonomialTermExpression
In [68]: type(temp[1] * m.y)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-68-e1ff132af45b> in <module>
----> 1 type(temp[1] * m.y)
TypeError: unsupported operand type(s) for *: 'float' and 'NoneType'
In [69]: type(temp[1] * Var())
Out[69]: numpy.ndarray
In [70]: # noted that when the variable is not constructed in a model, the behavior evaluates to an ndarray

How to use supported numpy and math functions with CUDA in Python?

According to numba 0.51.2 documentation, CUDA Python supports several math functions. However, it doesn't work in the following kernel function:
#cuda.jit
def find_angle(angles):
i, j = cuda.grid(2)
if i < angles.shape[0] and j < angles.shape[1]:
angles[i][j] = math.atan2(j, i)
The output:
numba.core.errors.LoweringError: Failed in nopython mode pipeline (step: nopython mode backend)
No definition for lowering <built-in function atan2>(int64, int64) -> float64
Am I using the function incorrectly?

The hint to the source of the problem is here:
No definition for lowering <built-in function atan2>(int64, int64) -> float64
The arguments returned by cuda.grid() (i.e. i, j which you are passing to atan2) are integer values because they are related to indexing.
numba can't find a version of atan2 that it can use that takes two integer arguments and returns a floating-point value:
float64 = atan2(int64, int64)
One possible solution is to convert your atan2 input arguments to match the type that numba seems to want to return from that function, which is evidently float64:
from numba import cuda, float64
import numpy
import math
#cuda.jit
def find_angle(angles):
i, j = cuda.grid(2)
if i < angles.shape[0] and j < angles.shape[1]:
angles[i][j] = math.atan2(float64(j), float64(i))
block_x = 32
block_y = 32
block = (block_x, block_y)
x = 256
y = 256
grid = (x//block_x, y//block_y) # not for arbitrary x and y
angles = numpy.ones((x, y), numpy.float64)
find_angle[grid, block](angles)

Working and manipulating numpy arrays with numba

Why can't Numba's jit compile a simple Numpy array operation?
Here is a minimal non-working example that reproduces Numba's failure to compile
import numpy as np
from numba import jit
rows = 10
columns = 999999
A = np.empty((rows, columns))
b = np.linspace(0, 1, num=rows)
#jit(nopython=True)
def replicate(A, b):
for i in range(A.shape[1]):
A[:, i] = b
return A #optional
replicate(a, b)
With the following error:
TypingError: Failed at nopython (nopython frontend)
Cannot resolve setitem: array(float64, 1d, C, nonconst)[(slice3_type, int64)] = array(float64, 1d, C, nonconst)
File "<ipython-input-32-db24fbe2922f>", line 12
Am I doing something wrong?
As an aside, I need nopython mode because in my real-life situation I need to perform array addition, multiplication with scalars and array populating with other arrays frequently. and my understanding is that in object mode I won't be able to do loop jitting and thus I won't see any real performance boost on the execution.

Numba doesn't support numpy slicing in nopython mode. Try unrolling the loops explicitly:
rows = 10
columns = 999999
a = np.empty((rows, columns))
b = np.linspace(0, 1, num=rows)
#jit(nopython=True)
def replicate(A, b):
for i in xrange(A.shape[0]):
for j in xrange(A.shape[1]):
A[i, j] = b[i]
return A #optional
replicate(a, b)