I'm in the process of developing a PDE package (similar to fenics) and am trying to come up with the best way for a user to define their functions. The current method I am using is fine. However, as the package grows the run-time compilation of the functions will take much longer.
The current method takes in a dictionary of function specs and redefines and recompiles the global default. As the package gets larger, it would be ideal if I could have everything compiled ahead of time; the problem is for numba's pycc AOT compilation, you can't pass in a function because there is no recognizable signature type (that I could find in the documentaion). I have also tried using cython but the function stays as a weak reference to a pyobject which ruins the performance.
The code in the module where the function exists looks like
import numpy as np
from numba import njit
#njit
def _user_defined_function(z, t):
return -z
def define_vector_func(func_specs, **numba_kwargs):
# func_specs is a dict, numba_kwargs are for jit decorator
# create a string to be compiled into a new function
exprs = []
for var in func_specs:
s = func_specs[var]
for i, (variable, expression) in enumerate(func_specs.items()):
s = s.replace(variable, f'z[{i}]')
exprs.append(s)
fp = '<ipython-cache-safe>' # needed to allow for caching
src = ', '.join(exprs)
src = f'def local_func(z, t): return np.array([{src}])\n'
lines = [src]
linecache.cache[fp] = (len(src), None, lines, fp)
code = compile(src, fp, 'exec')
exec(code)
# redefine global function and return a reference if user wants to use it
local_func_ptr = vars()['local_func']
local_func_ptr(np.ones(len(func_specs)), 0.0)
global _user_defined_function
_user_defined_function = njit(**numba_kwargs)(local_func_ptr)
_rec() # recompile other functions to recognize new function
return _user_defined_function
# rest of module
# ...
To use it from a separate module, you would do something like
from module_above import define_vector_func, other_funcs_from_module
if __name__ == '__main__':
func_spec = {'v': 'v - (v * v * v) / 3.0 - w + 0.08',
'w': '0.08 * (v - 0.8 * w + 0.7)'}
f = define_vector_func(func_spec, fastmath=True, cache=True)
# do what user wants
What would be the best route for compiling the module ahead of time and maintain performance?
For example, in my folder, I have my ipython notebook "program.ipynb" and a python file "functions.py" which has some functions in it, for example, "func"
from numpy import sqrt
def func(x):
return N + sqrt(x)
that is going to be used in "program.ipynb" which looks like that
from functions import func
N = 5
func(2)
--> name 'N' is not defined
To fix the bug i need to define the variable N in my functions.py file but isn't there a way around? I want to define all my global variables in my main programm (program.ipynb).
You can't access a variable like that, the best way would be:
functions.py
from numpy import sqrt
def func(x, N):
return N + sqrt(x)
program.ipynb
from functions import func
N = 5
func(2, N)
I'm trying to initialize two vectors in memory using gsl_vector_set(). In the main code it is initialized to zero on default, but I wanted to initialize them to some non-zero value. I made a test code based on a working function that uses the gsl_vector_set() function.
from ctypes import *;
gsl = cdll.LoadLibrary('libgsl-0.dll');
gsl.gsl_vector_get.restype = c_double;
gsl.gsl_matrix_get.restype = c_double;
gsl.gsl_vector_set.restype = c_double;
foo = dict(
x_ht = [0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,
0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0],
x_ht_m = [0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,
0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0]
);
for f in range(0,18):
gsl.gsl_vector_set(foo['x_ht_m'],f,c_double(1.0));
gsl.gsl_vector_set(foo['x_ht'],f,c_double(1.0));
When I run the code I get this error.
ArgumentError: argument 1: <type 'exceptions.TypeError'>: Don't know how to convert parameter 1
I'm new to using ctypes and gsl functions so I'm not sure what the issue is or what the error message means. I an also not sure if there is a better way that I should be trying to save a vector to memory
Thank you #CristiFati for pointing out that I needed gsl_vector_calloc in my test code. I noticed that in the main code I was working in that the vector I needed to set was
NAV.KF_dictnry['x_hat_m']
instead of
NAV.KF_dictnry['x_ht_m']
So I fixed the test code to mirror the real code a bit better by creating a class holding the dictionary, and added the ability to change each value in the vector to an arbitrary value.
from ctypes import *;
gsl = cdll.LoadLibrary('libgsl-0.dll');
gsl.gsl_vector_get.restype = c_double;
gsl.gsl_matrix_get.restype = c_double;
gsl.gsl_vector_set.restype = c_double;
class foo(object):
fu = dict(
x_hat = gsl.gsl_vector_calloc(c_size_t(18)),
x_hat_m = gsl.gsl_vector_calloc(c_size_t(18)),
);
x_ht = [1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,
1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0]
x_ht_m = [1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,
1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0]
for f in range(0,18):
gsl.gsl_vector_set(foo.fu['x_hat_m'],f,c_double(x_ht_m[f]));
gsl.gsl_vector_set(foo.fu['x_hat'],f,c_double(x_ht[f]));
After running I checked with:
gsl.gsl_vector_get(foo.fu['x_hat_m'],0)
and got out a 1.0 (worked for the entire vector).
Turned out to just be some stupid mistakes on my end.
Thanks again!
As example, my first module is
from sympy import *
x,y=symbols('x y')
def A():
equation=2*x+y
return equation
print(A())
the output is
2*x + y
my second module is
def B(x,y):
equation=2*x + y
return equation
I have to copy past the output of first module to second one each time from the terminal. Is there a way to pass this output automatically to the source code of B(x,y)?
Thank you for your attention
Lets say I have the following config file:
config.py:
x = 2
y = x * 2
I would like to import this in the file main.py, preferably using load_source command, but I also want to be able to change the value of x at the time of import such that the change in x propagates to the other variables in the config.py. For example, I want the following code, prints 6 and not 4.
main.py:
import imp
config = imp.load_source('', 'config.py')
config.x = 3
print config.y
What is the best way to do that? I know I can write functions in config.py to do this for me, but I prefer the config to be simple variable definitions only.
Put the code into a class:
class Config(object):
def __init__(self, x=2):
self.x = x
self.y = x * 2
Then, in your main program:
c = Config(3)
print c.y