We Keep Coding

Function inside class is not working when called upon (python)

I am creating simulation library, of which one is using geometric geometric brownian motion (gbm). I have created a class for it, which has a function update to update variable values. Here is the code for gbm class:
import numpy as np
from sn_random_numbers_gen import sn_random_numbers
from generic_simulation_class import simulation_class
class geometric_brownian_motion(simulation_class):
#class to generate simiulated paths usinig gbm
# attriibutes: name, mar_env, corr
#methods: update(to update parameters), generate_paths
def __init__(self, name, mar_env, corr=False):
super().__init__(name, mar_env, corr)
def update(self, initial_value = None, volatility=None, final_date=None):
if initial_value is not None:
self.initial_value = initial_value
if volatility is not None:
self.volatility = volatility
if final_date is not None:
self.final_date = final_date
def generate_paths(self, fixed_seed = False, day_count = 365):
if self.time_grid is None:
self.generate_time_grid()
M = len(self.time_grid)
J = self.paths
paths = np.zeros((M,J))
paths[0] = self.initial_value
if not self.correlated:
rand = sn_random_numbers((1,M,J), fixed_seed=fixed_seed)
else:
rand = self.random_numbers
short_rate = self.discount_curve.short_rate
for t in range(1, len(self.time_grid)):
if not self.correlated:
ran = rand[t]
else:
ran = np.dot(self.cholesky_matrix, rand[:, t, :])
ran = ran[self.rn_set]
dt = (self.time_grid[t]-self.time_grid[t-1]).days/day_count
paths[t] = paths[t-1]*np.exp((short_rate-0.5*self.volatility**2)*dt + self.volatility*np.sqrt(dt)*ran)
self.instrument_values = paths
Here is a use case of the class (this use case also calls upon some other pre-defined classes by me) :
import datetime as dt
from dx_frame import *
me_gbm = market_environment('me_gbm', dt.datetime(2020,1,1))
me_gbm.add_constant('initial_value', 36)
me_gbm.add_constant('volatility', 0.1)
me_gbm.add_constant('final_date', dt.datetime(2020,12,31))
me_gbm.add_constant('currency', 'EUR')
me_gbm.add_constant('frequency', 'M')
me_gbm.add_constant('paths', 10000)
csr = constant_short_rate('csr', 0.05)
me_gbm.add_curve('discount_curve', csr)
gbm = geometric_brownian_motion('gbm', me_gbm)
gbm.generate_time_grid()
paths_1 = gbm.get_instrument_values()
gbm.update(volatility=0.5)
paths_2 = gbm.get_instrument_values()
Here update function is called to change value of variable volatility. But, the variable remains the same. As the value of both paths_1 and paths_2 is same(I have enclosed imiage of paths_1 and paths_2 for reference). Every other function works fine. Can you please help me understand the problem ?

The update function is alright, the problem is with get_instrument_values, as the paths are not none it does not update and returns the same path.
Using fixed_seed = False solves the problem

access a class instance/variable from one running python file in another?

I have a problem that involves collecting data continuously from multiple sources.
My setup as it is currently, writes each data entry from each source to a MySQL db, and then, with another python program, does Select's that bring all the data together. I need to make INSERT's at roughly 1000/second and as it is my SELECT's can take 15-20 seconds each.
The whole process takes so long the data is obsolete before I get to do anything useful with it.
I have created a toy example to try and demonstrate what I am looking for.
program 1 'generateClasses':
import time
import random
from datetime import datetime
class Race:
def __init__(self,name):
hist = {}
now = datetime.now()
self.name = name
self.now = now
hist[now] = 0
self.v = 0
self.hist = hist # example variable's.
def update(self,name,v):
now = datetime.now()
hist = self.hist
hist[now] = v
self.v = v
self.now - now
self.hist = hist
class Looper:
def __init__(self,name):
self.a = Race(name)
def loop(self,name):
# simulating the streaming API
while True:
v = self.a.v
v += 1
self.a.update(name,v)
print(a,datetime.now(),v) # can i access this stream using the location displayed with the print(a)?
time.sleep(0.1) # this should be more like time.sleep(0.001)
def pickData(self,name):
v = self.v
self.loop(name)
print('The state at {} {} = '.format(self.now,self.v))
return self.hist
if __name__ == "__main__":
x = 'Some_ID'
a = Looper(x)
a.loop(x)
program 2:
from generateClasses import Looper
from datetime import datetime
import time
start_time = int((datetime.now() - datetime(1970, 1, 1)).total_seconds())
print(start_time)
x = 'Some_orher_ID'
a = Looper(x)
print('this will print')
a.loop(x)
print('this wont ever print')
a.pickdata(x)
# this last section is the functionality i am looking for in this program, but, as it is, it will never run.
x = ‘Some_ID’
while True:
now_time = int((datetime.now() - datetime(1970, 1, 1)).total_seconds())
print(start_time)
if int(now_time-start_time) == 10:
a.pickData(x)
# b.pickData(x)
# c.pickData(x)
# d.pickData(x)
# make further actions.....
What happens currently in my examples is that it creates its own loop using the class structure from the first program.
What I want it to do is call the the pickData() method from program 2 at timely intervals of my choosing on a loop running in another program.
Is my best option picking a db located in memory and getting a faster computer?
Maybe something can be done with the object location shown when you print the instance name?
I have uploaded to github if anybody fancies it..
I would be grateful of any suggestions.
also, recommendations for further reading would be appreciated also.

Python: Passing an object instance from one function to another?

I'm having trouble with passing a value from one script to another, trying to take it a step at a time but the big picture would be to print the value obj1.get_predval to my Django view and wait for the users' input.
active_learner.obj1.get_predval in my beta.py script doesn't work, it just prints out the initial value which makes sense because it's not running the main.py but I'm not sure how I'd pass the value of obj1.set_predval(machine_prediction) from main.py. It properly outputs the obj1.get_predval in the main.py script.
I'm assuming I have a fundamental misunderstanding, for now, all I'm trying to return is the value of obj1.get_predval in function beta.py, when it gets to the line return value and wait for user input then continue.
main.py script below
obj1 = MachinePred()
def main():
model = load_model('model_new.h5')
DATAFILE = "c_user_model_data100000.csv"
dataset = loadtxt(DATAFILE, delimiter=",")
X_pool, Y = dataset[:, 0:5], dataset[:, 5:]
sc_x, sc_y = StandardScaler(), StandardScaler()
X_pool, Y = sc_x.fit_transform(X_pool), sc_y.fit_transform(Y)
learner = ActiveLearner(
estimator = model,
query_strategy = uncertainty_sampling
)
for i in range(3):
query_idx, query_inst = learner.query(X_pool)
print("The machine queried:\n{}\nat index {}".format(
sc_x.inverse_transform(query_inst),
query_idx
)
)
machine_prediction = learner.predict(X_pool[query_idx])
obj1.set_predval(machine_prediction)
print("predvalue:", (obj1.get_predval()))
ratings = []
cc_factor = ["delay", "speed", "missing_words", "paraphrasing"]
for f in cc_factor:
user_answer = input("How would you rate the quality of {} between [1-5]: ".format(f))
ratings.append(user_answer)
print(ratings, np.array([ratings]).reshape(1,-1))
if __name__ == '__main__':
main()
beta.py
This is the script I'm trying to pass the value to below
import active_learner
print(A is: ", active_learner.obj1.get_predval)
mac.py Simple python script using the get and set methods below.
class MachinePred:
predval = 0 # Default value of the 'x' configuration setting
def __init__(self):
self.predval = 0
def set_predval(self, val):
self.predval = val
def get_predval(self):
return self.predval

So the solution to this was very simple, from my understanding it could be done using generator-based coroutines or splitting it into two functions inside a class and use an OO design. The coroutine method would use"yield" which would force exit the function returning the value then re-enter the function but this would limit your ability to use non-generator based coroutines which I did need to await input from my front end.
Using a class though you could put the active learner model and data in an "init method" then split from the machine_prediction = learner.predict(X_pool[query_idx]) for the first function after returning the values and perform the rest in a second function.

Python Classes and inheritance. Calling super method returns Error

I have the following python code. The code analyses a digital signal for amplification processing however I am having trouble applying inheritance concepts to this for some reason.
I have a parent class and a child class, which sets the parent class as a parameter to inherit its properties in the child class.
But when I try accessing a method within the parent class using OOP inheritance principles I am getting an error. I originally tried extending the class and not overloading it, but when I tried to access the dofilter() method in the extended parent class, it would throw an error. However, if I try overloading the dofilter() method in the child class and then use "super" to call it from the parent class, there is no error, but returns "NaN", which clearly means I am doing something wrong still. I know data is being passed to the objects, there should be no reason why it returns NaN. It has left me at a little bit of an impass now. Can anyone please explain why this is not working?
What I have tried/my summarized script, to paint a better picture of the issue:
# -*- coding: utf-8 -*-
"""
Spyder Editor
This is a temporary script file.
"""
import numpy as np
import matplotlib.pyplot as plt
# =============================================================================
# CLASSES
# =============================================================================
class fir_filter:
def __init__(self, coefficients):
self.coefficients = coefficients
self.buffer = np.zeros(taps)
def dofilter(self, value, offset):
result = 0
#Splice coefficients and buffer arrays into smaller arrays
buffer_newest = self.buffer[0:offset+1]
buffer_oldest = self.buffer[offset+1:taps]
coefficients1 = self.coefficients[0:offset+1]
coefficients2 = self.coefficients[offset+1:taps]
#Accumulate
self.buffer[offset] = value
for i in range(len(coefficients1)):
result += coefficients1[i]*buffer_newest[offset-1-i]
for i in range(len(coefficients2), 0, -1):
result += coefficients2[len(coefficients2)-i] * buffer_oldest[i-1]
return result
class matched_filter(fir_filter):
def __init__(self,coefficients):
self.coefficients = coefficients
def dofilter(self,value,offset):
super().dofilter(value, offset)
return result
########################################
#START OF MAIN SCRIPT
########################################
#...
#REMOVED- import data files, create vars, perform various calculations.
########################################
#... RESUME CODE pertinent variables here
m_wavelet = (2/(np.sqrt(3*a*(np.pi**(1/4)))))*(1-((n/a)**2))*np.exp((-n**2)/(2*(a**2)))
m_wavelet = m_wavelet [::-1]
result = np.zeros(l)
offset = 0
plt.figure(6)
plt.plot(m_wavelet)
plt.plot(ecg_3[8100:8800]/len(ecg_3)/300)
########################################
#instantiated object here, no errors thrown
########################################
new_filter = matched_filter(m_wavelet)
########################################
#issue below
########################################
for k in range(len(ecg_3)):
result[k] = new_filter.dofilter(ecg_3[k], offset) #<- Every item in the array is "Nan"
offset += 1
if (offset == 2000):
offset = 0
########################################
#More removed code/end of script
########################################
Important bits:
Parent class:
class fir_filter:
def __init__(self, coefficients):
self.coefficients = coefficients
self.buffer = np.zeros(taps)
def dofilter(self, value, offset):
result = 0
#Splice coefficients and buffer arrays into smaller arrays
buffer_newest = self.buffer[0:offset+1]
buffer_oldest = self.buffer[offset+1:taps]
coefficients1 = self.coefficients[0:offset+1]
coefficients2 = self.coefficients[offset+1:taps]
#Accumulate
self.buffer[offset] = value
for i in range(len(coefficients1)):
result += coefficients1[i]*buffer_newest[offset-1-i]
for i in range(len(coefficients2), 0, -1):
result += coefficients2[len(coefficients2)-i] * buffer_oldest[i-1]
return result
Child Class:
class matched_filter(fir_filter):
def __init__(self,coefficients):
self.coefficients = coefficients
def dofilter(self,value,offset):
super().dofilter(value, offset)
return result
Code and issue:
new_filter = matched_filter(m_wavelet)
for k in range(len(ecg_3)):
result[k] = new_filter.dofilter(ecg_3[k], offset)
How can I access "dofilter()", in "fir_filter", which is inherited in "matched_filter"?
Any insight would be much appreciated. I can provide more code if necessary.

Using multiprocessing module in class

I have the following program and I want to use multiprocessing module. It uses external files, in which I call the PSO class from another file. costfunc is a function from another file and the other args are just variables.
Swarm is a list containing as much objects as the value of ps, and each object has multiple attributes which need to update at every iteration.
Following Hannu implemented multiprocessing.pool and it is working, however it is taking much more time than running sequentially.
Would appreciate if you could tell me what are the reasons for it happening and how can I make it run faster?
# IMPORT PACKAGES -----------------------------------------------------------+
import random
import numpy as np
# IMPORT FILES --------------------------------------------------------------+
from Reducer import initial
# Particle Class ------------------------------------------------------------+
class Particle:
def __init__(self,D,bounds_x,bounds_v):
self.Position_i = [] # particle position
self.Velocity_i = [] # particle velocity
self.Cost_i = -1 # cost individual
self.Position_Best_i = [] # best position individual
self.Cost_Best_i = -1 # best cost individual
self.Constraint_Best_i = [] # best cost individual contraints
self.Constraint_i = [] # constraints individual
self.Penalty_i = -1 # constraints individual
x0,v0 = initial(D,bounds_x,bounds_v)
for i in range(0,D):
self.Velocity_i.append(v0[i])
self.Position_i.append(x0[i])
# evaluate current fitness
def evaluate(self,costFunc,i):
self.Cost_i, self.Constraint_i,self.Penalty_i = costFunc(self.Position_i,i)
# check to see if the current position is an individual best
if self.Cost_i < self.Cost_Best_i or self.Cost_Best_i == -1:
self.Position_Best_i = self.Position_i
self.Cost_Best_i = self.Cost_i
self.Constraint_Best_i = self.Constraint_i
self.Penalty_Best_i = self.Penalty_i
return self
def proxy(gg, costf, i):
print(gg.evaluate(costf, i))
# Swarm Class ---------------------------------------------------------------+
class PSO():
def __init__(self,costFunc,bounds_x,bounds_v,ps,D,maxiter):
self.Cost_Best_g = -1 # Best Cost for Group
self.Position_Best_g = [] # Best Position for Group
self.Constraint_Best_g = []
self.Penalty_Best_g = -1
# Establish Swarm
Swarm = []
for i in range(0,ps):
Swarm.append(Particle(D,bounds_x,bounds_v))
# Begin optimization Loop
i = 1
self.Evol = []
while i <= maxiter:
pool = multiprocessing.Pool(processes = 4)
results = pool.map_async(partial(proxy, costf = costFunc, i=i), Swarm)
pool.close()
pool.join()
Swarm = results.get()
if Swarm[j].Cost_i< self.Cost_Best_g or self.Cost_Best_g == -1:
self.Position_Best_g = list(Swarm[j].Position_i)
self.Cost_Best_g = float(Swarm[j].Cost_i)
self.Constraint_Best_g = list(Swarm[j].Constraint_i)
self.Penalty_Best_g = float(Swarm[j].Penalty_i)
self.Evol.append(self.Cost_Best_g)
i += 1

You need a proxy function to do the function call, and as you need to deliver arguments to the function, you will need partial as well. Consider this:
from time import sleep
from multiprocessing import Pool
from functools import partial
class Foo:
def __init__(self, a):
self.a = a
self.b = None
def evaluate(self, CostFunction, i):
xyzzy = CostFunction(i)
sleep(0.01)
self.b = self.a*xyzzy
return self
def CostFunc(i):
return i*i
def proxy(gg, costf, i):
return gg.evaluate(costf, i)
def main():
Swarm = []
for i in range(0,10):
nc = Foo(i)
Swarm.append(nc)
p = Pool()
for i in range(100,102):
results = p.map_async(partial(proxy, costf=CostFunc, i=i), Swarm)
p.close()
p.join()
Swarm = []
for a in results.get():
Swarm.append(a)
for s in Swarm:
print (s.b)
main()
This creates a Swarm list of objects, and within each of these objects is evaluate that is the function you need to call. Then we have parameters (CostFunc and an integer as in your code).
We will now use Pool.map_async to map your Swarm list to your pool. This gives each worker one instance of Foo from your Swarm list, and we have a proxy function that actually calls then evaluate().
However, as apply_async only sends an object from the iterable to the function, instead of using proxy as the target function to pool, we use partial to create the target function to pass the "fixed" arguments.
And as you apparently want to get the modified objects back, this requires another trick. If you modify the target object in Pool process, it just modifies the local copy and throws it away as soon as the processing completes. There would be no way for the subprocess to modify main process memory anyway (or vice versa), this would cause a segmentation fault.
Instead, after modifying the object, we return self. When your pool has completed its work, we discard the old Swarm and reassemble it from the result objects.