We Keep Coding

access a class instance/variable from one running python file in another?

I have a problem that involves collecting data continuously from multiple sources.
My setup as it is currently, writes each data entry from each source to a MySQL db, and then, with another python program, does Select's that bring all the data together. I need to make INSERT's at roughly 1000/second and as it is my SELECT's can take 15-20 seconds each.
The whole process takes so long the data is obsolete before I get to do anything useful with it.
I have created a toy example to try and demonstrate what I am looking for.
program 1 'generateClasses':
import time
import random
from datetime import datetime
class Race:
def __init__(self,name):
hist = {}
now = datetime.now()
self.name = name
self.now = now
hist[now] = 0
self.v = 0
self.hist = hist # example variable's.
def update(self,name,v):
now = datetime.now()
hist = self.hist
hist[now] = v
self.v = v
self.now - now
self.hist = hist
class Looper:
def __init__(self,name):
self.a = Race(name)
def loop(self,name):
# simulating the streaming API
while True:
v = self.a.v
v += 1
self.a.update(name,v)
print(a,datetime.now(),v) # can i access this stream using the location displayed with the print(a)?
time.sleep(0.1) # this should be more like time.sleep(0.001)
def pickData(self,name):
v = self.v
self.loop(name)
print('The state at {} {} = '.format(self.now,self.v))
return self.hist
if __name__ == "__main__":
x = 'Some_ID'
a = Looper(x)
a.loop(x)
program 2:
from generateClasses import Looper
from datetime import datetime
import time
start_time = int((datetime.now() - datetime(1970, 1, 1)).total_seconds())
print(start_time)
x = 'Some_orher_ID'
a = Looper(x)
print('this will print')
a.loop(x)
print('this wont ever print')
a.pickdata(x)
# this last section is the functionality i am looking for in this program, but, as it is, it will never run.
x = ‘Some_ID’
while True:
now_time = int((datetime.now() - datetime(1970, 1, 1)).total_seconds())
print(start_time)
if int(now_time-start_time) == 10:
a.pickData(x)
# b.pickData(x)
# c.pickData(x)
# d.pickData(x)
# make further actions.....
What happens currently in my examples is that it creates its own loop using the class structure from the first program.
What I want it to do is call the the pickData() method from program 2 at timely intervals of my choosing on a loop running in another program.
Is my best option picking a db located in memory and getting a faster computer?
Maybe something can be done with the object location shown when you print the instance name?
I have uploaded to github if anybody fancies it..
I would be grateful of any suggestions.
also, recommendations for further reading would be appreciated also.

Python Classes and inheritance. Calling super method returns Error

I have the following python code. The code analyses a digital signal for amplification processing however I am having trouble applying inheritance concepts to this for some reason.
I have a parent class and a child class, which sets the parent class as a parameter to inherit its properties in the child class.
But when I try accessing a method within the parent class using OOP inheritance principles I am getting an error. I originally tried extending the class and not overloading it, but when I tried to access the dofilter() method in the extended parent class, it would throw an error. However, if I try overloading the dofilter() method in the child class and then use "super" to call it from the parent class, there is no error, but returns "NaN", which clearly means I am doing something wrong still. I know data is being passed to the objects, there should be no reason why it returns NaN. It has left me at a little bit of an impass now. Can anyone please explain why this is not working?
What I have tried/my summarized script, to paint a better picture of the issue:
# -*- coding: utf-8 -*-
"""
Spyder Editor
This is a temporary script file.
"""
import numpy as np
import matplotlib.pyplot as plt
# =============================================================================
# CLASSES
# =============================================================================
class fir_filter:
def __init__(self, coefficients):
self.coefficients = coefficients
self.buffer = np.zeros(taps)
def dofilter(self, value, offset):
result = 0
#Splice coefficients and buffer arrays into smaller arrays
buffer_newest = self.buffer[0:offset+1]
buffer_oldest = self.buffer[offset+1:taps]
coefficients1 = self.coefficients[0:offset+1]
coefficients2 = self.coefficients[offset+1:taps]
#Accumulate
self.buffer[offset] = value
for i in range(len(coefficients1)):
result += coefficients1[i]*buffer_newest[offset-1-i]
for i in range(len(coefficients2), 0, -1):
result += coefficients2[len(coefficients2)-i] * buffer_oldest[i-1]
return result
class matched_filter(fir_filter):
def __init__(self,coefficients):
self.coefficients = coefficients
def dofilter(self,value,offset):
super().dofilter(value, offset)
return result
########################################
#START OF MAIN SCRIPT
########################################
#...
#REMOVED- import data files, create vars, perform various calculations.
########################################
#... RESUME CODE pertinent variables here
m_wavelet = (2/(np.sqrt(3*a*(np.pi**(1/4)))))*(1-((n/a)**2))*np.exp((-n**2)/(2*(a**2)))
m_wavelet = m_wavelet [::-1]
result = np.zeros(l)
offset = 0
plt.figure(6)
plt.plot(m_wavelet)
plt.plot(ecg_3[8100:8800]/len(ecg_3)/300)
########################################
#instantiated object here, no errors thrown
########################################
new_filter = matched_filter(m_wavelet)
########################################
#issue below
########################################
for k in range(len(ecg_3)):
result[k] = new_filter.dofilter(ecg_3[k], offset) #<- Every item in the array is "Nan"
offset += 1
if (offset == 2000):
offset = 0
########################################
#More removed code/end of script
########################################
Important bits:
Parent class:
class fir_filter:
def __init__(self, coefficients):
self.coefficients = coefficients
self.buffer = np.zeros(taps)
def dofilter(self, value, offset):
result = 0
#Splice coefficients and buffer arrays into smaller arrays
buffer_newest = self.buffer[0:offset+1]
buffer_oldest = self.buffer[offset+1:taps]
coefficients1 = self.coefficients[0:offset+1]
coefficients2 = self.coefficients[offset+1:taps]
#Accumulate
self.buffer[offset] = value
for i in range(len(coefficients1)):
result += coefficients1[i]*buffer_newest[offset-1-i]
for i in range(len(coefficients2), 0, -1):
result += coefficients2[len(coefficients2)-i] * buffer_oldest[i-1]
return result
Child Class:
class matched_filter(fir_filter):
def __init__(self,coefficients):
self.coefficients = coefficients
def dofilter(self,value,offset):
super().dofilter(value, offset)
return result
Code and issue:
new_filter = matched_filter(m_wavelet)
for k in range(len(ecg_3)):
result[k] = new_filter.dofilter(ecg_3[k], offset)
How can I access "dofilter()", in "fir_filter", which is inherited in "matched_filter"?
Any insight would be much appreciated. I can provide more code if necessary.

Class implementation in Python

I'm trying to create a class in Python and having some issues.
First I tried inheriting a VectorSystem to do trajectory optimization and I get the error regarding it not having 'AutoDiff'
RuntimeError: The object named [] of type
drake::pydrake::(anonymous)::Impl::PyVectorSystem does not
support ToAutoDiffXd
Code:
import numpy as np
from pydrake.systems.framework import VectorSystem
from pydrake.all import (DirectCollocation, PiecewisePolynomial, Solve)
# Define the system.
class ex1(VectorSystem):
def __init__(self):
VectorSystem.__init__(self,
1, # 1 input.
2) # 2 outputs.
self.DeclareContinuousState(2) # 2 state variable.
# xdot(t) = -x(t) - y(t); ydot(t) = -y(t) - x(t) + u
def DoCalcVectorTimeDerivatives(self, context, u, x, xdot):
xdot[:] = np.array([-x[0] - x[1], -x[1] - x[0] + u[0]])#.reshape(3,1) #u[0]
# y(t) = x(t)
def DoCalcVectorOutput(self, context, u, x, y):
y[:] = x
def runDircol(self, x0, xf, tf0):
N = 11
umax = 10.
context = self.CreateDefaultContext()
dircol = DirectCollocation(self, context, num_time_samples=N,
minimum_timestep=0.1, maximum_timestep=1.0)
u = dircol.input()
dircol.AddEqualTimeIntervalsConstraints()
dircol.AddConstraintToAllKnotPoints(u[0] <= .5*umax)
dircol.AddConstraintToAllKnotPoints(u[0] >= -.5*umax)
dircol.AddBoundingBoxConstraint(x0, x0, dircol.initial_state())
dircol.AddBoundingBoxConstraint(xf, xf, dircol.final_state())
R = 10.0 # Cost on input "effort".
dircol.AddRunningCost(R*u[0]**2)
# Add a final cost equal to the total duration.
dircol.AddFinalCost(dircol.time())
initial_x_trajectory = \
PiecewisePolynomial.FirstOrderHold([0., tf0], np.column_stack((x0, xf)))
dircol.SetInitialTrajectory(PiecewisePolynomial(), initial_x_trajectory)
result = Solve(dircol)
print(result.get_solver_id().name())
print(result.get_solution_result())
assert(result.is_success())
#import pdb; pdb.set_trace()
xtraj = dircol.ReconstructStateTrajectory(result)
utraj = dircol.ReconstructInputTrajectory(result)
return utraj,xtraj
if __name__ == "__main__":
# Declare model
plant = ex1() # Default instantiation
# Trajectory optimization
x0 = (0.0,0.0) #Initial state that trajectory should start from
xf = (1.0,1.0) #Final desired state
tf0 = 0.5 # Guess for how long trajectory should take
utraj, xtraj = plant.runDircol(x0, xf, tf0)
Second, I tried to inherit from the LeafSystem and had issues due to the templates. I cannot create a context by using plant.CreateDefaultContext(). I get the error:
TypeError: unbound method CreateDefaultContext() must be called with
ex1_[float] instance as first argument (got nothing instead)
And if I use plant().CreateDefaultContext() I get weird errors afterwards like getting wrong context.num_output_ports() or not being able to call plant.ToSymbolic()
(TypeError: unbound method ToSymbolic() must be called with ex1_[float] instance as first argument (got nothing instead)) etc ...
Code:
import numpy as np
from pydrake.all import LeafSystem_
from pydrake.systems.scalar_conversion import TemplateSystem
#TemplateSystem.define("ex1_")
def ex1_(T):
class Impl(LeafSystem_[T]):
def _construct(self, converter=None):
LeafSystem_[T].__init__(self, converter)
# one inputs
self.DeclareVectorInputPort("u", BasicVector_[T](1))
# two outputs (full state)
self.DeclareVectorOutputPort("x", BasicVector_[T](2), self.CopyStateOut)
# two positions, no velocities
self.DeclareContinuousState(2, 0, 0)
def _construct_copy(self, other, converter=None):
Impl._construct(self, converter=converter)
def CopyStateOut(self, context, output):
x = context.get_continuous_state_vector().CopyToVector()
output.SetFromVector(x) # = y
def DoCalcTimeDerivatives(self, context, derivatives):
x = context.get_continuous_state_vector().CopyToVector()
u = self.EvalVectorInput(context, 0).CopyToVector()
xdot[:] = np.array([-x[0] - x[1], -x[1] - x[0] + u[0]]) #.reshape(3,1) #u[0]
derivatives.get_mutable_vector().SetFromVector(xdot)
return Impl
if __name__ == "__main__":
# Declare model
plant = ex1_[None] # Default instantiation
#context = plant.CreateDefaultContext(DubinsPlant_[None]())
context = plant().CreateDefaultContext()
import pdb; pdb.set_trace()
sym_system = plant.ToSymbolic()
Would appreciate any help on solving one of these issues.
(Running on Ubuntu 16.04)

To answer your second question, plant is not an instantiation of ex1_[None]. So plant.ToSymbolic() will not work. A workable solution would be:
if __name__ == "__main__":
# Declare model
ex1 = ex1_[None]
plant = ex1()
context = plant.CreateDefaultContext()
ad_system = plant.ToAutoDiffXd()
sym_system = plant.ToSymbolic()

To answer your first question, I've unfortunately not updated VectorSystem to support subclassed type conversion:
https://github.com/RobotLocomotion/drake/issues/10745
Let me try that out in the next few minutes.
EDIT: Ah, may be more complicated. Please see update in the issue.

Python: Name not defined error even though function is clearly defined before call [duplicate]

This question already has answers here:
function name is undefined in python class [duplicate]
(2 answers)
Closed 5 years ago.
I'm new to python and having a weird issue with function definitions. I have checked around the forums and made sure to define my function before calling it, however that has not helped the issue. I keep getting a name not defined error when I try to call literally function in this one particular method.
from eight_puzzle import Puzzle
import math
################################################################
### Node class and helper functions provided for your convience.
### DO NOT EDIT!
################################################################
class Node:
"""
A class representing a node.
- 'state' holds the state of the node.
- 'parent' points to the node's parent.
- 'action' is the action taken by the parent to produce this node.
- 'path_cost' is the cost of the path from the root to this node.
"""
def __init__(self, state, parent, action, path_cost):
self.state = state
self.parent = parent
self.action = action
self.path_cost = path_cost
def gen_child(self, problem, action):
"""
Returns the child node resulting from applying 'action' to this node.
"""
return Node(state=problem.transitions(self.state, action),
parent=self,
action=action,
path_cost=self.path_cost + problem.step_cost(self.state, action))
#property
def state_hashed(self):
"""
Produces a hashed representation of the node's state for easy
lookup in a python 'set'.
"""
return hash(str(self.state))
################################################################
### Node class and helper functions provided for your convience.
### DO NOT EDIT!
################################################################
def retrieve_solution(node,num_explored,num_generated):
"""
Returns the list of actions and the list of states on the
path to the given goal_state node. Also returns the number
of nodes explored and generated.
"""
actions = []
states = []
while node.parent is not None:
actions += [node.action]
states += [node.state]
node = node.parent
states += [node.state]
return actions[::-1], states[::-1], num_explored, num_generated
################################################################
### Node class and helper functions provided for your convience.
### DO NOT EDIT!
################################################################
def print_solution(solution):
"""
Prints out the path from the initial state to the goal given
a tuple of (actions,states) corresponding to the solution.
"""
actions, states, num_explored, num_generated = solution
print('Start')
for step in range(len(actions)):
print(puzzle.board_str(states[step]))
print()
print(actions[step])
print()
print('Goal')
print(puzzle.board_str(states[-1]))
print()
print('Number of steps: {:d}'.format(len(actions)))
print('Nodes explored: {:d}'.format(num_explored))
print('Nodes generated: {:d}'.format(num_generated))
################################################################
### Skeleton code for your Astar implementation. Fill in here.
################################################################
class Astar:
"""
A* search.
- 'problem' is a Puzzle instance.
"""
def __init__(self, problem):
self.problem = problem
self.init_state = problem.init_state
self.num_explored = 0
self.num_generated = 1
def selectState(self, listOfStates):
'''
Selects the loweset cost node for expansion based on f(n) = g(n) + h(n)
'''
lowestCostPath = listOfStates[0].path_cost
index = int(1)
lowestNodeIndex = int(0)
while index != len(listOfStates):
scannedPathCost = listOfStates[index].path_cost
if index < scannedPathCost:
lowestCostPath = scannedPathCost
lowestNodeIndex = index
index += 1
return listOfStates[lowestNodeIndex]
def f(self,node, method):
'''
Returns a lower bound estimate on the cost from root through node
to the goal.
'''
return node.path_cost + self.h(node, method)
def getManhattanDistance(self, node):
'''
Evaluates the manhattan distance for a given state
'''
iterator = int(0)
misplacedCount = int(0)
totalDistance = int(0)
while iterator != len(node.state):
if iterator != node.state[iterator] and node.state[iterator] != 0:
misplacedCount = misplacedCount + 1
xCurrent = int(node.state[iterator]/3)
yCurrent = int(node.state[iterator]%3)
xDesired = int(iterator/3)
yDesired = int(iterator%3)
totalDistance = totalDistance + int(abs(xCurrent - xDesired)) + int(abs(yCurrent - yDesired))
iterator = iterator + 1
return totalDistance + misplacedCount
def h(self,node, method='man'):
'''
Returns a lower bound estimate on the cost from node to the goal
using the different heuristics.
'''
################################################################
### Your code here.
################################################################
if method == 'man':
self.getManhattanDistance(node)
return -1
elif method == 'rowcol':
return -1 # compute rowcol heuristic
elif method == 'misplaced':
return -1 # compute misplaced tiles the number of tiles out of place
elif method == 'null':
return -1 # compute null heuristic
else:
return 0
def method_stats(self, board, trials=100, method='man'):
'''
Returns an mean and standard deviation of the number of nodes expanded
'''
# write code here to randomly generate puzzles and
# compute the mean and standard deviation of the number
# nodes expanded. You can use np.mean() and np.std()
expanded_mean = 0.
expanded_std = 0.
for t in range(trials):
puzzle = Puzzle(board).shuffle()
solver = Astar(puzzle)
actions, states, num_explored, num_generated = solver.solve(method=method)
############################################################
### Compute upper bound for branching factor and update b_hi
### Your code here.
############################################################
return expanded_mean, expanded_std
def anotherFunction(self, node, method):
return 1
def generateStatesFor(self, node, method, listOfStates):
'''
Decides how to select an action from a list of available actions
'''
def solve(self, method='man'):
node = Node(state = self.init_state,
parent = None,
action = None,
path_cost = 0)
num_explored = int(0)
num_generated = int(0)
listOfStates = []
listOfStates.append(node)
print(listOfStates[0].state)
anotherFunction(self, node, method)
return retrieve_solution(node, num_explored=num_explored, num_generated=num_generated)
if __name__ == '__main__':
# Simple puzzle test
## board = [[3,1,2],
## [4,0,5],
## [6,7,8]]
board = [[7,2,4],
[5,0,6],
[8,3,1]]
puzzle = Puzzle(board)
solver = Astar(puzzle)
solution = solver.solve()
##print_solution(solution)
# Harder puzzle test
board = [[7,2,4],
[5,0,6],
[8,3,1]]
puzzle = Puzzle(board)
solver = Astar(puzzle)
##solution = solver.solve()
##print(len(solution[0]))
# branching factor test
method='man'
emean, estd = solver.method_stats(board, trials=100, method=method)
##print('mean and standard deviation: {0:.2f}, {1:.2f} using heuristic: {2}'.format(emean, estd, method))
The Error code:
Traceback (most recent call last): File "/Users/-/Downloads/HW1 2/code/my_hw1.py", line 214, in <module>
solution = solver.solve() File "/Users/-/Downloads/HW1 2/code/my_hw1.py", line 200, in solve
anotherFunction(self, node, method) NameError: name 'anotherFunction' is not defined [Finished in 0.1s with exit code 1]
As you can see, the function calling it is on line 200 and the function is defined at line 185. Any idea what the issue could be? I am also able to call the exact same "anotherFunction" method from other methods which aren't solve. Any tips would be appreciated.

When you define a function with "self" as an argument, you need to call that function from the class in which it is defined. For example, if you have an instance of the class myClass where anotherFunction is defined, the syntax would be myClass.anotherFunction(node, method). The "self" argument in the definition indicates anotherFunction is a member function of whatever class it is defined in - I would need to see more of your code to know what class that is.

Running functions defined within classes

I have a python class which houses some info. I have another file which some of these functions refer to. My get_date , is working fine however, none of my other functions seem to be working. I am getting the error AttributeError: PVData instance has no attribute 'time' when calling the time function.
class PVData:
def __init__(self):
self.date = yesterday()
self.data = load_data(self.date)
def change_date(self, date):
if self.date != date:
self.date = date
## self.refresh()
else:
self.date = date
self.date = load_data(self.date)
#time, temp, sun
self.time = []
self.temperature = []
self.sunlight = []
for minute in self.date:
self.time.append(minute[0])
self.temperature.append(minute[1])
self.sunlight.append(minute[2])
#power
self.dictonary[a] = []
for a in ARRAYS:
self.dictionary[ARRAYS[i]].append(power)
def get_date(self):
return self.date
def get_time(self, time_index):
return self.time[time_index]
def get_temperature(self):
return self.temperature
def get_sunlight(self):
return self.sunlight
def get_power(self, array):
return self.dictionary[array]
pvd = PVData()
The load_data function is (in another file):
def load_data(dateStr):
text = get_data_for_date(dateStr)
data = []
for line in text.splitlines():
time, temp, sun, powerStr = line.split(',', 3)
power = []
for p in powerStr.split(','):
power.append(int(p))
data.append((time, float(temp), float(sun), tuple(power)))
return data
which returns something such as:
[('19:00', 20.0, 0.0, (0, 0, 0, 0, 0, 0, 0, 21, 31, 52)), (tuple 2), etc etc]
The error seems to be resulting because time is not a valid parameter for self, but I thought that was defined where self.time = [].
Excuse my lack of knowledge, python is quite new to me. Any ideas of why this is not doing as required?

Move time as well as other variables that should be accessible from outside to def init(self). Please keep in mind, that python creates variables in runtime, so if you want variable has been accessible in any place - they should be created on class initialization.
Added:
From your code it looks like, you should move temperature and sunlight to def init(self) as well.