We Keep Coding

I cant figure out how to send an array out of the optimization.minimize function

I am building a Python application for use in sientific computing.
The application is a Model Predicive Controler (MPC) and I am using scipy.optimize.minimize function as the optimization algorithm.
solution_guess = minimize(objectiveFunction,
U_guess,
arg_guess,
callback= None,
method = "SLSQP")
where the objective function is a self made function where a simulation of my system is executed.
it looks some like this:
def objectiveFunction(x,*arg):
U_test = x
dt_test = arg[0]
setpoint_test = arg[1]
pred_horizion_length_test = arg[2]
initStateValue_test = arg[3]
# Defining Model Arrays
NS_pred_horizion_test = int(pred_horizion_length_test/dt_test)+1
pred_horizion_array_test = np.linspace(0, pred_horizion_length_test, NS_pred_horizion_test)
SP_array_test = np.zeros(NS_pred_horizion_test) + setpoint_test
Y_array_test = SP_array_test * 0
# Defining parameters for the testing model
timeDelay_test = 50
initDelayValue_test = 0
K_test = 4
Tc1_test = 30
Tc2_test = 60
# Defining Model Object
obj_model_test = model.secDegModel(dt = dt_test,
K = K_test,
Tc1 = Tc1_test,
Tc2 = Tc2_test,
timeDelay = timeDelay_test,
initStateValue = initStateValue_test,
initDelayValue = initDelayValue_test
)
###########################################
#|||||||||||||||||||||||||||||||||||||||||#
# Testing Values for U on Model #
#|||||||||||||||||||||||||||||||||||||||||#
###########################################
# Running simulation of "real" model function
for k in range(NS_pred_horizion_test):
Y_array_test[k] = obj_model_test.run(u_k = U_test)
error = np.sum(abs(SP_array_test-Y_array_test))
return error
What i struggle with no is how to get back the Y_array_test array so I can plot it every time the optimization is done. I tryed using global variables but i did not get it to work nor do I think its good coding manner to use global variables. does any one know a nice way to solv my problem? maybe using callback function? (if callback is the way to go, I do not fully understand how this method works or how to implement it in a nice way)

Why don't you just do the following?
Modify your objectiveFunction as follows
from numpy import savez
def objectiveFunction(x,*arg):
.
.
.
.
.
# Running simulation of "real" model function
for k in range(NS_pred_horizion_test):
Y_array_test[k] = obj_model_test.run(u_k = U_test)
# Just save Y_array_test in a file
# Add some call_no if you don't want to overwrite
# then filename would be 'Y_array_test_' + str(call_no)
# You could increment this call_no, every time by
# call_no = call_no + 1
savez(file_name, Y_array_test)
# Then you could plot it outside using matplotlib
error = np.sum(abs(SP_array_test-Y_array_test))
return error

Based on your comments on my first answer and reusing some code from my other answer (This itself is written by modifying #Scott (Scott Sievert) answer and using his drawnow Github package)
Imp note:
I didn't install drawnow Github package . Instead I just copied drawnow.py into my folder. (This is because I didn't find any way to install it via conda. I didn't wan't to use PyPi)
Modify your code as follows
from numpy.random import random_sample
from numpy import arange, zeros
from drawnow import drawnow
from matplotlib import use
from matplotlib.pyplot import figure, axes, ion
from matplotlib import rcParams
from matplotlib.pyplot import style
from matplotlib.pyplot import cla, close
use("TkAgg")
pgf_with_rc_fonts = {"pgf.texsystem": "pdflatex"}
rcParams.update(pgf_with_rc_fonts)
style.use('seaborn-whitegrid')
scott_fig = figure() # call here instead!
ion()
# figure() # call here instead!
# ion() # enable interactivity
solution_guess = minimize(objectiveFunction,
U_guess,
arg_guess,
callback= None,
method = "SLSQP")
def objectiveFunction(x,*arg):
.
.
.
.
.
def draw_fig():
# can be arbitrarily complex; just to draw a figure
# figure() # don't call!
scott_ax = axes()
scott_ax.plot(x, y, '-g', label='label')
# Need to add some pause element here
# otherwise you won't be able to see the figure as
# it will change too fast
# cla()
# close(scott_fig)
# show() # don't call!
# Running simulation of "real" model function
for k in range(NS_pred_horizion_test):
Y_array_test[k] = obj_model_test.run(u_k = U_test)
# Just plot Y_array_test now that you have updated it
drawnow(draw_fig)
# Then you could plot it outside using matplotlib
error = np.sum(abs(SP_array_test-Y_array_test))
return error

Jupyter/Plotly - how to replace or update plot from add_trace?

Consider this Jupyter Python code, which uses Plotly:
import plotly.graph_objs as go
import numpy as np
from ipywidgets import widgets
from IPython.display import display
import random
mybutton = widgets.Button(description="Redraw")
xs = np.linspace(start=0, stop=10, num=100)
fig = go.FigureWidget( layout=go.Layout() )
# NB: function needs to be written in a way, that returns np.array for input np.array!
# or - use np.vectorize, to apply it element-by-element
def TestFunc(inval):
return inval+2*random.random()
fig.add_trace(go.Scatter(x=xs, y=np.vectorize(TestFunc)(xs),
mode='lines',
name='Test'))
def on_button_clicked(b):
fig.add_trace(go.Scatter(x=xs, y=np.vectorize(TestFunc)(xs),
mode='lines',
name='Test'))
mybutton.on_click(on_button_clicked)
widgets.VBox([mybutton, fig])
What I want to do, is redraw the function anew, whenever I click the button. However, since I use add_trace in the button callback, I get new traces added - I don't get the original one replaced:
So, my question is:
How do I obtain a reference to a "trace", added with add_trace, so that I could replace it? (say, fig.traces[0] = ...)
What is the best way to redraw the figure with a new retrace, with the minimal amount of object instantiation (I guess, I could do fig = go.FigureWidget( ... ) ... upon each button click, but that would have to recreate everything; I'd think, just recreating the y array, and triggering a redraw would be more "optimized")

OK, found something - still not sure if this is the way to do it, so if someone knows better, please post...
But anyways, fig.add_trace returns a reference that you can store in a variable; eventually that variable also contains the .x and .y arrays, and the .y array can be directly replaced, like so:
import plotly.graph_objs as go
import numpy as np
from ipywidgets import widgets
from IPython.display import display
import random
mybutton = widgets.Button(description="Redraw")
xs = np.linspace(start=0, stop=10, num=100)
fig = go.FigureWidget( layout=go.Layout() )
# NB: function needs to be written in a way, that returns np.array for input np.array!
# or - use np.vectorize, to apply it element-by-element
def TestFunc(inval):
return inval+2*random.random()
mytrace = fig.add_trace(go.Scatter(x=xs, y=np.vectorize(TestFunc)(xs),
mode='lines',
name='Test'))
print(repr(mytrace))
def on_button_clicked(b):
mytrace.data[0].y = np.vectorize(TestFunc)(xs)
mybutton.on_click(on_button_clicked)
widgets.VBox([mybutton, fig])
The above code works as intended - but I'm not yet sure whether it's the most optimized way to do it...

Updating data of an Image Plane Widget

I am working on a visualization of different vector fields.
For this purpose I am using the Mayavi Library in Python 2.7 (I think), to create a Image Plane Widget (IPW) and a slider to change the data of the vector field while the visualization is open, but my IPW won't change.
It works if I render the IPW new each time the slider is changed, but that's not what I want.
Is there a way to change the data of an IPW while the program is running without rendering a new Plane each time?
I have written following code:
import numpy as np
from mayavi import mlab
from matplotlib.scale import scale_factory
from traits.api import HasTraits, Range, Instance, Array, \
on_trait_change
from traitsui.api import View, Item, Group
from mayavi.core.pipeline_base import PipelineBase
from mayavi.core.ui.api import MayaviScene, SceneEditor, \
MlabSceneModel
class Modell(HasTraits):
p = Array
n = Range(0, 9, 5)
#p is a 4 dimensional Array p[10][20][20][20]
scene = Instance(MlabSceneModel, ())
plot = Instance(PipelineBase)
#on_trait_change('n,scene.activated')
def update_plot(self):
self.src = mlab.pipeline.scalar_field(self.p[self.n])
if self.plot is None:
self.plot = self.scene.mlab.pipeline.image_plane_widget(self.src,
plane_orientation='z_axes',
slice_index=10,
vmin=0, vmax=120)
else:
'''here should be the update function, i tried using mlab_source.set(self.src=self.src)
and all variations of expressions in the brackets but nothing worked.
I also searched for functions in IPW itself but didn't find something that could help me.'''
#The layout of the dialog created
view = View(Item('scene', editor=SceneEditor(scene_class=MayaviScene),
height=400, width=400, show_label=False),
Group('_', 'n'),
resizable=True,
)
my_model = Modell(p=p)
my_model.configure_traits()
I tried updating the pipeline and updating the data with self.plot.update_pipeline() and self.plot.update_data() but this doesn't work either.

Ok I found the solution for my problem, the trick is to change the data directly through the pipeline. So in my code I just have to set the following command into the else segment:
self.plot.parent.parent.scalar_data = self.p[self.n]

Time dependent data in Mayavi

Assuming I have a 4d numpy array like this: my_array[x,y,z,t].
Is there a simple way to load the whole array into Mayavi, and simply selecting the t I want to investigate for?
I know that it is possible to animate the data, but I would like to rotate my figure "on the go".

It is possible to set up a dialogue with a input box in which you can set t.
You have to use the traits.api, it could look like this:
from traits.api import HasTraits, Int, Instance, on_trait_change
from traitsui.api import View, Item, Group
from mayavi.core.ui.api import SceneEditor, MlabSceneModel, MayaviScene
class Data_plot(HasTraits):
a = my_array
t = Int(0)
scene = Instance(MlabSceneModel, ())
plot = Instance(PipelineBase)
#on_trait_change('scene.activated')
def show_plot(self):
self.plot = something(self.a[self.t]) #something can be surf or mesh or other
#on_trait_change('t')
def update_plot(self):
self.plot.parent.parent.scalar_data = self.a[self.t] #Change the data
view = View(Item('scene', editor=SceneEditor(scene_class=MayaviScene),
show_label=False),
Group('_', 't'),
resizable=True,
)
my_plot = Data_plot(a=my_array)
my_plot.configure_traits()
You can also set up a slider with the command Range instead of Int if you prefer this.

Updating Points of TVTK Dataset in MayaVI (without flushing pipeline)

I want to use MayaVI for visualization of large simulation data, saved as a VTKUnstructuredGrid (or here TVTK Unstructured Grid). After loading the Grid, I want to quickly update the grid points using numpy arrays, without changing anything else in the model.
So far I update the points and then call the modified()-method, which flushes the complete pipeline and thus slows down the visualization a lot. My question is now: Is there any chance to update the points in a VTKDataset without reloading the whole pipeline?
I am doing the visualization using Traits; simplified my code looks like:
import numpy as np
from enthought.traits.api import HasTraits, Range, Instance, on_trait_change
from enthought.traits.ui.api import View, Item, HGroup, HSplit, VSplit
from enthought.tvtk.pyface.scene_editor import SceneEditor
from enthought.mayavi.tools.mlab_scene_model import MlabSceneModel
from enthought.mayavi.core.ui.mayavi_scene import MayaviScene
from enthought.mayavi import mlab
from enthought.tvtk.api import tvtk
from enthought.mayavi.modules.surface import Surface
from enthought.tvtk.pyface.scene_editor import SceneEditor
class Visu(HasTraits):
timestep = Range(50,100,50)
pts = tvtk.Points()
ugrid = tvtk.UnstructuredGrid()
scene = Instance(MlabSceneModel, ())
view = View(Item('scene', editor=SceneEditor(scene_class=MayaviScene), height=250, width=300, show_label=True),HGroup('_', 'timestep'), resizable=True )
#on_trait_change('scene.activated')
def ini(self):
filename = 'original3dplot'
reader = tvtk.LSDynaReader(file_name = filename)
reader.update()
self.ugrid = reader.get_output()
self.surface = self.scene.mlab.pipeline.surface(self.ugrid)
#on_trait_change('timestep')
def update_visu(self):
update_coord = np.loadtxt('newcoordinates'+str(self.timestep))
self.pts.from_array(update_coord)
self.ugrid.points = self.pts
self.ugrid.modified()
visualization = Visu()
visualization.configure_traits()