How to create visual profiler? - python

I would like to visualize some statistics about a program's running time in Python. Unfortunately this program has written in Fortran and C++, and I cannot use any language specific visualization tool. From the output of the program I have created a class including the start time and the end time of different functions, and subroutines. For example:
class MyProgramRuntime:
def __init__(self):
self.foo1CallTime
self.foo1EndTime
self.foo2CallTime
self.foo2EndTime
I would like to get a similar plot like on the picture on the following link: Picture

I am not sure if is this the best solution, but it works really nice for me. I have used the matplotlib.patches lib to plot as long rectangles as the actual function/subroutine runned in seconds. The variables in the class are float variables, I got them by extracting from the actual datetime the datetime of the program's beginning with datetime.timedelta.total_seconds() . The plotting looks like the following:
import matplotlib.pyplot as plt
import matplotlib.patches as patches
def plotRunningTimes(mpr): #mpr is an instance of the MyProgramRuntime class
fig = plt.figure()
ax = plt.subplot(111)
ax.add_patch( patches.Rectangle( (mpr.foo1CallTime, 0.1),
mpr.foo1EndTime-mpr.foo1Calltime,
0.1) )

Related

Python: Why does this command work when I didn't import command

I'm writing a Python GUI for the first time to plot some data, and have imported the following modules/commands to do so
from tkinter import *
from tkinter.filedialog import askopenfilename
from tkinter.ttk import *
import matplotlib
matplotlib.use('TkAgg')
from matplotlib.cm import get_cmap
from matplotlib.pyplot import figure
from scipy.interpolate import griddata
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg
Searching the internet, I found some code that worked to get my plot embedded in the GUI:
fig = figure()
fig.add_subplot(111).pcolormesh(X, Y, Z, cmap = get_cmap('BuPu'))
canvas = FigureCanvasTkAgg(fig, master = window) # A tk.DrawingArea.
canvas.draw()
canvas.get_tk_widget().pack(side = TOP, fill = BOTH, expand = 1)
My question is why does pcolormesh work in the second line above when I didn't import that command from matplotlib.pyplot? I'm new to Python, and I'm sure I'm missing something, but if I wanted to use the pcolormesh command in the past, I had to import it. Thanks.
Your function calls created objects and those objects have methods. You don't need to import those methods as they are attached directly to the object. They are different from importing a function as you don't have direct access to those methods: they can only be accessed from the object.
when you set fig = figure(), you created an object. If you print(repr(fig)) you'll see something like this.
'<matplotlib.figure.Figure object at 0x000000000784F208>'
so fig is a Figure object. This objects comes with some methods and properties attached to it. You can get a list of all the methods using dir(fig) (it's a long list, so I won't print it here).
fig.subplot(111) calls the subplot method attached to your fig object. It returns a new object (an AxesSubplot). That object has a method attached to it called pcolormesh. If you want to play around with that object, you can do this.
ax = fig.subplot(111)
ax.pcolormesh(X, Y, Z, cmap = get_cmap('BuPu'))
print(repr(ax))
In addition to doing your stuff, this will print <matplotlib.axes._subplots.AxesSubplot at 0x8996f28>.
I remember being similarly confused by matplot.pyplot when I first started out because you can often do nearly identical things with a function (matplotlib.pyplot.pcolomesh) and as a method of an object (in the above example, ax.pcolormesh). They mention it in the API explanation here and here.

Matplotlib is slow when plotting many graphs on a single plot

As the title says when I try to graph a bunch of graphs, it takes a very long time. For example if I try to plot a silly example like this 10000 times:
n=10000
numbers = []
for i in range(n):
numbers.append(i)
for i in range(n):
plt.plot(numbers)
plt.show()
It will take about a minute to show the plot.
I know doing this will make it faster
import matplotlib
matplotlib.use('GTKAgg')
But is there any other way to make plotting a bunch of graphs faster? Any suggestions would be much appreciated, thanks!
You could do a dynamic plot with plt.ion()
example:
import matplotlib.pyplot as plt
import numpy as np
fig = plt.figure()
su = fig.add_subplot(111)
su.axis([-1, 1, -1, 1])
plt.ion() # allows for dynamic plot
for i in range(1000):
y = np.random.random()
su.scatter(y**2, y)
plt.pause(0.4)
while True:
plt.pause(0.05)
This allows you to see points as they come up on your graph.
Is this what you want?
EDIT:
Maybe you could try using matplotlib.pyplot's savefig functionality
https://matplotlib.org/devdocs/api/_as_gen/matplotlib.pyplot.savefig.html
You could have your computer save all of your figures to separate png files. Then you can look at the pictures any time. This method requires minimal time on your part, just let the program run in the background for a bit while it makes the pngs, and you can view them without having to regenerate them at any time.

Plot semilogx with matplotlib then convert it into Bokeh

I plot a figure containing several curves using matplotlib and then try to convert it into bokeh:
import numpy as np
import matplotlib.pyplot as plt
from bokeh import mpl
from bokeh.plotting import show, output_file
num_plots = 6
colormap = plt.cm.gist_ncar
time = np.random.random_sample((300, 6))
s_strain = np.random.random_sample((300, 6))
def time_s_strain_bokeh(num_plots, colormap, time, s_strain):
plt.gca().set_color_cycle([colormap(i) for i in np.linspace(0, 0.9, num_plots)])
plt.figure(2)
for i in range(0, num_plots):
plt.plot(time[:,i], s_strain[:,i])
plt.grid(True)
# save it to bokeh
output_file('anywhere.html')
show(mpl.to_bokeh())
time_s_strain_bokeh(num_plots, colormap, time, s_strain)
it works fine. However, I want to have a semilogx plot. When I change plt.plot in the "for" loop into plt.semilogx, I have the following error:
UnboundLocalError: local variable 'laxis' referenced before assignment
What can I do to change the x-axis onto log scale?
I'm with the same issue! 1/2 of the solution is this (supose my data is in a Pandas dataframe called pd):
pd.plot(x='my_x_variable', y='my_y_variable)
p = mpl.to_bokeh()
p.x_mapper_type='log' # I found this property with p.properties_with_values()
show(p)
I edited this answare because I just found part 2/2 of the solution:
When I use just the code above, the plot is semilog (ok!), but the x axis is flipped (mirrored)!!!
The solution I found is explicitly redefine xlim:
p.x_range.start=0.007 # supose pd['my_x_variable'] starts at 0.007
p.x_range.end=0.17 # supose pd['my_x_variable'] ends at 0.17
With this my plot became identical with the matplotlib original plot. The final code looks like:
pd.plot(x='my_x_variable', y='my_y_variable)
p = mpl.to_bokeh()
p.x_mapper_type='log'
p.x_range.start= pd['my_x_variable'].iloc[1] # numpy start at 0, take care!
p.x_range.end= pd['my_x_variable'].iloc[-1]
show(p)
As of Bokeh 0.12, partial and incomplete MPL compatibility is provided by the third party mplexporter library, which now appears to be unmaintained. Full (or at least, much more complete) MPL compat support will not happen until the MPL team implements MEP 25. However, implementing MEP 25 is an MPL project task, and the timeline/schedule is entirely outside of the control of the Bokeh project.
The existing MPL compat based on mplexporter is provided "as-is" in case it is useful in the subset of simple situations that it currently works for. My suggestion is to use native Bokeh APIs directly for anything of even moderate complexity.
You can find an example of a semilog plot created using Bokeh APIs here:
http://docs.bokeh.org/en/latest/docs/user_guide/plotting.html#log-scale-axes

Drawing obtuse circular arc with arrowhead in matplotlib

I am new to Python, though I do have previous programming experience. My current interest is to generate good quality schematics (and maybe later on also data graphs).
I need to draw a circlar arc with an arrowhead at the end point. I have sieved through many posts and tried my hand at a few examples, but it seems I still miss some basic knowledge. So I need something like this:
http://i.stack.imgur.com/Z5V86.png
I was experimenting with the example below, trying to initially plot a circular arc, but even this does not work. I am using matplotlib.patches but I am open to other suggestions for what I need.
# -*- coding: utf-8 -*-
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.patches import Ellipse, Arc
from matplotlib import cm, colors, patches
from math import pi
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ellipse = Arc([2.5,2.5],1,1,0,0,pi,color='green', linewidth='0.5')
ax.add_patch(ellipse)
plt.show()
Well,
if you want an angle of pi, pass 180.
linewidth must be a scalar. pass .5 and not '.5' as linewidth parameter
you need to change the limits of the axes, or you won't see the arc. Add something like ax.set_xlim(1.5,3.1) and ax.set_ylim(2.4,3.7).
You may also want to opt for another strategy, something like
ax.plot([.5],[.5],marker=r'$\circlearrowleft$',ms=100)

How to plot a queue of points?

I have the following code.
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
plt.ion()
plt.show()
mapping = defaultdict(partial(deque, maxlen=10))
My mapping structure contains a queues with x, y, z values. I want to plot only the point in queue so as the queue changes the plot should also change. How would I do that?
Note: This is challenging because the queue is of a bounded size and the plot has to reflect only whats in the queue.
The hacky way to do this is:
ln, = ax.plot(x,y,z)
# some code that updates the x,y,z values -> new_x,new_y,new_z
ln.remove
ln, = ax.plot(new_x,new_y,new_z)
plt.draw()
which removes the line with the old data and adds one with the new data.
If you are willing to write code that depends on the internals of matplotlib (which is a bad idea as the internal will likely change under you), you can also do this by:
ln._verts3d = new_x,new_y,new_z
Patch here for added a function to do this: https://github.com/matplotlib/matplotlib/pull/1629
The animation module nicely wraps up many of the things needed to do animation (including a way to stream output directly to ffmpeg), and there is a very nice tutorial.
Line3D docs, Line2D docs

Categories